There are many types of math, with differing sorts of value, but I can say a little about the sort of math I find moving.
I agree with you. For the most part, applied souls dream up their advances and make them without relying on the mathematical machine. They invent the math they need to describe their ideas. Or perhaps they use a little of the pure mathematician’s machine, but quickly develop it in ways that are more important to their work than the previous mathematical meanderings.
I think you underestimate the role of mathematics as the grand expositor. It is the tortoise that trails forever beyond the hare of applied science. It takes the insights of applications, of calculus for example, and digests them. It reworks them, understands them, connects them, rigorizes them.
The work of mathematics is not useful in your mind because a mathematician does not make a truly new applied advance. A mathematician invents and connects notations to ease the traversal, the learning, and most importantly the storage in working memory of past insights.
What is the purpose of a category? An operad? A type theory? A vector bundle? The digit 0? When these languages were introduced, it could always be claimed they were worthless because the old languages could express the same content as these new languages. But somehow the new language makes it easier to conceptualize and think about the old ideas; it increases the working human RAM.
And what of the poor student? He who must learn so many subjects is grateful when it is realized that many of those subjects are in fact the same: http://arxiv.org/abs/0903.0340 . Mathematics digests theories and rewrites them as branches of a common base. It makes it possible to learn more insights quickly and to communicate them to the next generation.
So young applied scientists, perhaps generations later, benefit by more compactly and elegantly understanding the insights of their forebearers. Then, the mathematician dreams, they are freer to envision the next great ideas: http://arxiv.org/abs/1109.0955
So why the mathematician’s focus on solving specific problems? Why so much energy to characterize finite groups? It is not that these problems are important. It is that they serve as testbeds for new languages, for new characterizations of old insights. The problems of pure math are invented as challenges to understand an old applied language, not to invent a new one.
I do not think the situation is as simple as you claim it to be. Consider that a category is more general than a monoid, but there are many interesting theorems about categories.
As far as foundations for mathematical logic go, any one interested in such things should be made aware of the recent invention of univalent type theory. This can be seen as a logic which is inherently higher-order, but it also has many other desirable properties. See for instance this recent blog post: http://golem.ph.utexas.edu/category/2013/01/from_set_theory_to_type_theory.html#more
That univalent type theory is only a few years old is a sign we are not close to fully understanding what foundational logic is most convenient. For example, one might hope for a fully directed homotopy type theory, which I don’t doubt will appear a few years down the line.