I don’t think about my self-study as “know a bunch of math things”. Rather, it’s a) continually improve at mathematical reasoning, and b) accumulate a bunch of problem-solving strategies and ways-of-looking-at-the-world. I can
consider the analytic properties of a real polynomial (how quickly does the output change with the input? all polynomials are analytic. polynomial approximation theorem)
put on my group theory hat (what kinds of operations does the space of polynomials P admit?)
consider it from a linear algebraic standpoint (P is secretly R∞)
put on a number-theoretic hat (consider the distributional behavior of integer polynomials in a modular context)
think about how polynomials might be used in ML (hypothesis classes, complexity-of-fit, what’s the search space behavior like for SGD if the model space is just the coefficients for degree-50 polynomials?)
think about complexity theory (the nice closure properties of polynomial composition)
or even consider the computability properties of polynomials (this is a bit of a stretch, but… for every decision problem containing a finite amount of YES answers, there exists a polynomial with roots on the base-ten encodings of those YES inputs).
Then, when I actually need to do stuff with polynomials for my research, I can see how the difference of returns from two policies can be represented by a polynomial in the agent’s discount rate, which is a nice result (lemma 32). Insights build on insights.
So, I’m not trying to memorize everything. Leafing through Linear Algebra Done Right, I don’t remember much about what self-adjointness means, or Jordan normal form, or whatever. However, I don’t think that really matters. If I need to use the extraneous stuff, I know it exists, and could just pick it back up.
I am, however, able to regenerate fundamental things I actually use / run into in later studies. I have a mental habit of making myself regenerate random claims in every proof I consider. If we’re relying on the commutativity of addition on the reals, I reflexively supply a proof of that property. I came up with: use the Cauchy sequence limit notion of reals, then rely on the commutativity of rationals under addition for each member of the sequence.
It’s not like I’ve perfectly retained everything I need. I can tell I’m a little rusty on some things. However, I never do conscious reviewing (beyond building on past insights through further study, using the insights in my professional research, and rederiving things periodically). I have empirical feedback that this works pretty well. My PhD qualifier exam involved a matrix analysis question; without having even taken the class, I was able to get the right answer by reasoning using the skills and knowledge I got from self-study.
ETA: FWIW, when I talk with math undergrads at my university about areas we’ve both studied / solve problems with them, my impression is that my comprehension is often better.