There appears to be some intuitive distinction between math results that are just “recombinations of known ideas” (possibly very sophisticated recombinations!), and math results that feel “truly novel”, and the mathematical consensus is apparently that none of AI math results so far are of the latter type.
I think the conclusion is correct here but the model is not quite right. Most (or all) ideas originate from previous concepts and information, passed through a variable amount of reprocessing and abstraction. People understood the concepts of slope and area before the invention of calculus. Of course calculus took plenty of “creativity” to find, which I believe should be judged by the degree of transformation. It appears that LLMs are pretty good at combining known mathematical techniques in straightforward “right-out-of-the-packaging” ways e.g. a property of object X satisfies the assumptions of theorem Y, so X is an example of conjecture Z (exactly what happened with non-sofic groups btw). These can still be surprises, since the LLM has access to such an incredibly large array of techniques! As far as I can tell, however, they haven’t demonstrated the ability to abstract ideas further to the extent of being considered “new perspectives”.
I don’t claim to understand what’s going on there, or why LLMs would be limited in this manner (whatever “this manner” even means). As bearish on LLMs as I am, I don’t see why RLVR wouldn’t suffice to let them “solve” formal mathematics. Yet, something’s up with that, I think.
RLVR for mathematics is not amenable to the same kind of exploration that you can do for board games like Chess and Go. In those cases you can basically harvest endless data from MCTS self-play, and thereby empirically discover winning patterns and new ideas that can be used at runtime. For math, the analogue would require the training process to somehow harvest all of the ideas that the model would need at inference to solve whatever research problem it might be posed with—essentially it would need to do the operative part of that original research ahead of time. This does not seems remotely plausible to me. Indeed RLVR for LLMs appears to be limited to improving sampling of known ideative pathways that appear in the pretraining. This last point was discussed really nicely by both Steven Byrnes and Beren Millidgerecently, with some links to the relevant papers.
My view is that LLMs seem to be restricted in certain important aspects of math and other formal subjects by a lack of “conceptual fluidity”. I think this is the same blocker that you’ve described quite cogently in your expectations for LLM progress in hard-to-verify research. Actually I don’t think that the formal character of math makes that much of a difference in this regard! Sure, it makes it easier for the models to practice, but the most important driver of progress in math is still the development of qualitative concepts, and we don’t seem to have a way of generating such concepts in training (except for massive-scale empirics in the case of board games).
I think the conclusion is correct here but the model is not quite right. Most (or all) ideas originate from previous concepts and information, passed through a variable amount of reprocessing and abstraction. People understood the concepts of slope and area before the invention of calculus. Of course calculus took plenty of “creativity” to find, which I believe should be judged by the degree of transformation. It appears that LLMs are pretty good at combining known mathematical techniques in straightforward “right-out-of-the-packaging” ways e.g. a property of object X satisfies the assumptions of theorem Y, so X is an example of conjecture Z (exactly what happened with non-sofic groups btw). These can still be surprises, since the LLM has access to such an incredibly large array of techniques! As far as I can tell, however, they haven’t demonstrated the ability to abstract ideas further to the extent of being considered “new perspectives”.
RLVR for mathematics is not amenable to the same kind of exploration that you can do for board games like Chess and Go. In those cases you can basically harvest endless data from MCTS self-play, and thereby empirically discover winning patterns and new ideas that can be used at runtime. For math, the analogue would require the training process to somehow harvest all of the ideas that the model would need at inference to solve whatever research problem it might be posed with—essentially it would need to do the operative part of that original research ahead of time. This does not seems remotely plausible to me. Indeed RLVR for LLMs appears to be limited to improving sampling of known ideative pathways that appear in the pretraining. This last point was discussed really nicely by both Steven Byrnes and Beren Millidge recently, with some links to the relevant papers.
My view is that LLMs seem to be restricted in certain important aspects of math and other formal subjects by a lack of “conceptual fluidity”. I think this is the same blocker that you’ve described quite cogently in your expectations for LLM progress in hard-to-verify research. Actually I don’t think that the formal character of math makes that much of a difference in this regard! Sure, it makes it easier for the models to practice, but the most important driver of progress in math is still the development of qualitative concepts, and we don’t seem to have a way of generating such concepts in training (except for massive-scale empirics in the case of board games).