Cross-posted from Twitter; written for mathematicians.
About a year ago, I wrote that superhuman math AI will plausibly arrive in the late 2020s/early 2030s. It now looks like if anything, I was underestimating the pace of progress. In this post, I will try to say something about the sociological implications for mathematics.
I don’t want to focus too much on the predicament of the mathematical community today; instead, I’ll try to take a longer-term view, and say something about where I think human mathematics might eventually end up. For the sake of argument, I’ll assume that within the next decade or so, AI systems are developed that are vastly more capable than humans at every mathematical task (including proving theorems, building theories, and explaining mathematics to humans), and that there nevertheless remains a community of human mathematicians, alive and well, and with funding to pursue mathematics as they please. What would such a community be doing?
Of course, in this hypothetical, mathematicians are not pursuing mathematics for any productive purpose; that is handled by the machines. They are pursuing mathematics for its own sake. There is nothing at all new about this; it is the same purpose that Hardy famously defended in 1940 in A Mathematician’s Apology, declaring, “Judged by all practical standards, the value of my mathematical life is nil.” As a former set theorist, I can certainly say the same of my own work in that area, as I am sure would many others.
What, then, does the pursuit of mathematics for its own sake look like, in the AI era? There are many different activities that mathematicians might engage in, so this is largely a question about which of those I feel best embodies the spirit of mathematics as it is practiced today. I will go through a few different options in turn.
Mathematics as chess
One possibility is that humans will solve mathematical problems with known solutions. They might do this leisurely and collaboratively, or they might do it competitively, as in olympiads today. Perhaps there would be a “world championship of mathematics” lasting many days, with spectators observing competitors’ scratch work, interpreted with the help of AI and narrated by their favorite commentators. I call this “mathematics as chess”, since it resembles how the chess community has continued in spite of AI dominance.
If this were the most faithful remaining reflection of mathematics as it is practiced today, I think most mathematicians would find such an outcome disastrous.
Mathematics contests today are viewed as a way to train and encourage new talent; this is why they are almost always restricted by age or educational attainment. Mathematics itself is the activity these contests are viewed as preparation for. If as a response to AI progress there is an entirely new industry of widely-spectated math contests, there is no reason for mathematicians to begrudge this; but to claim that this is the spiritual successor of mathematics today is to completely misunderstand mathematical culture.
At its core, mathematics is about discovery. This is why mathematicians spend their time on unsolved problems, not solved ones, and why proper attribution is sacrosanct. Of course, mathematicians are rewarded with money and prestige, which are proximate motivations; but discovery is the cultural value that determines how these are distributed. Furthermore, mathematical values related to discovery can sometimes trump monetary rewards. This was famously demonstrated by Perelman, who turned down a $1m Millennium Prize, apparently because he felt that it should have been shared with Hamilton, whose work was important to his solution.
So no, mathematics is not chess.
Mathematics as an island
A second possibility is to reject non-human intelligence within mathematics entirely, and maintain a closed-off community to preserve mathematics as it was practiced prior to around 2023. Humans would continue to conduct research, solve problems that no-one else within the community had yet solved, and credit each other for being the first within the community to solve a problem.
As tempting as this may sound to some readers in the current moment, I suspect that such an approach will be taken by at most a small fringe of the mathematical community.
Firstly, there would be an inherent trade-off between the degree of separation with the outside world, and the degree of confidence one could have that others are playing by the same rules. Using AI assistance would be like doping in sport, except that even having a conversation with someone from the outside world could be viewed with suspicion. I find it hard to imagine such an environment providing sufficiently sustainable motivation to all but the most die-hard traditionalists.
But more than this, “mathematics as an island” has too much of an air of self-delusion about it. It amounts to pretending that machine intelligence does not exist, rather than coming to terms with it. I suspect that for most mathematicians, there would be too much dissonance with their truth-seeking values for them to dedicate their existence to such a path.
Mathematics as star-gazing
A third possibility is for humans to decide which problems to set AIs to work on. Of course, in this hypothetical world, humans have nothing to contribute when it comes to figuring out which sub-problems are the most promising, nor estimating the probability that a problem will be solved after spending a certain amount. So it will be more like choosing options from a menu. Perhaps we will have to debate whether we prefer a 10% chance of resolving the twin prime conjecture, or a 1% chance of resolving the Riemann hypothesis; some may relish this opportunity more than others.
However, there is a wrinkle: the results will be front-loaded. Superintelligent AI will not immediately solve all mathematical problems, for the same reason that it will not immediately prove that chess is a draw under perfect play: the search space is too large. But there will be diminishing returns to computation, and the relative growth rate of computational resources must eventually decline due to physical limits (in the very long run, resources can grow at most cubically with time, since that bounds how rapidly one can expand through 3-dimensional space). I’m not confident in this, but I suspect that a small relative increase in computation gives at best a small additional chance of solving any given problem, even if this amounts to a large increase in computation in absolute terms. So we should expect an initial flurry of excitement, but after a few generations, the proof of an old conjecture will be a rare moment of celebration.
One way around the decaying rate of AI-generated results is to continually come up with new problems to pose to AIs, or to have AIs generate entirely new theories autonomously. This will still be possible, because the search space is exponential. But the lowest-hanging fruit will soon be plucked, and I suspect that the trillionth most interesting theorem will involve enough arbitrary-seeming choices that the appeal will be similar to that of a puzzle from a puzzle hunt. I am sure that some people will amuse themselves by generating or designing puzzles for their Jupiter-brain to solve, but I don’t know if I would call that mathematics.
Regardless, “mathematics as star-gazing” is missing a second ingredient that is core to mathematics: human understanding. Many of the proofs produced by AIs may be possible for humans to understand, but unless we actually put in the work to understand them, I imagine most mathematicians would consider the proofs themselves to be meaningless trophies.
Mathematics as poetry
The final possibility I will consider is that humans will continue to engage with machine-generated proofs, but the primary activity of human mathematicians will be hermeneutics: selecting favorite results from the enormous AI-generated library of mathematics; digesting proofs (either of old results or of fresh “puzzles”), and presenting and explaining them to each other; re-proving known results by hand to advance their own understanding; and so on. The overall goal of these activities would be to advance the collective knowledge, understanding and appreciation of the frontiers of mathematics by humanity.
In such a world, people would write their own textbooks, proofs, and problems, perhaps with AI assistance, in order to share them with others. The obvious rejoinder to this suggestion is: why would humans be necessary to create such artifacts at all? The only possible answer to this question is: we value such artifacts by virtue of the fact that they were written (or selected) by a human. This is the same answer that must eventually justify all human art, literature, and poetry, hence “mathematics as poetry”.
Of course, such artifacts can be faked. But it will be like having an idle conversation with an old friend. You don’t worry about whether it was faked, because it has no economic value, and there was never any point in faking it. It exists only between the two of you, and each of you values it primarily for its authenticity.
Even in a world where AI can explain to us any result of our choosing better than any human, I still expect there to be a social aspect to human mathematics. Perhaps there will be a significant solitary aspect too, to cater to individual tastes; but I think most mathematicians care a significant amount about sharing ideas and experiences, and understanding things in common with their peers.
In my opinion, “mathematics as poetry” is the most likely outcome that at least resembles mathematics as it is practiced today. It sacrifices the primacy of human discovery, but at least replaces it by a creative process of mathematical expression, while keeping human understanding and interaction at the forefront.
Transitioning to such a world may be a process of considerable grief for many mathematicians, even if everything goes about as well as we can reasonably hope with AI. But for better or for worse, I think this may be the most palatable option on the menu.
Postscript
Although I have been treating superhuman math AI as a hypothetical, I actually think that this is eventually the most likely outcome. And if we restrict to purely formal problems such as theorem proving, I think superhuman math AI is likely to be here before the end of the decade. The progress of AI in mathematics has been extraordinarily rapid for the last 5 years, and there is little reason to think that the drivers of this progress (algorithms, computation and data) will stall in the near future. I say this as someone who was involved in the development of GSM8K, a mathematical benchmark from 2021 that was essentially obsolete by the end of 2023.
But without meaning to belittle what the mathematical community is going through, the future of human mathematics is not the most pressing challenge posed by AI in the near future, despite being close to my heart. It is currently unclear whether we will be able to remain in control of superhuman AI at all, and many of the concerning dynamics that were primarily hypothetical a few years ago are now playingoutin practice.
I am biased, but I think that ARC (where I work) is one of the most promising ways for mathematicians to contribute to AI safety. Other organizations that are hiring theorists include Resolution, PrinceInt and Simplex, and there are also training programs such as Iliad’s. To be honest, though, I don’t think any of us have exactly the right ideas yet, and my main hope is that someone with fresh eyes will spot something crucial that we are all currently missing.
If you are a mathematician who shares my view of where AI is headed, please take advantage of the additional foresight this grants you, and make the most of the last few years where human contributions to mathematics have practical as well as aesthetic value. I wish you the very best.
As a fellow former mathematician, I share your sense that “mathematics as poetry” seems like the most likely outcome, at least for pure maths and barring more fundamental AI-driven changes to the human condition (e.g., extinction, or humans being replaced by very different kinds of minds).
Rather, as a more explicit (and leading) form of the question, I prefer
How do mathematicians advance human understanding of mathematics?
This question brings to the fore something that is fundamental and pervasive: that what we are doing is finding ways for people to understand and think about mathematics.
The rapid advance of computers has helped dramatize this point, because computers and people are very different. For instance, when Appel and Haken completed a proof of the 4-color map theorem using a massive automatic computation, it evoked much controversy. I interpret the controversy as having little to do with doubt people had as to the veracity of the theorem or the correctness of the proof. Rather, it reflected a continuing desire for human understanding of a proof, in addition to knowledge that the theorem is true. On a more everyday level, it is common for people first starting to grapple with computers to make large-scale computations of things they might have done on a smaller scale by hand. They might print out a table of the first 10,000 primes, only to find that their printout isn’t something they really wanted after all. They discover by this kind of experience that what they really want is usually not some collection of “answers”—what they want is understanding.
The product of mathematics is clarity and understanding. Not theorems, by themselves. [...]
I think of mathematics as having a large component of psychology, because of its strong dependence on human minds. Dehumanized mathematics would be more like computer code, which is very different. [...]
In short, mathematics only exists in a living community of mathematicians that spreads understanding and breaths life into ideas both old and new. The real satisfaction from mathematics is in learning from others and sharing with others. All of us have clear understanding of a few things and murky concepts of many more. There is no way to run out of ideas in need of clarification. The question of who is the first person to ever set foot on some square meter of land is really secondary. Revolutionary change does matter, but revolutions are few, and they are not self-sustaining—they depend very heavily on the community of mathematicians.
To be clear, I do think that AI comes with bigger challenges for, and will have bigger impacts on, the maths community than the kind of ‘automatic computation’ or ‘computer code’ Thurston mentions in these quotes because LLMs can do a much wider range of maths tasks, including communicating about maths in many of the ways human mathematicians do, and so their use likely won’t be limited to narrow use cases outside an exclusively human-driven core activity of generating and transmitting ‘understanding’ to each other. And so in particular I agree with the broad picture you outline where AI use is a pervasive feature of a “mathematics as poetry” world.
My view (as an applied mathematician who’s been grappling with their future for some years now) is a bit in your poetry direction, but different. First of all, I am not entirely sure yet that humans will quickly have literally zero “math-economy-value” at all. I’d actually consider it quite plausible right now that in ten years, “human-in-the-loop-systems” can still significantly outperform pure AI systems at certain long term math tasks.
However, my plan for the possibility of literally zero math-economy-value of humans (and humans still being humans-as-today and existing, and a somehow stable society, etc) in a few years is as follows: I want to still understand key technical mathematical arguments which are relevant for modern technology and which lie within my field. I want to understand them as well as possible under certain constraints (for instance, without taking into account explanations from models which may have agendas and try to influence people). I recognize that I may be worse than AI in every measurable way at that. But I still think it may have some societal value (even though very little) to have humans (instead of just AI) which push towards this kind of deep mathematical understanding when thinking of trustworthiness, etc. Perhaps this is all just cope though, not sure. At least it also sounds more interesting than becoming a poet. Maybe rather “Mathematics as eternal study”?
I agree with all of this. One thing that scares me about the “mathematics as poetry” world is the sociological effect of human beings no longer rolling up their sleeves and really doing mathematics as it is understood today. Mathematical culture is very special; despite some exceptions it is an unusually open, honest, kind world. I worry that this is driven by the objective and verifiable nature of mathematical progress, combined with a high barrier to entry and a real place for good taste. If the human part of mathematics becomes entirely subjective and aesthetic, I worry that it could turn into a popularity contest that would suck the joy out of the thing itself.
I feel this is like saying: being a ship captain would suck all the joy out of being a ship. I mean, they are different experiences, but you still get to see places no else would.
Agree with two caveats that make me slightly more optimistic about “mathematics as stargazing”
Some conjectures are easy to understand, natural, and extremely difficult (eg Collatz, or P vs NP, or finding BB(10)). We should expect this to continue such that at least some problems current mathematicians can understand will remain unsolved by Jupiter brains.
AIs will have better inference scaling than humans, which means returns don’t diminish as fast.
The future of human mathematics
Cross-posted from Twitter; written for mathematicians.
About a year ago, I wrote that superhuman math AI will plausibly arrive in the late 2020s/early 2030s. It now looks like if anything, I was underestimating the pace of progress. In this post, I will try to say something about the sociological implications for mathematics.
I don’t want to focus too much on the predicament of the mathematical community today; instead, I’ll try to take a longer-term view, and say something about where I think human mathematics might eventually end up. For the sake of argument, I’ll assume that within the next decade or so, AI systems are developed that are vastly more capable than humans at every mathematical task (including proving theorems, building theories, and explaining mathematics to humans), and that there nevertheless remains a community of human mathematicians, alive and well, and with funding to pursue mathematics as they please. What would such a community be doing?
Of course, in this hypothetical, mathematicians are not pursuing mathematics for any productive purpose; that is handled by the machines. They are pursuing mathematics for its own sake. There is nothing at all new about this; it is the same purpose that Hardy famously defended in 1940 in A Mathematician’s Apology, declaring, “Judged by all practical standards, the value of my mathematical life is nil.” As a former set theorist, I can certainly say the same of my own work in that area, as I am sure would many others.
What, then, does the pursuit of mathematics for its own sake look like, in the AI era? There are many different activities that mathematicians might engage in, so this is largely a question about which of those I feel best embodies the spirit of mathematics as it is practiced today. I will go through a few different options in turn.
Mathematics as chess
One possibility is that humans will solve mathematical problems with known solutions. They might do this leisurely and collaboratively, or they might do it competitively, as in olympiads today. Perhaps there would be a “world championship of mathematics” lasting many days, with spectators observing competitors’ scratch work, interpreted with the help of AI and narrated by their favorite commentators. I call this “mathematics as chess”, since it resembles how the chess community has continued in spite of AI dominance.
If this were the most faithful remaining reflection of mathematics as it is practiced today, I think most mathematicians would find such an outcome disastrous.
Mathematics contests today are viewed as a way to train and encourage new talent; this is why they are almost always restricted by age or educational attainment. Mathematics itself is the activity these contests are viewed as preparation for. If as a response to AI progress there is an entirely new industry of widely-spectated math contests, there is no reason for mathematicians to begrudge this; but to claim that this is the spiritual successor of mathematics today is to completely misunderstand mathematical culture.
At its core, mathematics is about discovery. This is why mathematicians spend their time on unsolved problems, not solved ones, and why proper attribution is sacrosanct. Of course, mathematicians are rewarded with money and prestige, which are proximate motivations; but discovery is the cultural value that determines how these are distributed. Furthermore, mathematical values related to discovery can sometimes trump monetary rewards. This was famously demonstrated by Perelman, who turned down a $1m Millennium Prize, apparently because he felt that it should have been shared with Hamilton, whose work was important to his solution.
So no, mathematics is not chess.
Mathematics as an island
A second possibility is to reject non-human intelligence within mathematics entirely, and maintain a closed-off community to preserve mathematics as it was practiced prior to around 2023. Humans would continue to conduct research, solve problems that no-one else within the community had yet solved, and credit each other for being the first within the community to solve a problem.
As tempting as this may sound to some readers in the current moment, I suspect that such an approach will be taken by at most a small fringe of the mathematical community.
Firstly, there would be an inherent trade-off between the degree of separation with the outside world, and the degree of confidence one could have that others are playing by the same rules. Using AI assistance would be like doping in sport, except that even having a conversation with someone from the outside world could be viewed with suspicion. I find it hard to imagine such an environment providing sufficiently sustainable motivation to all but the most die-hard traditionalists.
But more than this, “mathematics as an island” has too much of an air of self-delusion about it. It amounts to pretending that machine intelligence does not exist, rather than coming to terms with it. I suspect that for most mathematicians, there would be too much dissonance with their truth-seeking values for them to dedicate their existence to such a path.
Mathematics as star-gazing
A third possibility is for humans to decide which problems to set AIs to work on. Of course, in this hypothetical world, humans have nothing to contribute when it comes to figuring out which sub-problems are the most promising, nor estimating the probability that a problem will be solved after spending a certain amount. So it will be more like choosing options from a menu. Perhaps we will have to debate whether we prefer a 10% chance of resolving the twin prime conjecture, or a 1% chance of resolving the Riemann hypothesis; some may relish this opportunity more than others.
However, there is a wrinkle: the results will be front-loaded. Superintelligent AI will not immediately solve all mathematical problems, for the same reason that it will not immediately prove that chess is a draw under perfect play: the search space is too large. But there will be diminishing returns to computation, and the relative growth rate of computational resources must eventually decline due to physical limits (in the very long run, resources can grow at most cubically with time, since that bounds how rapidly one can expand through 3-dimensional space). I’m not confident in this, but I suspect that a small relative increase in computation gives at best a small additional chance of solving any given problem, even if this amounts to a large increase in computation in absolute terms. So we should expect an initial flurry of excitement, but after a few generations, the proof of an old conjecture will be a rare moment of celebration.
One way around the decaying rate of AI-generated results is to continually come up with new problems to pose to AIs, or to have AIs generate entirely new theories autonomously. This will still be possible, because the search space is exponential. But the lowest-hanging fruit will soon be plucked, and I suspect that the trillionth most interesting theorem will involve enough arbitrary-seeming choices that the appeal will be similar to that of a puzzle from a puzzle hunt. I am sure that some people will amuse themselves by generating or designing puzzles for their Jupiter-brain to solve, but I don’t know if I would call that mathematics.
Regardless, “mathematics as star-gazing” is missing a second ingredient that is core to mathematics: human understanding. Many of the proofs produced by AIs may be possible for humans to understand, but unless we actually put in the work to understand them, I imagine most mathematicians would consider the proofs themselves to be meaningless trophies.
Mathematics as poetry
The final possibility I will consider is that humans will continue to engage with machine-generated proofs, but the primary activity of human mathematicians will be hermeneutics: selecting favorite results from the enormous AI-generated library of mathematics; digesting proofs (either of old results or of fresh “puzzles”), and presenting and explaining them to each other; re-proving known results by hand to advance their own understanding; and so on. The overall goal of these activities would be to advance the collective knowledge, understanding and appreciation of the frontiers of mathematics by humanity.
In such a world, people would write their own textbooks, proofs, and problems, perhaps with AI assistance, in order to share them with others. The obvious rejoinder to this suggestion is: why would humans be necessary to create such artifacts at all? The only possible answer to this question is: we value such artifacts by virtue of the fact that they were written (or selected) by a human. This is the same answer that must eventually justify all human art, literature, and poetry, hence “mathematics as poetry”.
Of course, such artifacts can be faked. But it will be like having an idle conversation with an old friend. You don’t worry about whether it was faked, because it has no economic value, and there was never any point in faking it. It exists only between the two of you, and each of you values it primarily for its authenticity.
Even in a world where AI can explain to us any result of our choosing better than any human, I still expect there to be a social aspect to human mathematics. Perhaps there will be a significant solitary aspect too, to cater to individual tastes; but I think most mathematicians care a significant amount about sharing ideas and experiences, and understanding things in common with their peers.
In my opinion, “mathematics as poetry” is the most likely outcome that at least resembles mathematics as it is practiced today. It sacrifices the primacy of human discovery, but at least replaces it by a creative process of mathematical expression, while keeping human understanding and interaction at the forefront.
Transitioning to such a world may be a process of considerable grief for many mathematicians, even if everything goes about as well as we can reasonably hope with AI. But for better or for worse, I think this may be the most palatable option on the menu.
Postscript
Although I have been treating superhuman math AI as a hypothetical, I actually think that this is eventually the most likely outcome. And if we restrict to purely formal problems such as theorem proving, I think superhuman math AI is likely to be here before the end of the decade. The progress of AI in mathematics has been extraordinarily rapid for the last 5 years, and there is little reason to think that the drivers of this progress (algorithms, computation and data) will stall in the near future. I say this as someone who was involved in the development of GSM8K, a mathematical benchmark from 2021 that was essentially obsolete by the end of 2023.
But without meaning to belittle what the mathematical community is going through, the future of human mathematics is not the most pressing challenge posed by AI in the near future, despite being close to my heart. It is currently unclear whether we will be able to remain in control of superhuman AI at all, and many of the concerning dynamics that were primarily hypothetical a few years ago are now playing out in practice.
I am biased, but I think that ARC (where I work) is one of the most promising ways for mathematicians to contribute to AI safety. Other organizations that are hiring theorists include Resolution, PrinceInt and Simplex, and there are also training programs such as Iliad’s. To be honest, though, I don’t think any of us have exactly the right ideas yet, and my main hope is that someone with fresh eyes will spot something crucial that we are all currently missing.
If you are a mathematician who shares my view of where AI is headed, please take advantage of the additional foresight this grants you, and make the most of the last few years where human contributions to mathematics have practical as well as aesthetic value. I wish you the very best.
Thanks to Kevin Ren for a number of comments.
As a fellow former mathematician, I share your sense that “mathematics as poetry” seems like the most likely outcome, at least for pure maths and barring more fundamental AI-driven changes to the human condition (e.g., extinction, or humans being replaced by very different kinds of minds).
Some of your discussion reminded me of Field Medalist Bill Thurston’s 1994 note “On Proof and Progress in Mathematics”:
Similarly, Thurston wrote on MathOverflow:
To be clear, I do think that AI comes with bigger challenges for, and will have bigger impacts on, the maths community than the kind of ‘automatic computation’ or ‘computer code’ Thurston mentions in these quotes because LLMs can do a much wider range of maths tasks, including communicating about maths in many of the ways human mathematicians do, and so their use likely won’t be limited to narrow use cases outside an exclusively human-driven core activity of generating and transmitting ‘understanding’ to each other. And so in particular I agree with the broad picture you outline where AI use is a pervasive feature of a “mathematics as poetry” world.
My view (as an applied mathematician who’s been grappling with their future for some years now) is a bit in your poetry direction, but different. First of all, I am not entirely sure yet that humans will quickly have literally zero “math-economy-value” at all. I’d actually consider it quite plausible right now that in ten years, “human-in-the-loop-systems” can still significantly outperform pure AI systems at certain long term math tasks.
However, my plan for the possibility of literally zero math-economy-value of humans (and humans still being humans-as-today and existing, and a somehow stable society, etc) in a few years is as follows: I want to still understand key technical mathematical arguments which are relevant for modern technology and which lie within my field. I want to understand them as well as possible under certain constraints (for instance, without taking into account explanations from models which may have agendas and try to influence people). I recognize that I may be worse than AI in every measurable way at that. But I still think it may have some societal value (even though very little) to have humans (instead of just AI) which push towards this kind of deep mathematical understanding when thinking of trustworthiness, etc. Perhaps this is all just cope though, not sure. At least it also sounds more interesting than becoming a poet. Maybe rather “Mathematics as eternal study”?
I agree with all of this. One thing that scares me about the “mathematics as poetry” world is the sociological effect of human beings no longer rolling up their sleeves and really doing mathematics as it is understood today. Mathematical culture is very special; despite some exceptions it is an unusually open, honest, kind world. I worry that this is driven by the objective and verifiable nature of mathematical progress, combined with a high barrier to entry and a real place for good taste. If the human part of mathematics becomes entirely subjective and aesthetic, I worry that it could turn into a popularity contest that would suck the joy out of the thing itself.
I feel this is like saying: being a ship captain would suck all the joy out of being a ship. I mean, they are different experiences, but you still get to see places no else would.
Agree with two caveats that make me slightly more optimistic about “mathematics as stargazing”
Some conjectures are easy to understand, natural, and extremely difficult (eg Collatz, or P vs NP, or finding BB(10)). We should expect this to continue such that at least some problems current mathematicians can understand will remain unsolved by Jupiter brains.
AIs will have better inference scaling than humans, which means returns don’t diminish as fast.
Maybe AI science as venture capital. People with money bet on which new area of science to spend compute on.