He also seems to be a bit of a doomer. :O Might be worth some ppl reaching out to.
Q: Have you discussed the intersection of AI and math much with other mathematicians?
A: Yeah, I have a lot of conversations. I’m pretty worried about AI in general — in math, and otherwise.
Q: Worried how?
A: I think there’s a good chance AI will lead to human extinction. I also think, separately from that, that AI will be better than mathematicians at math very soon.
Some people are advocating for a total pause in AI. I don’t think that’s wrong, I’d support that. But I doubt it will happen. So given that AI is moving forward, it’s important that we stay abreast of what it can do and how to integrate it into our lives. If pushing back against it leads to not learning how to integrate with this new… you can call it a species — call it whatever you want — on the planet, then I think there’s a bigger chance we might be left behind. That’s why I think it’s important to stay connected.
I think there’s a lot of “cope” going on among mathematicians, and a lot of overconfidence about the limitations of AI. It seems like most people are looking at the fact that LLMs are not currently as good as mathematicians at math and concluding for various reasons that it can’t be and never will be as good. Or they’re assuming that it will only be like an assistant, where it proves the simple lemmas, but that it won’t prove the major stuff. And I basically think that that’s all incorrect.
I think it’s going to make being a mathematician not a profession anymore. I don’t think mathematics is unique in that. I do think it might come sooner in math than in other places. Because math is much fewer soft skills; it’s much easier to train yourself. Once you have Lean and the LLMs integrated you can just go — go experiment, learn, prove stuff.
And oh, beyond that, in the first paragraph of the piece:
It also causes him worry. A sixth, nonmathematical, submission he made to the arXiv last year is titled “A Taxonomy of Omnicidal Futures Involving Artificial Intelligence.” This paper is coauthored with a longtime friend from math camp, the mathematician Andrew Critch, CEO and cofounder of EnculturedAI, which is geared toward finding a happy union between artificial intelligence and humanity. The paper’s abstract gives a bracing overview: “This report presents a taxonomy and examples of potential omnicidal events resulting from AI: scenarios where all or almost all humans are killed. These events are not presented as inevitable, but as possibilities that we can work to avoid.”
I plan to reach out, though I have no particular credentials. (I was a math undergrad and published a few papers.) If anybody has an actual connection, please do reach out to him. I’m also unsure what my ask will be. Sign a statement? I can try to put him in touch with the ControlAI folks, who I know are doing good work in Canada?
He also seems to be a bit of a doomer. :O Might be worth some ppl reaching out to.
And oh, beyond that, in the first paragraph of the piece:
It also causes him worry. A sixth, nonmathematical, submission he made to the arXiv last year is titled “A Taxonomy of Omnicidal Futures Involving Artificial Intelligence.” This paper is coauthored with a longtime friend from math camp, the mathematician Andrew Critch, CEO and cofounder of EnculturedAI, which is geared toward finding a happy union between artificial intelligence and humanity. The paper’s abstract gives a bracing overview: “This report presents a taxonomy and examples of potential omnicidal events resulting from AI: scenarios where all or almost all humans are killed. These events are not presented as inevitable, but as possibilities that we can work to avoid.”
I plan to reach out, though I have no particular credentials. (I was a math undergrad and published a few papers.) If anybody has an actual connection, please do reach out to him. I’m also unsure what my ask will be. Sign a statement? I can try to put him in touch with the ControlAI folks, who I know are doing good work in Canada?