Hi David! Thanks for your input and for the work on the Leiden Declaration. My essay was intentionally vague on what to do, as I don’t strongly believe in a particular approach that is likely to help, and also have not worked deeply enough in technical alignment to have strong opinions.
To be honest, one of the main goals of this essay was an implicit call for inaction: “Don’t go work for an AI company without sitting down and thinking about x-risk (but maybe do if you have a concrete theory of change after thinking about it, like Tsimerman seems to).”
Some obvious places for mathematicians to start looking for concrete directions are ARC and Lionel Levine’s work. Many major universities already have extant AI Safety groups, so senior mathematicians can probably pick up some low-hanging fruit just by getting affiliated to lend credibility, and asking to be looped in on the mathematical parts of their research.
My current conjecture (lightly held) is that senior mathematicians concerned about x-risk can have the most immediate impact not by trying to solve technical problems, as we are conditioned to do, but by throwing our weight around in universities and public discourse until we reach a critical mass. I understand that many individual mathematicians are becoming aware of the case for x-risk, but there is almost 0 public discourse or common knowledge about it. I think we could very easily completely shift the discourse if ~20 mathematicians wrote their own version of my essay, or there was a version of the Leiden Declaration centered around alignment concerns.
Hi David! Thanks for your input and for the work on the Leiden Declaration. My essay was intentionally vague on what to do, as I don’t strongly believe in a particular approach that is likely to help, and also have not worked deeply enough in technical alignment to have strong opinions.
To be honest, one of the main goals of this essay was an implicit call for inaction: “Don’t go work for an AI company without sitting down and thinking about x-risk (but maybe do if you have a concrete theory of change after thinking about it, like Tsimerman seems to).”
Some obvious places for mathematicians to start looking for concrete directions are ARC and Lionel Levine’s work. Many major universities already have extant AI Safety groups, so senior mathematicians can probably pick up some low-hanging fruit just by getting affiliated to lend credibility, and asking to be looped in on the mathematical parts of their research.
My current conjecture (lightly held) is that senior mathematicians concerned about x-risk can have the most immediate impact not by trying to solve technical problems, as we are conditioned to do, but by throwing our weight around in universities and public discourse until we reach a critical mass. I understand that many individual mathematicians are becoming aware of the case for x-risk, but there is almost 0 public discourse or common knowledge about it. I think we could very easily completely shift the discourse if ~20 mathematicians wrote their own version of my essay, or there was a version of the Leiden Declaration centered around alignment concerns.