Ideas for how to spend very large amounts of money to improve AI safety:
If AI companies’ valuations continue to skyrocket (or if new very wealthy actors start to become worried about AI risk), there might be a large influx of funding into the AI safety space. Unfortunately, it’s not straightforward to magically turn money into valuable AI safety work. Many things in the AI safety ecosystem are more bottlenecked on having a good founder with the right talent and context, or having good researchers.
Here’s a random incomplete grab-bag of ideas for ways you could turn money into reductions in AI risk at large scales. I think right now there are much better donation opportunities available. This is not a list of donation recommendations right now, it’s just suggestions for once all the low-hanging funding fruit has been plucked. Probably if people thought more they could come up with even better scalable opportunities. There’s also probably existing great ideas I neglected to list. But these at least give us a baseline and a rough sense of what dumping a bunch of money into AI safety could look like. I’m also erring towards listing more things rather than fewer. Some of these things might actually be bad ideas.
Bounties to reward AIs for reporting misaligned behavior in themselves or other agents.
Folks have run a couple small experiments on this already. It seems straight-forward to execute and like it could absorb almost unbounded amounts of capital.
Paying high enough salaries to entice non-altruistically-motivated AI company employees to work on safety.
This isn’t only bottlenecked on funding. Many people are very loyal to the AI companies they work for, and the very best employees aren’t very sensitive to money since they already have plenty of money. It seems absurdly expensive for Meta to try hiring away people at other AI companies, and they didn’t seem to get that much top talent from it. On the one hand, working on safety is a much more compelling case than working at Meta, but on the other hand, maybe people who aren’t already doing safety research find AI capabilities research more intrinsically fun and interesting or rewarding than safety research. I am also concerned that people who do capabilities research might not be great at safety research because they might not feel as passionate or inspired by it, and because it is a somewhat different skillset.
In the most extremely optimistic world, you could probably hire 50 extremely talented people by offering them $100M/year each (matching what Meta offered). You could probably also hire ~200 more junior people at $10M/year (the bottleneck on hiring more would be management capacity). So in total you could spend $7B/year.
Over time, I expect this to get more expensive since AI companies’ valuations will increase, and therefore, so will employee compensation.
Compute for AI safety research.
Day-to-day, the AI safety researchers I know outside of AI labs don’t seem to think they’re very bottlenecked on compute. However, the AI safety researchers I know inside AI labs claim they get a lot of value from having gobs and gobs of compute everywhere. Probably, AI safety researchers outside labs are just not being imaginative enough about what they could do with tons of compute. This also isn’t entirely money-bottlenecked. Probably part of it is having the infrastructure in place and the deals with the compute providers, etc. And running experiments on lots of compute can be more fiddly and time-consuming. Even so I bet with a lot more money for compute, people would be able to do much better safety research.
Very roughly, I guess this could absorb ~$100 million a year.
Compute for running AI agents to automate AI safety research.
This doesn’t work today since AIs can’t automate AI safety research. But maybe in the future they will be able to, and you’ll be able to just dump money into this almost indefinitely.
Pay AI companies to do marginal cheap safety interventions.
Maybe you can just pay AI companies to implement safety interventions that are only very slightly costly for them. For example, you could subsidize having really good physical security in their data centers. I think a lot of things AI companies could do to improve safety will be costly enough for the companies that it will be very hard to pay them enough to make up for that cost, especially in worlds where AI companies’ valuations have increased a lot from where they are today. But there’s probably still some opportunities here.
Raising awareness of AI safety.
There’s lots of proven ways to spend money to raise awareness of things (sponsor youtube channels, patronize movies about AI risk, etc). Maybe raising awareness of safety is good because it gets more people to work on safety or gets the government to do more sensible things about AI risk or lets consumers encourage companies to implement more safety interventions.
I couldn’t easily find an American public awareness campaign that cost more than ~$80M/year (for anti-smoking). Coca Cola spends ~$4 billion a year on advertising, but I think that if AI safety were spending as much money as Coca-Cola, it would backfire. I think maybe $500M/year is a reasonable cap on what could be spent?
Biodefense. Buy everyone in the US PPE.
One way that an AI could cause a catastrophe is via designing a bioweapon. One way to reduce the odds that a bioweapon causes a civilization-ending catastrophe is to make sure that everyone has enough PPE that they won’t die. Andrew Snyder-Beattie has elaborated on this idea here. I think this could absorb ~$3B ($3/mask * 350M Americans * 3 masks/person).
Buy foreign AI safety researchers gold cards.
Many great AI safety researchers are on visas. It would be convenient if they had green cards. You can buy green cards now for $1M each. Let’s say there’s a hundred such people, so this opportunity could absorb $100M.
Overall, these are not amazing opportunities. But they give a lower bound and illustrate how it’s possible to turn money into reduced risk from AI at scale, even if you don’t have more entrepreneurs building new organizations. In practice, I think if money slowly ramps up into the space over time, there will be much better opportunities than these, and you will simply see AI safety organizations that have grown to be major research institutions that are producing wonderful research. This is just a floor.
A lot of these ideas came from other people and have generally been floating around for a while. Thanks to everybody I talk to about this.
In the most extremely optimistic world, you could probably hire 50 extremely talented people by offering them $100M/year each (matching what Meta offered). You could probably also hire ~200 more junior people at $10M/year (the bottleneck on hiring more would be management capacity). So in total you could spend $7B/year.
Over time, I expect this to get more expensive since AI companies’ valuations will increase, and therefore, so will employee compensation.
I don’t know that the idea is fundamentally good but at least is scales somewhat with the equity of the safety-sympathetic people at labs?
Ideas for how to spend very large amounts of money to improve AI safety:
If AI companies’ valuations continue to skyrocket (or if new very wealthy actors start to become worried about AI risk), there might be a large influx of funding into the AI safety space. Unfortunately, it’s not straightforward to magically turn money into valuable AI safety work. Many things in the AI safety ecosystem are more bottlenecked on having a good founder with the right talent and context, or having good researchers.
Here’s a random incomplete grab-bag of ideas for ways you could turn money into reductions in AI risk at large scales. I think right now there are much better donation opportunities available. This is not a list of donation recommendations right now, it’s just suggestions for once all the low-hanging funding fruit has been plucked. Probably if people thought more they could come up with even better scalable opportunities. There’s also probably existing great ideas I neglected to list. But these at least give us a baseline and a rough sense of what dumping a bunch of money into AI safety could look like. I’m also erring towards listing more things rather than fewer. Some of these things might actually be bad ideas.
Bounties to reward AIs for reporting misaligned behavior in themselves or other agents.
Folks have run a couple small experiments on this already. It seems straight-forward to execute and like it could absorb almost unbounded amounts of capital.
Paying high enough salaries to entice non-altruistically-motivated AI company employees to work on safety.
This isn’t only bottlenecked on funding. Many people are very loyal to the AI companies they work for, and the very best employees aren’t very sensitive to money since they already have plenty of money. It seems absurdly expensive for Meta to try hiring away people at other AI companies, and they didn’t seem to get that much top talent from it. On the one hand, working on safety is a much more compelling case than working at Meta, but on the other hand, maybe people who aren’t already doing safety research find AI capabilities research more intrinsically fun and interesting or rewarding than safety research. I am also concerned that people who do capabilities research might not be great at safety research because they might not feel as passionate or inspired by it, and because it is a somewhat different skillset.
In the most extremely optimistic world, you could probably hire 50 extremely talented people by offering them $100M/year each (matching what Meta offered). You could probably also hire ~200 more junior people at $10M/year (the bottleneck on hiring more would be management capacity). So in total you could spend $7B/year.
Over time, I expect this to get more expensive since AI companies’ valuations will increase, and therefore, so will employee compensation.
Compute for AI safety research.
Day-to-day, the AI safety researchers I know outside of AI labs don’t seem to think they’re very bottlenecked on compute. However, the AI safety researchers I know inside AI labs claim they get a lot of value from having gobs and gobs of compute everywhere. Probably, AI safety researchers outside labs are just not being imaginative enough about what they could do with tons of compute. This also isn’t entirely money-bottlenecked. Probably part of it is having the infrastructure in place and the deals with the compute providers, etc. And running experiments on lots of compute can be more fiddly and time-consuming. Even so I bet with a lot more money for compute, people would be able to do much better safety research.
Very roughly, I guess this could absorb ~$100 million a year.
Compute for running AI agents to automate AI safety research.
This doesn’t work today since AIs can’t automate AI safety research. But maybe in the future they will be able to, and you’ll be able to just dump money into this almost indefinitely.
Pay AI companies to do marginal cheap safety interventions.
Maybe you can just pay AI companies to implement safety interventions that are only very slightly costly for them. For example, you could subsidize having really good physical security in their data centers. I think a lot of things AI companies could do to improve safety will be costly enough for the companies that it will be very hard to pay them enough to make up for that cost, especially in worlds where AI companies’ valuations have increased a lot from where they are today. But there’s probably still some opportunities here.
Raising awareness of AI safety.
There’s lots of proven ways to spend money to raise awareness of things (sponsor youtube channels, patronize movies about AI risk, etc). Maybe raising awareness of safety is good because it gets more people to work on safety or gets the government to do more sensible things about AI risk or lets consumers encourage companies to implement more safety interventions.
I couldn’t easily find an American public awareness campaign that cost more than ~$80M/year (for anti-smoking). Coca Cola spends ~$4 billion a year on advertising, but I think that if AI safety were spending as much money as Coca-Cola, it would backfire. I think maybe $500M/year is a reasonable cap on what could be spent?
Biodefense. Buy everyone in the US PPE.
One way that an AI could cause a catastrophe is via designing a bioweapon. One way to reduce the odds that a bioweapon causes a civilization-ending catastrophe is to make sure that everyone has enough PPE that they won’t die. Andrew Snyder-Beattie has elaborated on this idea here. I think this could absorb ~$3B ($3/mask * 350M Americans * 3 masks/person).
Buy foreign AI safety researchers gold cards.
Many great AI safety researchers are on visas. It would be convenient if they had green cards. You can buy green cards now for $1M each. Let’s say there’s a hundred such people, so this opportunity could absorb $100M.
Overall, these are not amazing opportunities. But they give a lower bound and illustrate how it’s possible to turn money into reduced risk from AI at scale, even if you don’t have more entrepreneurs building new organizations. In practice, I think if money slowly ramps up into the space over time, there will be much better opportunities than these, and you will simply see AI safety organizations that have grown to be major research institutions that are producing wonderful research. This is just a floor.
A lot of these ideas came from other people and have generally been floating around for a while. Thanks to everybody I talk to about this.
I don’t know that the idea is fundamentally good but at least is scales somewhat with the equity of the safety-sympathetic people at labs?