This requires having more views about worlds where money is most valuable. There’s a U-curve where
excellent policies pass: we’re in the clear. low value of money
decent policies pass, or maybe they fail by a thin margin: more work required. high value of money
nothing is even close to passing: we’re doomed. low value of money
It’s complicated to determine where the peak of the U-curve is, or even which side of it you’re on. I think I’d have a hard time figuring out which side of a bet to take to maximize expected value.
For example, it’s not obvious to me that the doomer should prefer to bet YES on a datacenter moratorium getting passed.
If excellent policy passes that let’s say shuts down AI, money would be valuable in the selfish sense? Like you can buy something from it, this is the main sense I was thinking of when talking about value of money. Are you thinking of how useful is money in advancing safe AI?
One good scenario with low payoffability is one where there is ASI aligned to humanity as a whole, such that only a few prestige goods are meaningfully scarce and money matters much less to material quality of life.
This could be the default outcome of good policy if alignment is moderately hard—not so easy that we get it for free with capabilities, but not so hard that we can’t have a moonshot program or temporary pause that allows it to catch up. (We can throw in ”aligned to human or sentient interests as a whole rather than some corporate/political egomaniac” as part of “good.”)
Ofc no policy can guarantee that alignment is moderately easy!
This requires having more views about worlds where money is most valuable. There’s a U-curve where
excellent policies pass: we’re in the clear. low value of money
decent policies pass, or maybe they fail by a thin margin: more work required. high value of money
nothing is even close to passing: we’re doomed. low value of money
It’s complicated to determine where the peak of the U-curve is, or even which side of it you’re on. I think I’d have a hard time figuring out which side of a bet to take to maximize expected value.
For example, it’s not obvious to me that the doomer should prefer to bet YES on a datacenter moratorium getting passed.
If excellent policy passes that let’s say shuts down AI, money would be valuable in the selfish sense? Like you can buy something from it, this is the main sense I was thinking of when talking about value of money. Are you thinking of how useful is money in advancing safe AI?
Yeah I was thinking of the value of money for reducing x-risk.
One good scenario with low payoffability is one where there is ASI aligned to humanity as a whole, such that only a few prestige goods are meaningfully scarce and money matters much less to material quality of life.
This could be the default outcome of good policy if alignment is moderately hard—not so easy that we get it for free with capabilities, but not so hard that we can’t have a moonshot program or temporary pause that allows it to catch up. (We can throw in ”aligned to human or sentient interests as a whole rather than some corporate/political egomaniac” as part of “good.”)
Ofc no policy can guarantee that alignment is moderately easy!