I’m kind of curious here. Maybe in 50 years people will look back and say ‘people in the early 21st were really freaked out about AI, like it was the biggest risk. But here we are and AI is fine, great actually’. I think we now are right that AI is a big deal, but what we do now might mean it plays out fine, if we muster well!
were the people who were worried about overpopulation the ones who made it turn out fine? afaict, declining birth rates due to contraceptive usage are mostly because of economic (kids are expensive) and cultural factors (people want to do stuff other than raise kids) rather than people deciding not to have kids because of overpopulation
The declining birth rates aren’t the main savior, it was the food production and transportation improvements.
So the analogy to AI would be: Alignment research makes enough progress fast enough (perhaps at AI companies partially motivated by mundane alignment concerns like trying to tamp down on sycophancy and reward hacking) that Claude Mythos 3 is aligned enough to trust with alignment research, and so when Anthropic hands off everything to Claude Mythos 3, it’s able to quickly solve the rest of the problems and align its successor, and so on to ASI. In retrospect there are some heroes like Borlaug, but also just a lot of normal science research that happened to go fast enough to work.
Yes, the difference is that if that happens, their superintelligent AI assistant will gently correct them: “As per my standing instructions to help you become more the person you wish you were, I must correct you on this point. AI was in fact a huge problem in the late twenties, and could easily have gotten everyone killed, but these risks were prevented by the following policy decisions and the following technical alignment discoveries...”
This, but on a faster scale. People in January 2027 will say: “People in June 2026 were freaked about AI, but they had it easy compared to what we have now.”
I’m kind of curious here. Maybe in 50 years people will look back and say ‘people in the early 21st were really freaked out about AI, like it was the biggest risk. But here we are and AI is fine, great actually’. I think we now are right that AI is a big deal, but what we do now might mean it plays out fine, if we muster well!
were the people who were worried about overpopulation the ones who made it turn out fine? afaict, declining birth rates due to contraceptive usage are mostly because of economic (kids are expensive) and cultural factors (people want to do stuff other than raise kids) rather than people deciding not to have kids because of overpopulation
The declining birth rates aren’t the main savior, it was the food production and transportation improvements.
So the analogy to AI would be: Alignment research makes enough progress fast enough (perhaps at AI companies partially motivated by mundane alignment concerns like trying to tamp down on sycophancy and reward hacking) that Claude Mythos 3 is aligned enough to trust with alignment research, and so when Anthropic hands off everything to Claude Mythos 3, it’s able to quickly solve the rest of the problems and align its successor, and so on to ASI. In retrospect there are some heroes like Borlaug, but also just a lot of normal science research that happened to go fast enough to work.
Yes, the difference is that if that happens, their superintelligent AI assistant will gently correct them: “As per my standing instructions to help you become more the person you wish you were, I must correct you on this point. AI was in fact a huge problem in the late twenties, and could easily have gotten everyone killed, but these risks were prevented by the following policy decisions and the following technical alignment discoveries...”
This, but on a faster scale. People in January 2027 will say: “People in June 2026 were freaked about AI, but they had it easy compared to what we have now.”