To be honest, I now think the post has far less value than people thought in in 2024, and this is not because it’s factual statements are wrong, but rather that Richard Ngo and a lot of LWers way underestimated how intractable it was to avoid polarization of AI safety issues in worlds where AI safety is highly salient politically without deep changes to our electoral systems.
(To be clear, this is a US-centric view, but this is justified given that the by far most likely countries to have ASI in their borders initially are the US and China, due to both compute and researcher advantages over the rest of the world, and the US’s laws are more tractable to change for AI safety than China’s laws on AI).
The basic reason for this is that social sorting has fundamentally led to the breakdown of conditions that allowed low polarization without something like proportional voting/score voting, and this is a problem that AI safety people are ill-equipped to solve or even ameliorate significantly, unfortunately.
And this means absent electoral reform, we should assume by default as AI safety becomes higher salience, it will become more polarized, and this means there’s a tradeoff between making AI higher salience to achieving political goals.
I’m giving it a 0 for this reason, because in the end the post was basically irrelevant to how AI safety should act, and how to handle the tradeoff of polarization vs getting enough salience to get laws passed is unfortunately more complicated than the post makes it out to be, and much tougher to solve.
To be honest, I now think the post has far less value than people thought in in 2024, and this is not because it’s factual statements are wrong, but rather that Richard Ngo and a lot of LWers way underestimated how intractable it was to avoid polarization of AI safety issues in worlds where AI safety is highly salient politically without deep changes to our electoral systems.
(To be clear, this is a US-centric view, but this is justified given that the by far most likely countries to have ASI in their borders initially are the US and China, due to both compute and researcher advantages over the rest of the world, and the US’s laws are more tractable to change for AI safety than China’s laws on AI).
The basic reason for this is that social sorting has fundamentally led to the breakdown of conditions that allowed low polarization without something like proportional voting/score voting, and this is a problem that AI safety people are ill-equipped to solve or even ameliorate significantly, unfortunately.
And this means absent electoral reform, we should assume by default as AI safety becomes higher salience, it will become more polarized, and this means there’s a tradeoff between making AI higher salience to achieving political goals.
I’m giving it a 0 for this reason, because in the end the post was basically irrelevant to how AI safety should act, and how to handle the tradeoff of polarization vs getting enough salience to get laws passed is unfortunately more complicated than the post makes it out to be, and much tougher to solve.