I think both of these camps are seeing real things. I think:
We should not race to superintelligence because we’re not prepared to have a reasonable chance of surviving AND It’s extremely hard to stop due to underlying dynamics of civilization (competitive dynamics, moloch, etc)
We should try to stop the race, but be clearsighted about the forces we’re up against and devise plans that have a chance of working despite them.
I think you’re overstating how difficult it is for the government to regulate AI. With the exception of SB 53 in California, the reason not much has happened yet is that there have been barely any attempts by governments to regulate AI. I think all it would take is for some informed government to start taking this issue seriously (in a way that LessWrong people already do).
I think this may be because the US government tends to take a hands off approach and assume the market knows best which is usually true.
I think it will be informative to see how China handles this because they have a track record of heavy-handed government interventions like banning Google, the 2021 tech industry crackdown, extremely strict covid lockdowns and so on.
From some quick research online, the number of private tutoring institutions and the revenue of the private tutoring sector fell by ~80% when the Chinese government banned for-profit tutoring in 2021 despite education having pretty severe arms race dynamics similar to AI.
I think both of these camps are seeing real things. I think:
We should try to stop the race, but be clearsighted about the forces we’re up against and devise plans that have a chance of working despite them.
I think you’re overstating how difficult it is for the government to regulate AI. With the exception of SB 53 in California, the reason not much has happened yet is that there have been barely any attempts by governments to regulate AI. I think all it would take is for some informed government to start taking this issue seriously (in a way that LessWrong people already do).
I think this may be because the US government tends to take a hands off approach and assume the market knows best which is usually true.
I think it will be informative to see how China handles this because they have a track record of heavy-handed government interventions like banning Google, the 2021 tech industry crackdown, extremely strict covid lockdowns and so on.
From some quick research online, the number of private tutoring institutions and the revenue of the private tutoring sector fell by ~80% when the Chinese government banned for-profit tutoring in 2021 despite education having pretty severe arms race dynamics similar to AI.