Capability restraint might exacerbate risks of great power conflict.
What? I think the opposite is true. Absent capability restraint, the situation is going to get very intense. China, for example, will be rightly fearful of what the US will do to them if the US does an intelligence explosion.
A more abstract argument I don’t buy nearly as much: AGI, being smarter and more numerous and thinking faster than humans, will be like accelerating history itself. A century will pass in a decade, or maybe in a year, or maybe in a month, depending on takeoff speeds. Therefore, a century’s worth of turmoil and conflicts will happen during that time as well.
I think Joe is arguing that unilateral capability restraint might exacerbate such risks.
For example, to the extent that safety-related considerations end up motivating or rationalizing especially drastic forms of international action aimed at shutting down or significantly restricting AI development in other countries, I think this could well be harmful even relative to more baseline forms of economic and military competition.
Like, the US government is all in on safety but China is not complying. Which would probably get intense, and rightfully so.
I still agree with you because:
I don’t think that is very likely at all; if one country/coalition is all-in on safety, I’d guess there is something motivating their fear that would apply to everyone else as well.
The default outcome of nobody being all-in on safety seems obviously much worse as you point out.
There is an argument to be made that great power conflict is justified in the credible expectation of total annihilation otherwise. Obviously this is extremely controversial as it can be easily misused, and IIRC has been used to deface the EA/rationalist community (or Nick Bostrom?) in the past, but I don’t think that takes away from it’s merit.
What? I think the opposite is true. Absent capability restraint, the situation is going to get very intense. China, for example, will be rightly fearful of what the US will do to them if the US does an intelligence explosion.
A more abstract argument I don’t buy nearly as much: AGI, being smarter and more numerous and thinking faster than humans, will be like accelerating history itself. A century will pass in a decade, or maybe in a year, or maybe in a month, depending on takeoff speeds. Therefore, a century’s worth of turmoil and conflicts will happen during that time as well.
I think Joe is arguing that unilateral capability restraint might exacerbate such risks.
Like, the US government is all in on safety but China is not complying. Which would probably get intense, and rightfully so.
I still agree with you because:
I don’t think that is very likely at all; if one country/coalition is all-in on safety, I’d guess there is something motivating their fear that would apply to everyone else as well.
The default outcome of nobody being all-in on safety seems obviously much worse as you point out.
There is an argument to be made that great power conflict is justified in the credible expectation of total annihilation otherwise. Obviously this is extremely controversial as it can be easily misused, and IIRC has been used to deface the EA/rationalist community (or Nick Bostrom?) in the past, but I don’t think that takes away from it’s merit.