I think Joe is arguing that unilateral capability restraint might exacerbate such risks.
For example, to the extent that safety-related considerations end up motivating or rationalizing especially drastic forms of international action aimed at shutting down or significantly restricting AI development in other countries, I think this could well be harmful even relative to more baseline forms of economic and military competition.
Like, the US government is all in on safety but China is not complying. Which would probably get intense, and rightfully so.
I still agree with you because:
I don’t think that is very likely at all; if one country/coalition is all-in on safety, I’d guess there is something motivating their fear that would apply to everyone else as well.
The default outcome of nobody being all-in on safety seems obviously much worse as you point out.
There is an argument to be made that great power conflict is justified in the credible expectation of total annihilation otherwise. Obviously this is extremely controversial as it can be easily misused, and IIRC has been used to deface the EA/rationalist community (or Nick Bostrom?) in the past, but I don’t think that takes away from it’s merit.
I think Joe is arguing that unilateral capability restraint might exacerbate such risks.
Like, the US government is all in on safety but China is not complying. Which would probably get intense, and rightfully so.
I still agree with you because:
I don’t think that is very likely at all; if one country/coalition is all-in on safety, I’d guess there is something motivating their fear that would apply to everyone else as well.
The default outcome of nobody being all-in on safety seems obviously much worse as you point out.
There is an argument to be made that great power conflict is justified in the credible expectation of total annihilation otherwise. Obviously this is extremely controversial as it can be easily misused, and IIRC has been used to deface the EA/rationalist community (or Nick Bostrom?) in the past, but I don’t think that takes away from it’s merit.