I totally take the point that the US and China could start to do Plan A but then switch to a worse, more authoritarian version that e.g. doesn’t have ZDR, but isn’t that also an argument against the other plans on offer too? Do you have a proposal for how to stop governments from being authoritarian?
FWIW, I don’t think anything short of Plan S can avoid concentration of power. Roughly speaking, for any given Plan X that navigates the AI risk while avoiding concentration of power, there is a neighboring Plan X* that is the same except that it concentrates power in the hands of the entities implementing the plan. If a consortium of companies and governments is prompted to implement Plan X, it would always be rational for them to do Plan X* instead. “Plan A” and “Plan A but without ZDR” is just one specific example pair.
Like, AI 2040 argues:
In short, the answer is that anyone concerned about loss of control should think Plan A is an improvement, along with anyone concerned about concentration of power—except for the people in whom the power would concentrate by default.
For these reasons, we expect strong opposition to Plan A from the leading AI companies. We expect them to rationalize arguments for why the deal is bad and why instead what’s best for America and humanity is a different strategy that just so happens to allow them to continue accumulating massive amounts of power.
China, by contrast, is an example of an actor in whom power would not concentrate by default.
Sure, suppose that’s true. Why would China agree to Plan A specifically, instead of Plan A but no ZDR?
This is IMO roughly the same failure mode as “if we have multipolar takeoff with several mutually misaligned ASIs, some of them would ally with humans and so humans would survive”. But no, the entities actually holding the world-changing power in the now can negotiate among themselves to screw everyone else out of having any power in the future.
It is theoretically possible to insist on anti-authoritarian implementations if the public is very aware of what’s happening and screens politicians and policies for that very strongly. But I am really, really pessimistic about that, given the current “vote for the least worst guy” US paradigm. However politically infeasible Plan S may seem, it seems more feasible than this.
I dunno, maybe there are some weaknesses in this argument and some way to design a plan such that there aren’t neighboring “except we also take all the power” plan variants; plans where the plan-implementers structurally can’t collide. I don’t currently see it, though.
There are degrees of concentration of power. A consortium of multiple governments—some of which are actual democracies thanks to the transparency requirements which help prevent AI-assisted executive power grabs—is way less bad than a single global dictator, for example.
I agree though that AGI and RSI are technologies that inherently concentrate power by default. It’s going to be really hard to resist that innate tendency. But I think Plan A does basically the best we can of the available plans so far, except maybe Plan S.
FWIW, I don’t think anything short of Plan S can avoid concentration of power. Roughly speaking, for any given Plan X that navigates the AI risk while avoiding concentration of power, there is a neighboring Plan X* that is the same except that it concentrates power in the hands of the entities implementing the plan. If a consortium of companies and governments is prompted to implement Plan X, it would always be rational for them to do Plan X* instead. “Plan A” and “Plan A but without ZDR” is just one specific example pair.
Like, AI 2040 argues:
Sure, suppose that’s true. Why would China agree to Plan A specifically, instead of Plan A but no ZDR?
This is IMO roughly the same failure mode as “if we have multipolar takeoff with several mutually misaligned ASIs, some of them would ally with humans and so humans would survive”. But no, the entities actually holding the world-changing power in the now can negotiate among themselves to screw everyone else out of having any power in the future.
It is theoretically possible to insist on anti-authoritarian implementations if the public is very aware of what’s happening and screens politicians and policies for that very strongly. But I am really, really pessimistic about that, given the current “vote for the least worst guy” US paradigm. However politically infeasible Plan S may seem, it seems more feasible than this.
I dunno, maybe there are some weaknesses in this argument and some way to design a plan such that there aren’t neighboring “except we also take all the power” plan variants; plans where the plan-implementers structurally can’t collide. I don’t currently see it, though.
There are degrees of concentration of power. A consortium of multiple governments—some of which are actual democracies thanks to the transparency requirements which help prevent AI-assisted executive power grabs—is way less bad than a single global dictator, for example.
I agree though that AGI and RSI are technologies that inherently concentrate power by default. It’s going to be really hard to resist that innate tendency. But I think Plan A does basically the best we can of the available plans so far, except maybe Plan S.