Exiting the “semi-anarchic default condition”, if it happens, seems likely to be a slow and distributed process, since no one group can make global decisions until we exit that condition. The state of thought and discussion generally, and opinions of prominent people like Nick Bostrom particularly, around the issue will probably influence the general current of “small” decisions toward or away from an exit. Thus, getting closer to the right answer here may slightly increase our chances in the long run. Not a primary concern, but worth some discussion, I think.
I’m pretty sure that the semi-anarchic default condition is a stable equilibrium. As soon as any power structure started to coalesce, everybody who wasn’t a part of it would feel threatened by it and attack it. Once having neutralized the threat, any coalitions that had formed against it would themselves self-destruct in internal mistrust. If it’s even possible to leave an equilibrium like that, you definitely can’t do it slowly.
On the other hand, the post-semi-anarchic regime is probably fairly unstable… anybody who gets out from under it a little bit can use that to get out from under it more. And many actors have incentives to do so. Maybe you could stay in it, but only if you spent a lot of its enforcement power on the meta-problem of keeping it going.
My views on this may be colored by the fact that Bostrom’s vision for the post-semi-anarchic condition in itself sounds like a catastrophic outcome to me, not least because it seems obvious to me that it would immediately be used way, way beyond any kind of catastrophic risk management, to absolutely enforce and entrench any and every social norm that could get 51 percent support, and to absolutely suppress all dissent. YMMV on that part, but anyway I don’t think my view of whether it’s possible is that strongly determined by my view that it’s undesirable.
Hmm. I think you’re right. I just realized I don’t have any actual models for how we might exit the semi-anarchy without friendly superintelligence (it seemed hard, so I assumed gradualism), and it seems dangerous to try.
Furthermore, in reference to the crux in your original comment, the semi-anarchy doesn’t seem dangerous enough for a world government to improve our chances. What we’re looking for is global coordination capacity, and we can improve that without building one.