A key concern in the present context is whether the consequences of civilization continuing in the current semi-anarchic default condndition are catastrophic enough to outweigh reasonable objections to the drastic developments that would be required to exit this condition. [Emphasis in original]
That only matters if you’re in a position to enact the “drastic developments” (and to do so without incurring some equally bad catastrophe in the process). If you’re not in a position to make something happen, then it doesn’t matter whether it’s the right thing to do or not.
Where’s there any sign that any person or group has or ever will have the slightest chance of being able to cause the world to exit the “semi-anarchic default condition”, or the slightest idea of how to go about doing so? I’ve never seen any. So what’s the point in talking about it?
Exiting the “semi-anarchic default condition”, if it happens, seems likely to be a slow and distributed process, since no one group can make global decisions until we exit that condition. The state of thought and discussion generally, and opinions of prominent people like Nick Bostrom particularly, around the issue will probably influence the general current of “small” decisions toward or away from an exit. Thus, getting closer to the right answer here may slightly increase our chances in the long run. Not a primary concern, but worth some discussion, I think.
I’m pretty sure that the semi-anarchic default condition is a stable equilibrium. As soon as any power structure started to coalesce, everybody who wasn’t a part of it would feel threatened by it and attack it. Once having neutralized the threat, any coalitions that had formed against it would themselves self-destruct in internal mistrust. If it’s even possible to leave an equilibrium like that, you definitely can’t do it slowly.
On the other hand, the post-semi-anarchic regime is probably fairly unstable… anybody who gets out from under it a little bit can use that to get out from under it more. And many actors have incentives to do so. Maybe you could stay in it, but only if you spent a lot of its enforcement power on the meta-problem of keeping it going.
My views on this may be colored by the fact that Bostrom’s vision for the post-semi-anarchic condition in itself sounds like a catastrophic outcome to me, not least because it seems obvious to me that it would immediately be used way, way beyond any kind of catastrophic risk management, to absolutely enforce and entrench any and every social norm that could get 51 percent support, and to absolutely suppress all dissent. YMMV on that part, but anyway I don’t think my view of whether it’s possible is that strongly determined by my view that it’s undesirable.
Hmm. I think you’re right. I just realized I don’t have any actual models for how we might exit the semi-anarchy without friendly superintelligence (it seemed hard, so I assumed gradualism), and it seems dangerous to try.
Furthermore, in reference to the crux in your original comment, the semi-anarchy doesn’t seem dangerous enough for a world government to improve our chances. What we’re looking for is global coordination capacity, and we can improve that without building one.
It seems to me that this is the crux:
That only matters if you’re in a position to enact the “drastic developments” (and to do so without incurring some equally bad catastrophe in the process). If you’re not in a position to make something happen, then it doesn’t matter whether it’s the right thing to do or not.
Where’s there any sign that any person or group has or ever will have the slightest chance of being able to cause the world to exit the “semi-anarchic default condition”, or the slightest idea of how to go about doing so? I’ve never seen any. So what’s the point in talking about it?
The mean person has 1 / 7 billionth control over the fate of humanity. There’s your slightest chance right there!
Edit: In other words, the world is big but not infinite. We are small but not infinitesimal.
Exiting the “semi-anarchic default condition”, if it happens, seems likely to be a slow and distributed process, since no one group can make global decisions until we exit that condition. The state of thought and discussion generally, and opinions of prominent people like Nick Bostrom particularly, around the issue will probably influence the general current of “small” decisions toward or away from an exit. Thus, getting closer to the right answer here may slightly increase our chances in the long run. Not a primary concern, but worth some discussion, I think.
I’m pretty sure that the semi-anarchic default condition is a stable equilibrium. As soon as any power structure started to coalesce, everybody who wasn’t a part of it would feel threatened by it and attack it. Once having neutralized the threat, any coalitions that had formed against it would themselves self-destruct in internal mistrust. If it’s even possible to leave an equilibrium like that, you definitely can’t do it slowly.
On the other hand, the post-semi-anarchic regime is probably fairly unstable… anybody who gets out from under it a little bit can use that to get out from under it more. And many actors have incentives to do so. Maybe you could stay in it, but only if you spent a lot of its enforcement power on the meta-problem of keeping it going.
My views on this may be colored by the fact that Bostrom’s vision for the post-semi-anarchic condition in itself sounds like a catastrophic outcome to me, not least because it seems obvious to me that it would immediately be used way, way beyond any kind of catastrophic risk management, to absolutely enforce and entrench any and every social norm that could get 51 percent support, and to absolutely suppress all dissent. YMMV on that part, but anyway I don’t think my view of whether it’s possible is that strongly determined by my view that it’s undesirable.
Hmm. I think you’re right. I just realized I don’t have any actual models for how we might exit the semi-anarchy without friendly superintelligence (it seemed hard, so I assumed gradualism), and it seems dangerous to try.
Furthermore, in reference to the crux in your original comment, the semi-anarchy doesn’t seem dangerous enough for a world government to improve our chances. What we’re looking for is global coordination capacity, and we can improve that without building one.