Hm, doesn’t the power imbalance thesis actually get the sign wrong? When many actors have comparable power, costly wars become more likely. When one agent has overwhelming dominance, costly conflict tends to decrease, as the weaker side either submits or gets crushed quickly. The unipolar period from ~1990–2015 is a strong example: the US had the most extreme power advantage in modern history, yet it was one of the most peaceful periods for interstate war and mass atrocities.[1]
What’s most important, I think, are the values of whoever holds the power. Power imbalance + liberal democratic norms = Pax Americana. Power imbalance + fanatical ideology = genocide. And with ASI, the power imbalance might be near-infinite and the crucial question will be which values the ASIs or their principals have. That’s why fanaticism (and malevolence) matter.
On the elephant/rider point: I totally agree that “conflicts involving ideology should [not] always be blamed on ideology”, certainly not exclusively. We explicitly acknowledge this:
Of course, no single factor fully explains any historical atrocity. In addition to ideological fanaticism, other crucial causes and risk factors include political and economic instability (e.g., Weimar Germany), power-seeking and competition between individuals and groups (present in essentially all atrocities), inequality and exploitation (e.g., in Congo Free State), [...].
But it seems clear that ideological fanaticism at least contributed to many of the worst atrocities in recent history, and some atrocities like the Holocaust, the Great Purge, the Cultural Revolution, plausibly would not have happened if not for ideological fanaticism.
Of course, there were some wars during this period, such as in Iraq and Afghanistan, and the US certainly deserves significant blame for how these were conducted, particularly in Iraq. But casualty figures were orders of magnitude lower than the great power conflicts and ideologically-driven atrocities of the 20th century. And notably, these wars were partly responses to 9/11—itself a product of ideological fanaticism—and involved regimes like Saddam Hussein’s dictatorship and the Taliban.
The main examples in my mind are the Mongol conquests and Western colonialism, which I think were the biggest atrocities in history and were more due to power imbalance than fanaticism.
But there’s maybe a more general point I want to make. I think focusing on benevolence isn’t the right path. Let’s say we build a powerful benevolent thing. How can we make sure it stays benevolent to us? After all, value drift is always possible, we don’t have any math to rule it out.
For a while I thought the solution should be some kind of “continuous alignment”, being able to influence the powerful thing as it evolves. But then I realized that it’s simpler than that. Being able to influence a powerful thing that doesn’t want to be influenced is simply another synonym for “having power”. The problem of making sure a powerful thing stays benevolent is exactly the same as the problem of making sure power is spread out, so the powerful thing can be kept in check. The two things mean the same thing.
So now that’s what I’m arguing for in these threads. I want people to get over the framing of “power imbalance is ok as long as the thing is benevolent, so we should focus on ensuring benevolence”, and switch to “power is always subject to value drift, so power imbalance is dangerous in itself, and we should focus on making power spread out”. It feels a really important point to me. Does that make sense?
Hm, doesn’t the power imbalance thesis actually get the sign wrong? When many actors have comparable power, costly wars become more likely. When one agent has overwhelming dominance, costly conflict tends to decrease, as the weaker side either submits or gets crushed quickly. The unipolar period from ~1990–2015 is a strong example: the US had the most extreme power advantage in modern history, yet it was one of the most peaceful periods for interstate war and mass atrocities.[1]
What’s most important, I think, are the values of whoever holds the power. Power imbalance + liberal democratic norms = Pax Americana. Power imbalance + fanatical ideology = genocide. And with ASI, the power imbalance might be near-infinite and the crucial question will be which values the ASIs or their principals have. That’s why fanaticism (and malevolence) matter.
On the elephant/rider point: I totally agree that “conflicts involving ideology should [not] always be blamed on ideology”, certainly not exclusively. We explicitly acknowledge this:
But it seems clear that ideological fanaticism at least contributed to many of the worst atrocities in recent history, and some atrocities like the Holocaust, the Great Purge, the Cultural Revolution, plausibly would not have happened if not for ideological fanaticism.
Of course, there were some wars during this period, such as in Iraq and Afghanistan, and the US certainly deserves significant blame for how these were conducted, particularly in Iraq. But casualty figures were orders of magnitude lower than the great power conflicts and ideologically-driven atrocities of the 20th century. And notably, these wars were partly responses to 9/11—itself a product of ideological fanaticism—and involved regimes like Saddam Hussein’s dictatorship and the Taliban.
The main examples in my mind are the Mongol conquests and Western colonialism, which I think were the biggest atrocities in history and were more due to power imbalance than fanaticism.
But there’s maybe a more general point I want to make. I think focusing on benevolence isn’t the right path. Let’s say we build a powerful benevolent thing. How can we make sure it stays benevolent to us? After all, value drift is always possible, we don’t have any math to rule it out.
For a while I thought the solution should be some kind of “continuous alignment”, being able to influence the powerful thing as it evolves. But then I realized that it’s simpler than that. Being able to influence a powerful thing that doesn’t want to be influenced is simply another synonym for “having power”. The problem of making sure a powerful thing stays benevolent is exactly the same as the problem of making sure power is spread out, so the powerful thing can be kept in check. The two things mean the same thing.
So now that’s what I’m arguing for in these threads. I want people to get over the framing of “power imbalance is ok as long as the thing is benevolent, so we should focus on ensuring benevolence”, and switch to “power is always subject to value drift, so power imbalance is dangerous in itself, and we should focus on making power spread out”. It feels a really important point to me. Does that make sense?