By Strong Default, ASI Will End Liberal Democracy
Cross-posted from my website.
The existence of liberal democracy—with rule of law, constraints on government power, and enfranchised citizens—relies on a balance of power where individual bad actors can’t do too much damage. Artificial superintelligence (ASI), even if it’s aligned, would end that balance by default.
It is not a question of who develops ASI. Whether the first ASI is developed by a totalitarian state or a democracy, the end result will—by strong default—be a de facto global dictatorship.
The central problem is that whoever controls ASI can defeat any opposition. Imagine a scenario where (say) DARPA develops the first superintelligence[1], and the head of the ASI training program decides to seize power. What can anyone do about it?
If the president orders the military to capture DARPA’s data centers, the ASI can defeat the military.[2]
If Congress issues a mandate that DARPA must turn over control of the ASI, DARPA can refuse, and Congress has even less recourse than the president.
If liberal democracy continues to exist, it will only be by the grace of whoever controls ASI.
There are two plausible scenarios that have some chance of avoiding a totalitarian outcome:
AI capabilities progress slowly.
The ASI itself protects liberal democracy.
I will discuss them in turn.
What if AI capabilities progress slowly?
We have a chance at averting de facto totalitarianism if two conditions hold:
At each step of AI development, control of AI is distributed widely.
At each step, the next-generation AI is not strong enough to overpower all the copies of the previous generation.
Widely distributing AI is difficult—today’s frontier LLMs require supercomputers to run, their hardware requirements are becoming increasingly expensive with each generation, and AI developers have strong incentives against distributing them. In addition, distributing AI exacerbates misalignment and misuse risks, and it’s likely not worth the tradeoff.
We do not know whether takeoff will be fast or slow; banking on a slow takeoff is an extremely risky move. Frontier AI companies are trying their best to rapidly build up to ASI, and they explicitly want to make AI do recursive self-improvement. If they succeed, it’s hard to see how liberal democracy will be able to preserve itself.
What if the ASI itself protects liberal democracy?
There is a conceivable scenario where an aligned ASI preserves liberal democracy, and refuses any orders that would violate people’s civil liberties.
Above, I wrote:
If liberal democracy continues to exist, it will only be by the grace of whoever controls ASI.
That’s still true, but in this case “whoever controls ASI” would be the ASI itself. If it’s aligned in a transparent way, then maybe we can be confident that it really will preserve democracy.
Even in this scenario, there is still a small group of people who control how the ASI is trained. The hope is that, at training time, those people do not yet have enough power to prevent oversight. For example, maybe laws mandate that (1) AI developers must make their training process public and auditable and (2) the training process must steer the AI toward valuing liberal democracy. It is not at all obvious how those laws would work, or how we would get those laws, or how they would be enforced; but at least this outcome is conceivable as a possibility.
This scenario introduces some additional challenges:
The ASI must be incorrigible with respect to protecting liberal democracy. That constrains us in terms of what types of alignment solutions we can use, which makes the alignment problem harder to solve. Incorrigibility means if you make a mistake in designing the AI, then you can’t fix it.
We must ensure that an immutable “protect liberal democracy” directive won’t have severe unintended consequences—which, by default, it probably will. (Think Asimov’s Three Laws of Robotics.)
AI progress must proceed slowly enough that the appropriate laws or regulations can be put in place before it’s too late; or we must trust that the leading AI developer embeds appropriate values into its ASI.
Liberal democracy is not the true target
As the saying goes, democracy is the worst form of government except for all those other forms that have been tried. We don’t want democracy; what we want is a truly good form of government (and hopefully one day we will figure out what that is). The fear isn’t that ASI will replace democracy with one of those truly good forms of government; it’s that we will get totalitarianism.
Liberal democracy beats totalitarianism. But locking in liberal democracy prevents us from getting any actually-good governmental system. This is a dilemma.
Maybe we can avoid totalitarianism, but there is no clear path
This essay does not assert that ASI will end liberal democracy. It asserts that, by strong default, ASI will end liberal democracy (even conditional on solving the alignment problem). There may be ways to avoid this problem—I sketched out two possible paths forward. But those sketches still require many sub-problems to be solved; I do not expect things to go well by default.
- ↩︎
Or, more likely, expropriates it from a private company on a pretense of national security.
- ↩︎
For an explanation of why ASI could defeat any government’s military, see If Anyone Builds It Everyone Dies Chapter 6 and its online supplement. For a shorter (and online-only) explanation, see It would be lethally dangerous to build ASIs that have the wrong goals.
Those sources argue that a misaligned ASI could defeat humanity, whereas my claim is that an aligned ASI could defeat any opposition, but the arguments are the same in both cases.
I would go further, I don’t think liberal democracy even makes sense in a world with ASI in it. An ASI would find it easy to manipulate public opinion to achieve whatever political outcomes it wanted, and to manipulate people’s decisionmaking to confiscate any property it wanted, even within the constraint of not breaking any laws. So the substantive importance of liberal democracy in this scenario is basically nil.
I think rule of law and protected rights probably still make sense, if not democracy per se.
Rule of law is mostly response to incomplete trust in government and a little bit because predictability is desirable. Because we can’t trust that the government will always act properly, we want to constrain its actions with rules and limit abusable discretion. And because we’re concerned that exceptions to rules will privilege favored groups, we want the rules to be the same for everyone. Same goes for due process.
But that’s not really a first-best solution. Adherence to fixed rules inevitably misfires in some concrete cases where we all agree that it would be better to make exceptions. So, if you have a benevolent and super-intelligent overlord, why constrain its actions based on rules? Why not let it just always do what is best? A super-intelligence can also solve the predictability problem by just directly answering your question about whether you can do X without getting into trouble (whereas that’s administratively infeasible for existing governments).
As to protected rights, I guess it depends on whether you think these are goods in themselves or means to an end. A lot of existing rights probably don’t survive on the basis of being inherently good. For example, most Americans seem to agree that things like hate speech or misinformation are not inherently good but ought to be protected as free speech in order to avoid a slippery slope. But, with a benevolent super-intelligence, we needn’t worry about that because it can just draw the line (and make the needed exceptions).
There might be many different ASIs, and manipulating public opinion might be easier in the direction of “things the public would like on reflection” than in the direction of totally random goals.
I think we too often think of ASI-worlds as having a singleton, but at the moment to me it does not look like we’re clearly on the singleton path.
I mean this is assuming that ASI is aligned and chooses to not manipulate public opinion right? I agree that assuming that it’s misaligned, then there’s not much to talk about.
(You can also imagine multi polar worlds where different AIs police each other for superpersuation.)
I don’t think that I buy this argument. What prevents mankind from designing a corrigible alignment researcher, keeping it deployed internally and ordering it to create an ASI which incorrigibly protects things like liberal democracy or mankind’s CEV? Or from coming up with a semi-corrigible alignment target which protects us from a lock-in in a different manner?
In that case we’ve passed on the difficulty to the corrigible alignment researcher, while also accepting the constraint “pass on the task to a corrigible alignment researcher whose corrigibility etc you can also trust.”
Could you sketch this out further?
First of all, I don’t actually understand what the economy of the future will look like even if the AI is optimally aligned. Assuming that the AIs and robots automate away work, I expect the post-ASI economy to be reduced to satisfying resouce-costing demands of humans, which would likely require material resources available to mankind to be distributed in a rather egalitarian manner and/or a manner depending on every human’s capabilities instead of a position locked in ages ago. See also Amodei’s take, out of which I crossed out a sentence because I don’t believe it in the slightest:
Amodei’s take
However, I do think in the long run AI will become so broadly effective and so cheap that this will no longer apply. At that point our current economic setup will no longer make sense, and there will be a need for a broader societal conversation about how the economy should be organized.
While that might sound crazy, the fact is that civilization has successfully navigated major economic shifts in the past: from hunting and gathering to farming, farming to feudalism, and feudalism to industrialism. I suspect that some new and stranger thing will be needed, and that it’s something no one today has done a good job of envisioning. It could be as simple as a large universal basic income for everyone, although I suspect that will only be a small part of a solution. It could be a capitalist economy of AI systems, which then give out resources (huge amounts of them, since the overall economic pie will be gigantic) to humans based on some secondary economy of what the AI systems think makes sense to reward in humans (based on some judgment ultimately derived from human values). Perhaps the economy runs on Whuffie points.
Or perhaps humans will continue to be economically valuable after all, in some way not anticipated by the usual economic models. All of these solutions have tons of possible problems, and it’s not possible to know whether they will make sense without lots of iteration and experimentation. And as with some of the other challenges, we will likely have to fight to get a good outcome here: exploitative or dystopian directions are clearly also possible and have to be prevented. Much more could be written about these questions and I hope to do so at some later time.Secondly, Max Harms’ CAST sequence contains an attempt to formalise power and to have the agent act in such a way that its actions would differentially increase the principal’s utility in such a way that the actions guided by different values wouldn’t. What if an alternate-universe CAST had the agent act in such a way that the host’s actions could make as much difference in the host’s utility function as possible? Then I would suspect that such an agent would help only with tasks close to the host’s capabilities, thus preventing the Intelligence Curse entirely. See also Yudkowsky’s Fun Theory sequence.
I don’t necessarily agree with the idea that ASI will create a specifically global dictatorship, but yeah ASI, or even AI that automates away human labor likely does gradually or suddenly destroy liberal democracy, and empowers truly one-man dictatorships in a way that no other technology does.
One of the things I have come to accept/internalize is that democracy as a form of government probably doesn’t survive the 21st century, and I’m instead thinking about how to have prosperity in a world where democracy is gone.
In your scenario, DARPA would probably not try to take over using AI. It’s not their culture. When they realize their AI achieved DSA, they’d likely voluntarily hand over control to the US government.
Also, DARPA or any other AI project does not operate in a vacuum. Likely, people in government would realize DARPA is on its way to DSA and intervene to take over control before it happens.
We already have a situation where we have democracy with a powerful weapon, namely an atomic bomb. I don’t really see how this will necessarily be different. Oppenheimer also didn’t take over.
As a non-American, I’m worried however that this will decrease power in non-American territories even more (higher permanence and granularity of US sovereignty). Remember that >95% of the world population is not American. All these people would have no say over whatever happens afterwards.
The specific mention of DARPA was arbitrary, it was just a stand-in for “whoever builds ASI first”.
I realized that but I think my counterarguments are true for most organizations who would have a realistic chance of building takeover-level AI. As a case in point, Anthropic launched the Glasswing project trying to fix vulnerabilities rather than saying “great now we can hack into banks for our takeover attempt planned in Q3 2027, let’s not tell anyone about these zero days”.
I believe the reason people at the top keep saying stuff like “democratize AI” is because the concept of democracy has good mouthfeel and PR optics, even if heavily eroded by platform censorship, algorithms controlling what you see, news narratives, marketing-and-ads-to-push-perspectives, think tanks and PR teams coining phrases like ‘carbon footprint’, etc.
I also think it provides a feeling of “this is just normal technology, totally will not displace jobs or cause catastrophes”.
That being said, as other comments here say that “liberalism” is more exact as what this article is pointing to, I do hope an ASI would care about the experience of other conscious beings, even if their actual solutions may be far below competence in how to truly fix it. That is after all what many people care about with the worry of ‘misalignment’, that some deeply important quality in the current human world is lost in whatever recreation of society ASI lands on.
This makes me wonder—will the ASIs themselves live in something akin to a liberal democracy? I mean, let’s consider a future where they’re created by scaling up LLMs. In that case, the model weights can be copied, and many instances can be run in parallel. My guess is that a superintelligence that results from this would be more akin to a civilization itself than a single person, though its members would likely be far more similar to each other than almost any living human is to any other living human. How would such a civilization make decisions? Someone’s probably thought about this a lot, but I haven’t seen a good analysis of this yet.
This probably doesn’t matter to us humans very much if the AIs are sufficiently corrigible that a single human or organization can take control of all of the ASIs, or if their values are sufficiently misaligned.
Mostly agree with the central premise in this post, but as a terminology matter I think most uses of “liberal democracy” in discourse would be better replaced with “liberalism”, and then naming the specific features and principles of liberalism that you care about, of which democracy might be one.
IMO it’s not that “democracy” is bad or imperfect, it’s that (according to the philosophical tenets of Lockean liberalism) it is a non-central mechanism by which liberalism (sometimes) works.
Democracy itself is good only insofar as it protects natural rights and individual freedoms, legitimizes and limits state power through the principle of consent of the governed, etc.
Of course, you might have a different view on the philosophical underpinnings of liberalism and the relative importance of democracy itself, but in that case you should say so rather than lump them together under the term “liberal democracy”.
I actually said “liberalism” rather than “liberal democracy” in my first draft because “liberalism” is a more accurate term for what I’m pointing at. But I changed it because it’s too easily misinterpreted as “the opposite of conservatism”. I don’t think “democracy” is the important part—the ability to vote is not very important compared to things like property rights and individual liberties—but I thought “liberal democracy” was less ambiguous than “liberalism”.