Often I see people claim it’s essential for America to win the AI race against China (in whatever sense) for reasons like these:
“What is the reason we want America to win the AI race? It’s because we want to make sure free open societies can defend themselves” (Alec Stapp)
“We should seek to win the race to global AI technological superiority and ensure that China does not… to ensure that our way of life is not displaced by the much darker Chinese vision“ (Marc Andreessen)
“Will it be one in which the United States and allied nations advance a global AI that spreads the technology’s benefits and opens access to it, or an authoritarian one, in which nations or movements that don’t share our values use AI to cement and expand their power?” (Sam Altman)
“In Machines of Loving Grace, I discussed the possibility that authoritarian governments might use powerful AI to surveil or repress their citizens in ways that would be extremely difficult to reform or overthrow. Current autocracies are limited in how repressive they can be by the need to have humans carry out their orders, and humans often have limits in how inhumane they are willing to be. But AI-enabled autocracies would not have such limits.” (Dario Amodei)
“The torch of liberty will not survive Xi getting AGI first. (And, realistically, American leadership is the only path to safe AGI, too.)” (Leopold Ashenbrenner)
“whoever wins the race for AI, that nation’s values are going to be reflected in AI. If China wins the race for AI, AI will be a tool for global surveillance and control as carried out by a communist nation” (Ted Cruz)
Those claims slide between a few different actual threat models:
Government Capture by China: China will overthrow and control the US government, maybe as part of general domination of the whole world.
Defeat in Cold War: China will have greater wealth and prestige, so just as our prestige inspires many parts of the world to adopt our way of life today, much of the world will adopt the Chinese governance and cultural models instead.
Protection From Our Conquest: China will fortify its own regime, so that it can’t be overthrown, whereas if we win the AI race, we can promptly overthrow the Chinese government and replace it with a new regime aligned with our values.
The Dario quote points to (3) with unusual directness. The “race rather than slowdown” ending of AGI 2027 also supposes that our AI lead will create interest in overthrowing the Chinese government. But most of the quotes I gave as examples above are interpreted as (1): that an AI-enabled Chinese government would overthrow Western governments.
My main point here is that (1) seems unfounded to me. China is not an aggressive nation at all. As far as I can tell, China has literally never attacked a non-bordering country in its entire history, nor have they ever tried to overthrow a foreign government by covert or manipulative means. China is also unique among nuclear powers for its unconditional no-first-use policy, which at face value implies they would withhold a nuclear response to even an overwhelming conventional invasion. Further:
The Chinese haven’t built a network of military bases abroad or binding military alliances; they have a single foreign base in Djibouti and a single mutual-defense treaty with North Korea. In contrast, America maintains over 700 bases and a huge alliance network with NATO and the Asian military allies Japan, South Korea, Australia, and the Philippines.
Chinese military spending is 1.7% of GDP, versus 2.1% for France and 3.4% for America. Chinese foreign-aid spending is 0.07% of GDP versus the much larger 0.8% for France and 1.2% for America.
China has almost no history of covertly backing palace coups abroad, in contrast to America, Russia, and France.
More broadly, China is a very inward-looking country compared to other major powers. Only 0.1% of Chinese residents were born abroad, much fewer than the 15% in America and 14% in France, fewer even than the 0.3% and 3% in India and Japan respectively. The Chinese government has peacefully compromised on almost all border disputes in central and southeast Asia, often taking a minority of the contested territory. (The Indian border is the exception.)
In particular, there’s less history of Chinese overseas expeditions that are motivated by ideology alone. The British Empire famously destroyed overseas slavery at great cost to themselves, and the Americans and Soviets interfered in foreign regimes and civil wars, all for basically ideological rather than self-interested motives. The use of power for this kind of crusading is much less precedented in China.
To many American voters and elites, tracing back to Woodrow Wilson more than 100 years ago, “the justification of America’s international role was messianic: America had an obligation, not to the balance of power, but to spread its principles throughout the world” (Kissinger). That isn’t the historical attitude of the Chinese government, whose leaders perceive foreign intervention or expansion as threatening to Chinese identity and culture.
American exceptionalism is missionary. It holds that the United States has an obligation to spread its values to every part of the world. China’s exceptionalism is cultural. China does not proselytize; it does not claim that its contemporary institutions are relevant outside China.
— Kissinger’s On China
It’s true that China doesn’t practice liberal governance. The core of liberalism is freedom of contract, limitations on government interference, and equal access to independent courts. In China, the CCP explicitly rejects limited government and exercises highly invasive control over business, speech, association, and religion. In China there’s no private ownership of land and no independent judiciary.
If you think it’s prudent to disable and overthrow the Chinese government when it becomes achievable militarily, then that’s certainly one (bellicose) position you could hold. Then you could say that a downside of losing the AI race is that the CCP may defend itself. But it’s unwise to project this ideological aggression onto the CCP itself without evidence.
Addendum: It would have been mistaken for a European to say, in 1895, “Who cares about American industrialization? They have almost no army and have barely left their far-away continent.” Soon afterward that European might find the Americans replacing his regime or dismantling his empire. So a counterargument here is that in general, countries that become wealthy and militarily powerful become aggressive regardless of how passive they seemed before. Under this reasoning, China has had limited imperial ambitions in the past only because it e.g. lacked naval superiority. This has to be an argument based on a general view of human nature and government.
The epistemics at work here...
The “non-bordering” part is doing an enormous amount of work here. Choosing only to attack bordering countries is perfectly compatible with conquering much of the world. Arguably, if conquering the world were a country’s sole goal, then going after immediate neighbors one at a time is usually the best strategy.
Hong Kong? Taiwan? (In both cases, the “foreign” part is kind of in dispute, Hong Kong more so.) The fact that the post doesn’t mention Taiwan at all raises my eyebrow.
The fact that China lets in very few immigrants does not provide evidence against the hypothesis that they think they’re better than everyone else and that the world should belong to them. If they also had near-zero emigration and trade with the outside world, then the term “inward-looking” would fit. But this is about as far from “inward-looking” as you can get: https://en.wikipedia.org/wiki/Belt_and_Road_Initiative
This is understating the case. The post does not mention disappearing dissidents, Uyghurs, or reeducation camps. I’ll also point at CCP attempts to police speech across the globe, such as by pressuring companies to not mention Taiwan as a nation separate from China.
While that is true, even in that case I don’t think China would be a threat to the United States based on an analysis of the world map.
The Chinese disputes with Taiwan, or on the Indian border, aren’t a threat to the West. Likewise, the Chinese conflicts involving Uyghurs, Tibetans, and to a lesser degree their maritime borders don’t reflect the kind of foreign meddling that I’m talking about, which is ideologically motivated overseas intervention or conspiracy of the kind practiced by America, the Soviet Union, and the British and French Empires.
This is also not the kind of “universalist missionary ideology” behavior I’m referring to. My understanding is that Belt and Road is a bunch of construction ($800B) and investment ($600B) contracts and that the largest recipients are Pakistan, Saudi Arabia, and Nigeria, and then other recipients include Indonesia, Iraq, and the Congo.
Random details:
You didn’t say this outright but I would flag that “thinking you’re better than everyone else” doesn’t imply “thinking the world should belong to you.” Many cultures think they are the best.
I do agree with a counterargument you didn’t make, which is that China is just militarily and especially navally weak, but that basically all countries that become rich and powerful become aggressive even if they’re peaceful beforehand. One could say that in 1895 it would be wrong to predict “Who cares about American industrialization? They have almost no army at all and have barely left their far-away continent” when in reality Americans would be dismantling their regimes and empires after becoming more powerful. I’m not sure how to evaluate this general notion but maybe it’s true.
China siezing control of Taiwan would in fact be a threat to the West, it’s the primary point of discussion as far as I’m aware given Taiwan’s dominance in semiconductor manufacturing. This isn’t some in the weeds point either, this is like, the main point of geopolitical contention and a constant point of discussion.
This kind of “foreign policy via psychologizing a whole country” seems like a very strange way to think about geopolitics and I would imagine a very bad predictive model compared to like “what gives one superpower strategic advantage over another”.
I don’t think China wants Taiwan due to strategic advantage over other superpowers. I think the PRC is actually approaching the situation with Taiwan much more cautiously than they would in the alternative world where semiconductor manufacturing does not take place in Taiwan, due to its geopolitical importance and the concomitant risks.
It is worth keeping in mind that reunification is a live debate even in Taiwan, though certainly most Taiwanese would rather not reunify. But the reasons that motivate China to want Taiwan are pretty unique and are significantly less likely to generalize than e.g. the reasons the U.S. wants Greenland, for example.
This is compatible with almost any model though
This is very obviously (conditional on them taking decisive military action), why they would be willing to take action despite retaliation from the United States.
I think a historically well performing prior for modeling the foreign policy of superpowers is “what is in their geopolitical interest”. I’d privledge it pretty highly over arbitrary narratives, as psychologizing like “America has a missionary ideology” for say “democracy” would lead you to making a lot of bad predictions about American foreign policy. Superpowers in general operate in a pretty realpolitik way, like I don’t think it’s worth it trying to dissect “why does Putin say the invasion of Ukraine was justified actually”.
I don’t think you can accurately predict what superpowers do when you ignore their internal politics.
If you take the invasion of Ukraine by Russia it’s worth understanding why Putin gained an increase in both domestic approval and power through it.
Oh yeah definitely agree with this, should’ve included that as well. I generally was trying to point towards “I think a realpolitik model of thinking about foreign policy is a better predictive model than trying to ascribe broad traits to a whole nation like ‘how inward looking are they’ or looking at stated motivations by leaders without interpreting that via the incentive structures at play”.
I found this post was pretty disappointing in its argumentation, for reasons you describe, and I fairly strongly support its conclusion.
Why, though? What are the arguments that make you support the conclusion, if the post was disappointing in its argumentation? If you mean that you’re convinced by other arguments in the post not addressed by Localdeity’s comment (e.g., the argument from military spending and foreign bases, or Kissinger’s articulation of the difference between missionary and cultural exceptionalism), it would be more informative to say which ones, because rationalists care about what arguments and evidence exist, not which conclusions to support: unless you can establish yourself as an authority whom people should defer to without evidence, there’s no reason anyone should care what conclusions you support, which means your comment is lowering the forum’s signal-to-noise ratio and is therefore a bad comment.
I’m a supporter of helping protect Taiwan, but it’s useful to know the Chinese view is that Taiwan is a part of their country going back to the civil war, one they have never officially conceded as having ended and have never acknowledged Taiwan as anything but an internal issue to them.
Sure at some point it gets kinda silly to keep insisting “someday we’ll get them” instead of just giving up, but that does mean it’s not a good example of China being hostile towards foreign powers.
Without commenting on the evidence or the conclusion, I just want to say that the link between evidence and conclusion seems extremely weak—I’m pretty sure the Mongol hordes also never attacked a non-bordering country yet it would be laughable to call them non-aggressive.
They did, they launched naval invasions of Japan and Java.
For what its worth, China has also launched multiple naval invasions throughout its history (though relatively few considering how long the civilization has existed). Two examples:
Sri Lanka (Kotte Kingdom)
Taiwan (Dutch Formosa)
Cool! I didn’t know Zheng He invaded Sri Lanka. Arjun should update the post.
Good post, thank you for opening this discussion.
I’ve long been waiting for the rationalist community to have a “Sword of Good” kind of reckoning about the US vs China, as in “choosing between good and bad is about deciding which is which”. To lay out some rails, I think there are two general approaches to this question.
1) The objective approach. Here you find some numbers, like percent of healthcare coverage and so on, and compare the two countries on these numbers. I’ve given this a lot of thought and believe that it comes down to two “topline metrics”, which ought to weigh more than any other metric when deciding a country’s moral worth. They are: how many people the country represses internally, and how many people the country kills in foreign wars. These are the topline metrics for a government being good domestically and being good internationally. For example, Nazi Germany had horrible repression internally and started horrible aggressive wars, and that’s the entire reason we think it was bad. Well, today’s US has much higher incarceration rate than today’s China and also kills much more people in foreign wars, so there’s that.
2) The aspirational approach. Here one can say that China is simply nationalist, while the US has a more universalist mission to spread democracy and free speech and so on. There’s something to be said for this view, but I think it’s getting less true over time. The last democracy-building success stories were many decades ago, the more recent US wars have just been messing countries up. And whether the US cares about universal human rights is also a bit of trick question. For example, Anthropic recently said its red line was “mass domestic surveillance”. When I pointed out that I as a non-US person shouldn’t be surveiled either, multiple US folks (including AI lab employees) told me right here on LW that I should be more ok with being surveiled by the US. It gave me the impression that my rights aren’t quite as real, and that I should expect today’s US to simply follow nationalist interests when it comes to me, not be a light unto me or something.
Is your claim that non-imprisoned Chinese should be considered non-repressed, or at least comparably repressed to Americans?
I think with these kind of subjective things you can fudge a lot. “The Chinese are repressed” “Sure, but they have healthcare” “Sure, but they have Uyghurs” “Sure, but you have reservations” and there you go again. It’s the same problem that effective altruism was trying to solve, that the “society for curing rare diseases in cute puppies” is not effective, so you’ve got to find numbers that are hard to fudge. Or with measuring crime, theft reports are affected by a lot of factors and you need to look at murder rate which is harder to fudge. Same logic here. You’ve got to choose some objective numbers that reflect reality.
Given that, I stand by my choice of numbers. In Stalin’s USSR, Nazi Germany and Maoist China the internal repression was reflected in incarceration rates. But in today’s China, the incarceration rate is 1/5th that of the US. This means the repression in today’s China is much lower than these historical examples, and not obviously worse than in the US.
Have you looked into:
Financial repression (disposable income as % of GDP, limits on currency conversion)
Censorship level (% of movies shown in theaters that are pre-approved by government censors, % of foreign websites that are accessible)
Population/movement controls (not sure how to measure these, but e.g. “one-child policy” and hukou household registration system, which have no counterparts in the US)
After thinking for a few days, I agree with your objection. Incarceration rate is a signal but not as strong a signal as I thought, unless it spikes really high. (Still feel that foreign wars are a strong signal though, I wonder if you have objections to that too.)
China has little ability to project power across the ocean so it can only have wars with its neighbors. Nevertheless, since CCP took over in 1949 it has fought in
Korean War (1950–1953)
Sino-Indian War (1962)
Sino-Soviet Border Conflict (1969)
Sino-Vietnamese War (1979)
with total deaths of ~190,690 – 427,522 for China and ~207,941 – 240,988 for its enemies (according to Gemini Pro). This is roughly half of the total deaths of wars the the US has fought across the globe during the same period, but OOMs more than the US if we only count its neighbors (say the western hemisphere).
To put it another way, in order to compare their willingness to fight wars, you have to factor out their capabilities (including e.g. alliances and vulnerabilities to counterattacks and sanctions), after which it’s far from clear that China is less war-like than the US.
I’m 43 years old and the last of these wars ended before I was born.
Also, I wonder about your comparison of deaths. You say 400-700K killed in wars involving China. If we look at the biggest wars involving the US in the same period, the Korea war killed 2.5-4M, the Vietnam war killed 1.5-3M, and the war on terror killed 1-4M and is ongoing (all numbers from Wikipedia). Maybe the US is responsible for only double China’s number, or maybe more. I guess it depends how many deaths should be blamed on the US and how many on others.
Do you know the Hide your strength, bide your time quote (commonly attributed to Deng Xiaoping, who didn’t actually say it (his successor did) but did apply a similar policy)? If we apply an instrumental convergence / deceptive alignment lens on this, we have to conclude that we can’t really make any conclusions based on surface behavior, about how “aligned” (peaceful or warlike) a country is, while it’s below some capabilities threshold, at least if it’s collectively smart enough.
(I thought of mentioning this analogy in my previous comment, but it seemed too obvious to spell out explicitly. Did I underestimate the inferential distance? Do you still disagree?)
No country is aligned, and in particular neither China nor the US is aligned to me (a Russian emigrant). They’d both treat me badly if it was in their nationalist interests.
Mostly I’m trying to make the same point as the OP: many people, including LWers, are fanning up the military and AI arms race on the US side, and in my opinion there’s no strong justification for this.
I agree that there’s no strong justification for this, but would make other arguments in this direction rather than point out the data that you did (China caused fewer deaths in external wars), mainly that AGI/ASI is likely to be either unaligned/uncontrollable or radically change the power structure and/or values of whatever country “wins” the AI race, so we can’t form strong judgments based on past evidence.
BTW who on LW is fanning this race? If I try to recall what I’ve seen, it’s mostly about trying to slow China down (through things like export controls and securing American AI labs) rather than speed the US up, which seems unobjectionable? Do you see LWers doing more than this, or think this is objectionable too?
If Alice and Bob are doing something that might be plausibly interpreted as a race, and Alice is not
[doing the things that you’d expect her to do, if she wanted to move faster, if she thought about the situation as a race],
but she is
[doing the things that you’d expect her to do, if she wanted to slow Bob down, if she thought about the situation as a race],
then this should make an impartial observer Charlie increase his credence that at the very least Alice (if not Bob) is thinking about the situation as a race.
Fair. I choose the number of legal oppositional parties and the number of peasants prevented from migrating to cities by the hukou system.
The former is obvious, the latter is a spicy take. Of course, the US isn’t exactly having its best moment these days, but I still doubt that the percentage of Americans who would prefer living in China would be even within one order of magnitude of that of hopeful Chinese immigrants.
Why are your numbers more indicative, though? There’s tons of possible numbers to choose from that go every which way. When I was figuring out for myself how to compare countries, I tried to choose numbers that were hardest to fudge, most indicative of bigger trends and least cherrypicked. That seems like the only way to clear things up, like defining QALYs in EA.
About migration as a criterion, consider Filipino migrants in Saudi Arabia. The Philippines are a flawed democracy, while Saudi Arabia is an absolute monarchy with a poor human rights record and a higher incarceration rate. Why the migration then? Economic reasons. If China becomes as rich per capita as the US, I’m not sure they’d be as eager to migrate. Migration often depends on which country is richer, not which country is more free.
I’m not claiming that they’re more indicative, but I do claim that they aren’t obviously less indicative. Since everyone can cherrypick, it’s not clear in what way are objective numbers better than vibes anyhow.
IMO CCP-led China will never come close, and the level of repression is an important factor of that. Small-ish petrostates aren’t relevant here.
Under the leadership of the Chinese Communist Party, China has achieved miraculous growth and eradicated absolute poverty despite a similar level of repression. What exactly leads you to believe that, under such repression, China could never reach the level of development enjoyed by the United States?
Why not consider what might happen if the repression were relaxed? For example, what if the household registration system and the detention and deportation system were abolished? Rural residents lacking basic urban job skills would flood into cities in a disorderly manner and establish slums, which would severely hinder urban planning and trigger a deterioration in public order. At the same time, a significant portion of the government’s main revenue comes from land sale proceeds. Without these systems, the government would be unable to generate revenue from urbanization and use it for critical infrastructure such as high-speed rail, which would in turn severely impact economic development!
My impression is that things are as relaxed as they are going to get, and the trend these days seems to be in the direction of increasing repression.
China and the UAE are, in fact, quite similar: the UAE’s GDP comes primarily from non-oil sectors, and these industries rely heavily on the kafala system to bring in cheap labor from the Third World. Meanwhile, at least over the past few decades, China’s developed coastal regions and major cities (such as Shanghai) have developed labor-intensive industries to take on international industrial outsourcing, while its densely populated inland regions (such as Henan) have served as internal colonies and sources of cheap labor.
i’d guess that the incarceration rate among chinese americans is at most roughly as large as the incarceration rate in china though. [1] controlling for the two countries having different people seems important if we’re trying to assess the repressiveness of each country’s governing system. (that said: chinese americans are also richer than chinese chinese, and one would want to control for that as well, introducing a correction in the other direction)
(that said: my overall position is that it is very bad for the US to race with china)
chatgpt guesses that the chinese american incarceration rate is about 2-3 times lower than china’s
This reminds me of the Tucker Carlson interview with Russian President Vladimir Putin. Putin’s discussion of Russian and Ukrainian history seemed bizarre and unmotivated to some of my friends, but it seemed to me like the obvious intent was to explain how he understands Russia and Russia’s legitimate interests, in order to draw an intelligible distinction between aggressive and defensive acts along lines that might not otherwise make sense to foreigners with different assumptions, so that we wouldn’t be forced to construe Russia’s campaign in Ukraine as aggressive.
He was trying to explain how not to have a domino theory about Russia in Ukraine.
Other commenters have complained about your “only ever attacked people sharing a border” threshold. The map of the world doesn’t really help the US. Not only would we be pretty disturbed if China conquered all of Eurasia going border-by-border, but a small country in the Americas participating in the Belt and Road initiative could invite China to establish some sort of protectorate, after which China would only have to traverse a series of borders to get to the US. Likewise, saying that the Uighurs in Xinjiang are within China’s borders doesn’t help; the Tibetans didn’t always used to be and now they are, except for the refugees in India, which now shares a border with China.
But of course it’s equally no good to assume that a state will behave aggressively just because it could. What you would need to develop to be persuasive here is the same class of task Putin attempted when speaking with Carlson: some idea of how Chinese state decisionmakers understand themselves, and their national interests, that credibly constrains anticipations so that we can have some sense of what sorts of actions they are and aren’t likely to take.
For instance, many Americans would likely assume that “no attacks without shared borders” would preclude an invasion of Taiwan by the People’s Republic of China, but in fact both governments notionally agree that they’re in the same country, so to the Chinese, it would be intraborder, not even cross-border.
My threat model is actually
More of the lightcone will be controlled by the Chinese government (or its successor).
My current guess is that the long-term future looks better if American actors have more bargaining power over the long-term future than Chinese actors. If space is subdivided amongst the key actors (including the Chinese government, the U.S. government, and possibly others), I worry that the parts controlled by the Chinese government would be illiberal, in the same kinds of ways that China is now. One particularly bad version of this is an AI-enabled surveillance state that locks in something like current Chinese ideology, curtailing the potential for moral improvement. I think this is less likely in the parts of space controlled by the U.S. government, because I think those are reasonably likely to be founded on fairly liberal values, perhaps similar in spirit to the U.S. constitution.
I didn’t disagree-vote and I’m not sure what those people are disagreeing about.
I think your point is reasonable and novel among the comments or Twitter replies. I don’t have a strong take on “lock-in,” though; I guess it’s definitely possible to imagine some technology that creates lock-in but in general I think it’s overrated. For example, I don’t think mass surveillance per se creates lock-in for governance, and I don’t think the American Constitution has “locked us in” to its words as much as others say.
I do think that governance in general changes over time as technology changes in somewhat deterministic ways, e.g. I think that it intuitively seems correct that over the last 100 years democracy and parliamentarianism has gradually been replaced by administrative rule because the balance of military power has changed: masses can’t be used by counter-elites to threaten the state the way they could during the age where amateur riflemen were an important military power, roughly between the American Revolution and Spanish Civil War.
Note that this claim itself is an example of illiberal values. Compare, for example, a historical European Protestant being hostile to Catholics because most of European GDP is controlled by Catholics. One core premise of liberalism is that if you’re able to productively generate wealth in a non-aggressive way, then others shouldn’t interfere with that. Implicitly, liberalism treats growth as positive-sum, as opposed to the zero-sum mindset of “what proportion of wealth do I control?”
Now, it’s a coherent position to say that AI is sufficiently important and powerful that it justifies exceptions to standard liberal principles, but I do think that it’s important that people are explicit about making this move. You can also hold this position and try to only use strategies which are consistent with liberal values, but that seems like a slippery place to be (e.g. are export controls a good example of acting according to liberal values?)
By “non-aggressive” are you only talking about external aggression (i.e., towards other countries) or also making a claim about internal repression (similar to cousin_it’s claim that it’s no worse than the US)?
I think this is probably right overall, but it’s worth noting that from an outsider’s perspective it seems that the difficulty of amending of the U.S. constitution seems to have in many ways hampered the potential for moral improvement in America, rather than fostering it.
I think both
Our Constitution has been functionally amended by very large (often gradual) changes in jurisprudence over time, so it’s changed more than an outside observer would guess based on the amendment process alone.
The written Constitution actually has preserved the morality of our system and made our country more successful because it promotes limited government via the federal system as well as making it easier to develop strong positions related to some of the Bill of Rights. But it’s hard to separate this from American culture perhaps, due to (1).
I just want to mention that I think the question to ask here is something like what predicts best the future of a country and what are the things that generalise, how do you predict a country well?
I would like to recommend that people read a longer history of China so that they can understand the underlying cultural forces at play since that shapes identity quite a lot. Identity is often carried forth and is a key underlying factor that should be taken into account together with game theory among other things.
You do not predict other human beings as fully rational, you understand that their wants and needs are shaped by their history and generally a good predictive system is to find what things the country identifies with.
What are the historical driving forces of China? How did they transform and how do they show up across the political spectrum in China today? What is the Chinese cultural identity? What are the economical and cultural forces shaping its future?
I’ve spent 50 hours or so on Chinese history which whilst not a lot in the larger scheme is probably more than most here. Whilst I’m sorry for the ethos argument, I mainly want to point out that you should go do it yourself and come back with a view on what chinese cultural identity is like and what predicts future nation actions well. I don’t really care what you think as long as you spend some time thinking about it more deeply instead of dismissing a more “naive” viewpoint out of hand.
I would like to back Arjun’s point up here and I think that the pure game theory viewpoint is a bad predictor of actions in this case. It just isn’t a localised game where AGI is all that matters, people and states do not think that way and so I don’t believe that we should hyperstition it into existence that way.
I’ve spent hundreds of hours on Chinese history and still barely know what I’m talking about when it comes to modern China. Especially when it comes to geopolitics, I don’t think there’s that much crossover from the imperial, pre-modern China to today’s China. The geopolitical situation is totally different, China’s institutions are totally different, and modernity transforms everyone in the same direction to a significant degree.
Okay, fair enough.
When you’re saying this I’m wondering what level of granularity that we’re talknig about?
I feel that some of the economic actions and political actions that we’ve seen in terms of religious persucution, and similar have happened in the past and it is something that we should predict to continue happen within china in the future? Of course there’s a modernisation push there but that doesn’t mean that China will become expansionist all of a sudden. Of course you can’t predict things on a granular level but do you believe that it doesn’t help on a larger level? If that is the case, why not?
I don’t think it helps that much on a larger level.
To take your example of religious persecution, I don’t think there’s a meaningful historical trend there. The most infamous religious persecution in Chinese history was Daoist persecution of Buddhists, but China’s relationship with Buddhism was complex; several emperors were predominantly Buddhist, and Tibetan Buddhism in particular held huge sway over the court at times (also over steppe peoples). This doesn’t really map cleanly to modern treatment of Tibet’s religious institutions in my view, besides the general desire of every government to control religious sources of power. (see: Church of England, Oda Nobunaga vs. Enryaku-ji, Saudis and Mecca, etc.)
There was a degree of Christian persecution as well, esp. later on when it became identified as a tool of Western power, and there’s a degree of Christian persecution today as well. But is this a meaningful continuity? Consider three culturally similar countries: China, Korea, Japan. All three historically persecuted Christians for basically the same reasons. Can we extrapolate the same modern behavior for all three? Of course not, the actual outcome was heavily dependent on what path the country followed into modernity. South Korea is ~30% Christian, Japan is ~1% but not persecuted, North Korea is negligible and strongly persecuted, China is ~3% (officially anyway) and somewhat persecuted.
Fair enough, I’m also realising that I probably can’t prove and that I probably shouldn’t rely on my intuition in this case unless I start doing some forecasting stuff for china and my brier score turns out to be good for future actions.
So I think you’re right in that it is a general tendencies for governments who want more power and governments who are newcomers and want to break from the past traditions are more likely to do this.
You also look at the idea of a chinese resurrection after their perceived decline from 1850-1950 and see that you want a break from the past but what is then that past identity that they want from it? I think it partly has to do with their close knit relationship to buddhism and similar as it is one of the beliefs that aren’t instrumental to power (whilst something like confucian beliefs are more instrumental to power).
Now, of course things like game theory, like the general change that modernisation implies, things like the economics of AGI are going to be big and important factors for any type of strategic thinking. I think culture is also a predictor and it helps to understand the basis for values and interpretations in a society and knowing chinese history is getting a deeper understanding for the forces that shaped their culture. I think in the light of that it is easier to see something like a break from the past happening specifically with religious persecution as it is the opposite of what has been the culture before since they had to break up with their past identity (of losing) to one of unity and strength.
This might just be me retrofitting my narrative and so I should stop talking until I prove some degree of predictive accuracy or at least I have evidence to show someone doing this and improving their predictive accuracy. The feeling I got was one of being less scared and more understnding but that might be wrong.
Edit: Also thanks for being concrete and going to an example, I like that.
Yes, this is all true and really important, regardless of what you think about China re: AI policy.
I think a missing mood in a lot of discourse is that the US has historically been an imperfect standard bearer for these principles, and today is straying further from them in alarming ways. But that doesn’t make it less important that the CCP explicitly rejects them. Conversely, there are some things the CCP can be admired for in terms of state capacity and effectiveness. “Communism with Chinese characteristics” is actually pretty good in some ways; they just need to drop the “communism” part! This is of course extremely unlikely / unrealistic, but if they could just pick some better 19th and 20th century European intellectuals to base their political philosophy around, the whole world would be better off...
as the author says
glancing at the details of how this works, one could stipulate they’re already one third of the way to George
Implicitly I read you as most notably as (and please permit simplification and glossing over many additional points an subtleties, in order for me to later get to my biggest concern): “Look, China has been rather benign on a world stage, suggesting this may rather likely well stay so.”
But: To strictly derive, benign, kind intents, you’d have to claim that Chinese policy would have been unsmart for an ultimately self-serving regime or elite or population. Else your observation has very little epistemic content other than maybe to confirm: they were smart.
So: Would China, or, say, Xi Jinping or its ruling elite, or whaterver the relevant entity, really have had much to gain from aggressing countries in the meantime? Or, say, from aggressing them more than they did[1] in the past?
My “concern”: MAYBE RATHER NOT!?
Thinking about how much China was able to cumulatively grow, amass power and influence over the past decades, it seems to me rather really difficult to claim they did ‘much wrong’ in terms of pursuing an aim of becoming wealthy, powerful, important on the world scene.
Suggests: Non-aggression[2] fully paid off. That is of course an interesting lesson you could say. That it’ll also have to pay off, in a narrow material or so sense, once you have AGI, is, nevertheless, is simply not implied.
Sadly, this imho rather invalidates much of what I see the core intended positive conclusion from your post. I’d love to get relief of worries from your argument, but I think there’s no deep foundation for it.
Of course, this doesn’t proof it’s better for the US to be the one to unlock AGI. I have a lot of sympathy with anyone telling: Look, China was (rather) not aggressive so far and we might have not more to worry about it than, say, the (currently particularly nuts) US first getting AGI. But I also have, despite the super sad and dangerous state and development of democracies as of late, remaining sympathy for anyone defending the idea of furthering regimes who at least very officially have some core western values ChristianKI pointed out in a separate comment here, even if we’ve always been super duper bad in fully adhering to them. I therefore am worried if we end up simply dismissing the latter as “China Derangement Syndrome”—or at least worried if we’d end up automatically dismissing all forms of such argument as CDS—even if the term you coin can be useful to describe quite some of the exaggerated negative pictures about China.
I’m not in any way sanguine about to which degree some more or less regularly reported quarrels about, say, South China sea things etc. are important or exactly not here; idk much about them at all.
Again, simply to the degree that this describes the state of China’s history well; other commenters have more to say on that than I.
How China fights its bordering neighbour.
The American sphere of care does not limit itself to a specific ethnic core. Despite recent anti-immigrant backlash, what it means to be an “American” is not tied up with your genetic makeup.
This is not true of China. You can’t “become” Chinese any more than you can change your ethnicity.
The Chinese government cares for the Chinese almost exclusively, with little concern given to anyone else, and certainly no one else can join the group of people they are concerned for. This is the sort of government that uses AI for the betterment of China, with everyone else being an afterthought. If the US wins at the very least there’s a system in place that is used to accepting people from every corner of the world into the political body.
China destroyed its navy in the Ming period and eschewed colonial expansion because they thought the rest of the world was low in quality, compared to China, and therefore not worth colonizing. By all accounts, they still think that.
I would suggest considering both their naval build up and their position on the South China sea and its historical use of trade/economic and political pressure to advance it’s influence and boarders, with military might as a backup for and enforcement mechanism “By all accounts” is a bit far.
What’s an example of one account by which China currently thinks the rest of the world is low in quality?
As someone who has both had a professional career in and out of China, I am utterly confused by the statement that the China today thinks ”rest of the world [is] low in quality”.
Care to share why you think that is the case?
I tend to view the people saying this as invoking a magic spell, as though geography will allow them to summon an AI based on America’s highest ideals instead of its practical realities (while denying the same magic spell to China). Like how, if “humane” means “That which, being human, we wish we were,” then “humane values” is what many people really mean when they say “human values.”
Even so, if I try to ask myself what each country’s actual highest ideals are, I have a harder time doing this for China than the US. Mostly because I’m American, yes, but also because China is a much older country, whose ideals have gone through more iterations over the millennia. If an AI views humans at its parents, then Confucian filial piety might honestly be a good part of an effective strategy for getting a good future compared to America’s view of youthful rebelliousness, I’m not sure. But an unwavering loyalty to whoever happens to lead the current CCP seems much less likely to go well when applied to an AI, compared to a culture of viewing current leadership as transient and fallible a la America’s ideals of civil dissent, peaceful transfers of power, limits of just authority, freedom of conscience, and personal autonomy.
Possibly slightly related side note that I realize is veering off topic:
My current perspective is that I think the “right” moral philosophy is some kind of infinite regress of alternating levels of act and rule consequentialism, where each level is justified by appeal to the level above it. In practice we deal with our bounded rationality by only looking two and three levels up when the current level is sufficiently obviously not doing what the first level up seems to want. Almost no one ever gets far enough out of distribution or has enough drive to abstract to go four and five levels up. AKA “The Rules say to do A, but that seems off; oh, they say to do A because the ones who made the Rules wanted to cause outcome Q. But in this case A doesn’t cause Q, so we need to appeal to Higher Rules that say to do B instead. That’s assuming we still want Q. The Higher Rules wanted Q because it was a proxy for R, which the Even Higher Rules tried to bring about...” It still all grounds out in What Do We Actually Want, which is somewhat-but-not-entirely a mystery to us.
I believe a large fraction of political discourse is just people talking past each other by implicitly appealing to different layers of this hierarchy. Some of our best and most famous documents make this explicit. The preambles of America’s Constitution and Declaration of Independence are both like this, one as a justification for establishing laws and one as a justification for breaking them. In the Bible, Jesus’ baptism by John has this character, since he’s conforming to ritual laws and traditions for their own sake even when they don’t quite make sense for him specifically. The koan “If you meet the Buddha on the road, kill him,” points the opposite way, towards there being a time to break rules for a higher purpose.
America has a concept of universal human rights and equality coming out of Christian belief. If you believe in those things, the important of spreading universal human rights and equality has some importance even when bringing democracy to Iran is not the primary motivation of the war.
If we have all powerful AGI, I think having that AGI be brought up to value universal human rights and equality is quite important. I can’t see AI alignment going well for humanity without AGI valuing those concepts.
I remember one of the reasons the DoD had developed such an anti-China view was because back in the 2010s, China had tended to break trade agreements constantly, showing that it wasn’t a credible dealmaker and further cooperation was not worth it.
I wish I knew which lesswrong comment said this before.
Is China Expansionist? Maybe.
I think this is a good post, much needed in this community given the dominating perspective I see when approaching China which seems to me to mostly see them as a similar entity to the USSR as an adversary, when from my reading on China, this model seems to fit incredibly poorly.
However, I do think it is worth noting that expansionism does often beget expansionism, and to say that China is not expansionist is somewhat misleading. The Taiwan situation is relatively unique, but it is difficult for me to model what happens after China reunifies with Taiwan (which will happen eventually, that is not what I would prefer nor what the majority of Taiwanese people would prefer, but it will happen and everyone should just be pricing this in as best they can).
Will China then turn to start attempting to annex Mongolia? Parts of Outer Manchuria currently owned by Russia? Parts of Japanese island chains? I think on these possibilities I’m at like 30%, 70% and 40% respectively.
Compare to America
But, of course, one needs to keep that in perspective with American behavior and how that appears to outsiders. Perhaps within America their current behavior is seen as strange and erratic, though not fundamentally destabilizing or cause to question the legitimacy/supremacy of their system of government. If the “system of government” we are thinking about is Liberal Democracy writ large than I agree that it is not a fundamental indictment of Liberal Democracy. However the system of government is not “Liberal Democracy writ large” it is the specific constitutional arrangement that the United States operates under, since that is what governs the country.
That system has led to outcomes that seem fundamentally at odds with principles of self-determination that I consider fundamental to Liberal Democracy (saber-rattling over Greenland comes to mind—perhaps many Americans see this as mostly a joke, but I certainly do not). But their system is, in fact, even more extreme than this:
The UK is not predominantly federalist, but even they acknowledge the right of Scotland or Northern Ireland to leave if they wanted to leave, and have held referendums on that matter. In the US such state-level self-determination is considered illegal outright. This, to me, seems to conflict with the principles of self-determination that are core to acting as Liberals on the geopolitical stage.
I would agree with many points here. China in recent history suffered years of colonizations and war here and there by at least 8 western countries, which made them miserable and resentful towards those who exploited others. It was eager to gain more power after WW2 because it realized that some countries can take advantage of the less powerful ones very easily. AFAIK gaining more power is intended to protect/defend the country itself and maintain stability, not offense. Even throughout the education system, taking advantage of the weak is never celebrated and always condemned. This is why I think by values China is not a very offense driven country, nor aims to spread its own ideologies, AFAIK. (Commenting only in relation to expansion perspective.) This is not saying countries should not be cautious/defend themselves.
To pick up some repeating arguments.
Re Taiwan: I don’t think this serves as a counterargument. The remnants of the non-communist Kuomintang forces retreated to Taiwan as a last base. For mainland China to consider this a “left over territory to conquer which we didn’t complete in 1949” is historically and culturally more justifiable than what international foreign intervention usually means.
Re China’s internal despotism: China being not militaristically expansionist does not equal China being a humanistic society or desirable system of state or societal organization. However, being a controlling, despotic system with inward facing repression, also doesn’t falsify the introductory claim. Contrary to many other despotic states, China undoubtedly has the military capability to expand forcefully (and ironically probably enough indifference to human life to not care about the loss of armed personnel).
Re roads and belts initiative: I agree it is not just an infrastructure project. Quite the opposite: it is an effective mixture of soft- and hard-power (we develop your country without meddling in your internal affairs but we lock you into economic dependence on the way). It is however distinguishable from the way development aid by western countries has often played out for other countries’ elected governments.
All in all: China’s track record internally is much worse than western countries, judged against standard definitions of human rights. China’s track record externally is way better than western countries, judged against international law and human rights. We have to accept the ambiguity.
As to what happens if China aligns AGI first: can’t know obviously, but some aspects to consider.
Chinese communism had for the longest time a system of balanced, competing internal organizations. After the death of Mao I believe it was part of their success story, much as the two-term presidential limit in the US has been. It guaranteed change in leadership and thereby adaptation to new situations. This condition has changed with Xi Jinping. There is a probability that this also changes long-term certainties and stabilizing factors that contributed to China not being expansionist in a traditional way.
“Traditional way” is the trigger. Usage of AGI for dominance is anything but traditional. Might lead to a more flexible usage. My best bet would be: still not a war scenario, but rapidly increasing other nations’ dependence by exploiting vulnerabilities (and of course taking over Taiwan and claiming great portions of ocean if it hasn’t happened by then). It seems to fit best to the historic behaviour and the need for safety.
I believe there is no case to argue China’s AGI would be morally more or less desirable than a US version. If they align in any way to their country’s historic record, we are discussing trading a “golden prison with obvious brutal control” against “war machine with efficient but surgical internal control.”
Re the addendum: by 1895 the US track record for wars against established and independent neighbouring nations was already down the toilet.
I think this makes sense, but if you apply it to “affording a child” China is doing way worse in this regard (1.02 TFR vs 1.62 for the US).
But any single measure like this can be really noisy. For example it surprised me to learn that incarceration rate for Taiwan is 236/100k vs 36 for Japan (165 for China, 541 for USA, 103 for South Korea), and I think you’ll be hard pressed to find anyone who thinks Taiwan is even 2 times more repressive than Japan, much less 6. (BTW the incarceration rate for Asians in the US is apparently 72.)
I don’t think it’s been mentioned by other comments, but imo China doesn’t have any AI labs that care about alignment as a real issue as much as even OpenAI in the US, let alone Anthropic.
“The Dario quote points to (3) with unusual directness”
This feels like a misreading of the Dario quote.
Anyway, I appreciate you differentiating different models of harm.
It seems that much of the controversy surrounding this article stems from the claim that “China only attacks its neighbors,” but this claim is inaccurate. The fact is that “since 1949, the People’s Republic of China has never invaded another sovereign state with the aim of annexing it, ceding undisputed territory, or establishing a puppet regime.”
To me, I think the “likelihood to dominate others” factor is less salient than the “likelihood to produce safe AGI” factor. Are there good arguments that China is better on AGI safety?
They seem to have any central policy at all, which is good if you think the AI technologies currently under development will produce unsafe AGI unless they are intelligently centrally managed.
I am glad this article exists, particularly because those of us who live in the U.S. should always be scrutinizing our own biases and patriotic framings.
That said, I think a fulsome discussion of whether China would use AGI to control other nations should at least include the following topics: 1) Uyghurs, 2) Tibet, 3) Taiwan, and 4) Chinese investment and contracting in Africa. I’m not an expert here—someone else can probably think of additional case studies.
I also think that, granted that the U.S. is a much more bellicose country on the international stage, I’m not sure if a non-intrusive country is likely to stay that way if given a total and complete advantage over other countries. On the one hand, history seems to show that countries will use their decisive military advantages to dominate other countries if they are able. On the other hand, if China got aligned AGI first, then it seems like they would have everything they could ever want at their fingertips and they would only need to care about the rest of us a tiny bit to respect our autonomy.
If country-autonomy is really part of the Chinese cultural DNA, perhaps their aligned AGI would even assist in protecting country autonomy. If the AGI did that forever, it would either be because Chinese attitudes toward intervention remained constant (unlikely) or the Chinese created an aligned but incorrigible AGI such that respecting country autonomy got locked in forever.
I don’t understand the relevance of Uyghurs or Tibetans, who are within China’s own borders. The Taiwan conflict is in close to the same category. Chinese investment and contracting isn’t imperial or manipulative in the CIA-style sense I was talking about. My understanding is that Belt and Road is a bunch of construction ($800B) and investment ($600B) contracts and that the largest recipients are Pakistan, Saudi Arabia, and Nigeria, and then other recipients include Indonesia, Iraq, and the Congo.
That makes sense. I added the following addendum:
However, your point below is not true I think:
The American government discussed but ultimately did not decisively defeat the Soviet Union when they alone had the atom bomb. I’m not sure how “decisive” that would have been, though, because the Soviets had a strong conventional position and I’m not sure how many atom bombs were available in 1946.
I thought of the U.S. getting to nukes first as a possible counter example, but I discounted it for the reason you provided (not that many and questions about decisiveness) and the fact that only four years passed between the U.S. dropping the bombs and the Soviet Union successfully developing their own bomb.
Also, nuclear weapons are the kind of weapon that has significant blowback considerations (e.g., radiation blowing into Europe or climate risks for something as big as taking out the full USSR—though that would not have been feasible in that period).
I’m generally against “race with China” frame, because whatever is going to happen after AGI is going to be much weirder even in non-extinction scenarios, but if we consider unusually-non-weird scenarios, I think you miss multiple unpleasant ways to project power, which are not “conquest”, but functionally are hard to distinguish from conquest in negative consequences. For example, if all China-friendly dictatorships get AGI-derived tech first (“deploy total mass surveillance with this one trick, first ten countries get discount”), it’s going to be very unpleasant, even if China inself won’t anything bad directly.
No offense. But imagine Nazi Germany wasn’t defeated, conquered most of Europe (taking bordering nations one at a time), then stopped its conquest (but continued its operation of exterminating Gays, Jews, Schizophrenics and so on).
Then you could write this same post and call it “Nazi Derangement Syndrome”