AI Safety in China: Part 2
I’ve given things a lot more thought and wanted to make an edited summary of my views regarding AI Safety in China.
AI Safety is almost nonexistent in China. The only toe-hold it has is rooted in the expat EA community, which is almost exclusively doing field building. This hasn’t been effective thus far, and we need to switch tactics. My best guess is that a total of 10-20 people have a job in China in the field of AI safety, most of them are expats, and most of them are EA.
EA, for various cultural reasons, is a toxic brand in China. It’s not any single component of EA, but rather the idea of altruism itself. Ask anyone who’s lived in China for and few years and they will understand where I’m coming from. I think the best way forward for AI safety in China is to disassociate with EA. Rationality is more easily accepted, but spreading related ideas is not the most effective way to address AI safety in China.
Blue-skies research, possibly in something legible like interpretability, is the best way I can think of to actually build the AI safety field in China. The best way of making a field high-status is to overpay researchers and have them tell their friends.
Money talks. Top-tier talent is often willing to jump ship to startups for the right compensation. I might be biased here since I’ve only worked in smaller companies. But if I had a large bag of money, I could probably leverage familial academia/business connections to hire star AI researchers.
The Chinese government makes its decisions based on expert opinion and ideas from thought leaders in a particular field. The Chinese elites think like engineers and tend to take ideas seriously. AI safety isn’t considered as weird as it is in the West.
Due to AI safety’s current lack of penetration in the Chinese sphere, I think any near-term attempts at an AI restriction treaty are unlikely to succeed—we don’t think it’s a real issue yet. Imagine if the Saudi King called the US President and tried to negotiate social media restrictions to prevent the Abrahamic End Times. That’s where we’re at. Then again, maybe someone in the Politburo is willing to take AI at least as seriously as Glenn Beck. It’s hard to tell.
China doesn’t think like the West. Due to the Century of Humiliation, we are not going to be OK with being second best in a world-changing technology as we become the largest economy in the world. Our AI policy isn’t OK with second place in the long run. Any AI-restriction treaty that China will accept requires not just Chinese parity, but Chinese superiority in AI. Major tradeoffs, like the US coercing Taiwan into reunification, may be required for China to sign on.
Then again, Beijing is hard to predict. It may agree to an AI disarmament treaty in 6 months, or it might confiscate private GPUs in an effort at mass mobilization, spending billions to build the next LLM. It might do both.
There are vague purity tests in EA about only hiring people who are super-duper not going to work in AI capabilities ever for AI safety research. This is extremely repugnant to Chinese researchers.
On a personal level, I want to enter the field of AI safety immediately, and I think I would be more useful building the field in China as opposed to working in the US. However, I have only a single semester’s worth of graduate studies thus far and my application to go from non-degree-seeking to degree-seeking student status was rejected. I am probably going to take a few online ML courses over the next year before I feel ready enough to actually request funding, but am open to suggestions.
Again, there’s very few (<10) people working on technical alignment in China right now, and I feel a bit lost. Any advice is welcome.
I’d like to know more about this. What’s the deal with altruism in China? Why is altruism disliked?
There was some good discussion about this in the previous AI safety in China thread.
Very illuminating, thank you for the pointer.
E.S.: personal opinion
Because proclaimed altruism is almost always not.
In particular, SBF and the current EA push to religiously monopolize AI capability and research triggers a lot of red flags. There are even upvoted posts debating whether it’s “good” to publicize interpretability research. This screams cultist egoism to me.
Asking others to be altruistic is also a non-cooperative action. You need to pay people directly not bully them to work because of the greater good. A society in which people aren’t allowed to have their self-interest as a priority is a society of slave bees.
Altruism needs to be self-initiated and shown, not told.
Just to avoid misinterpreting you, do you mean to say your personal opinion here sheds light on why the idea of altruism is culturally disliked in China?
(Asking since I’m following Gordon’s comment to learn about why altruism is culturally disliked in China, so I’d like to filter direct responses from personal opinion tangents for myself)
It’s probably more so that that the vast majority of families in China have recent historical memories of being duped by sophisticated folks proclaiming to be altruistic, morally superior, etc., and engaging in activities allegedly to the benefit of the wider population.
I’m inclined to believe in the possibility of a memetic ‘inoculation’ so I can see why there would be a huge difference in how such ideas are viewed, to a lesser extent this applies to all of the former Warsaw Pact countries too.
Plus there’s the natural tendency of an older and longer settled society to be more conscious of social status maneuvering.
Most of the population probably won’t have anything against genuinely altruistic people who don’t try to claim any sort of social status, moral authority, etc.
Would you say that China knows the bankruptcy of heroic-sacrifice culture (collectivist duty), and westerners have not really experienced that, and they know that westerners just romanticize it from a distance without ever really living it?
EA is funny in that it consists mostly of people who know about incentives and consequences and they try to pay their people well and tell them not to burn themselves out and to keep room for themselves, but it is still named “altruism”, and it still does a bit of hero-eating on the side from time to time.
What kind of money are you talking about?
What’s the likely response from the Chinese government if you would do such a project with significant funding?
I vaguely remember that talented Chinese grads from the best universities majoring in ML get starting salaries of $100,000+. Star researchers… probably a few million? My best guess is between 50% and 100% of US counterparts—a lot of Chinese are willing to accept lower pay to work in a familiar culture.
The most likely response from the government is nothing. If they cared at all about the subject, I wouldn’t be so worried.
Edit: competent college and grad schools with hireable-in-the-West ML skills are a lot cheaper. Something like $30,000+.
Maybe there’s a backdoor approach. Instead of trying to convince the general public about the importance of alignment perhaps there’s a model for creating ‘alignment departments’ in existing tech and AI companies. Another idea could be outreach for high school or university students that want to pursue careers in tech.
From reading Part 1 it seems the ways that we’ve tried to spread the message in Western countries won’t work in China because of different cultural/social norms so maybe try surveying people in China on how to get the message across and incite interest and action.
Yes! That sounds like it could work! But as long as it isn’t something people can get a career in, it’ll just stay in the realm of a cool but easily forgotten idea. This is why I think it’s so important to hire people to work in technical alignment. If it is even a slightly theoretical possibility, it would get people thinking about it more seriously.
I feel there may be more to it, focus testing might provide some answers, get to the heart of why they don’t take tech disruption more seriously.
Here’s a crazier but more specific possible course of action: When I was visiting Albania in 2011 I heard from locals about foreign religious groups that offered free English language courses. Naturally, I assumed it was mainly for recruitment and now know it’s a fairly common tactic.
So the idea is what if someone offered coding classes (or English courses) and used it as a platform to discuss AI? And you could be upfront about it in the marketing: “Learn coding and AI fundamentals from an experienced expert”
These are difficult questions, and unfortunately it’s possible that the situation for AI alignment in China might be even worse than we can see here. There’s a lot of really weird, secret downsides to most moves that could be made.
However, I actually do have a really, really solid recommendation that I think there’s no downside for, and massive upsides. I thought about it for a while, and I think Tuning your cognitive strategies might be an extremely good fit for AI safety researchers in China. These are my reasons:
Cognitive tuning offers human intelligence amplification in the near-term. A person tries it, and they get smarter, within a day or an hour or so. Human intelligence amplification has recently been endorsed as a potentially critical element for solving AI alignment in time; regardless of whether or not late-stage AI will give us resources to help us solve alignment, we are more likely to solve it in time if we have smarter alignment researchers. If we find ways to make humans smarter, that’s good news across the board.
People in China might or might not be slightly better than westerners at paying attention to their own thoughts (the important thing is building the habit from a young age, some cultures will be better at this than others). But people in China will almost certainly be better at avoiding the risks than westerners were, because the problem with cognitive tuning is that it violates Schelling Fences, and people in China (and Asian cultures more generally) are probably much better at noticing and budgeting Schelling Fence violations due to an ambient conformity culture.
The big bottleneck with cognitive tuning is figuring out how it works and how to do it right. It’s basically its own field of research, but it’s at the extreme early phase with massive untapped potential. The bottleneck is that almost nobody in AI safety knows about it, and almost nobody is trying it and sharing details about how it worked and what they learned from the process. There’s incredible potential for growth and incredible potential for results, no matter what country pioneers it.
Human intelligence amplification, with immediate results, is something that will be broadly smiled upon in China. It’s basically a startup. Nobody in the government would frown upon citizens for trying that. In fact, China might soon become a leader in neurofeedback, and this could potentially be retooled for human intelligence amplification.
+1 for tuning your cognitive strategies. For ~1 day a month, I experience a substantial increase in my ability to have quality thoughts. I’d read BWT quite a while ago, and when I re-read it recently, I realized “Oh, that’s what’s happening each month”. Improving my “Tuning your cognitive strategies” skill is now a high priority for me.
I’m really glad to see people are taking it seriously. This is legitimately something that could save the world, no matter what your AI assumptions are, or where the innovation/research takes place.
Keep in mind that budgeting Schelling Fences is really important, the author (squirrelinhell) tried that as well as like 5 other things at once (including self-inflicted sleep deprivation), in addition to being a roko’s basilisk researcher, and they literally died from it. When you do even one really intense mind-improving thing, it’s hard to keep track of all the things you took for granted but might no longer apply- and even harder when you’re doing lots of weird stuff at once.
Oh, I’m quite wary of mental modifications. I’ve both had some poor experiences myself, and listened to enough stories by people who’ve done far more substantial changes to their cognition, to know that this is dangerous territory.
Incidentally, I showed that skill from BWT to someone who claims to have done great amounts of mental surgery. They stated that the skill isn’t a bad solution to the problem, but the author of the page didn’t know what the problem even is. Namely, that people didn’t spend enough time thinking alone as a kid due to repeated distractions, which caused firmware damage. N.B. they only read the “Tuning your cognitive strategies page”. I think they also claimed that this damage was related to school, or perhaps social media?
I’m not sure what to make of that claim, but the fact is that I have many instinctive flinches away from entering the state of mind which that skill-page describes. These flinches are, I think, caused by a fear of failure to solve problems or produce valuable thoughts. Which, you know, does sound like the kind of damage that school or social media could do.
That’s really interesting, do you have a list of resources you could recommend to me for things that are similar to/better than BWT? I wasn’t aware that finding more was even possible.
I don’t have anything that is better than BWT. I’ve just read, and heard, people who claim to have done substantial mental modifications talking about their experience. The guy I was talking about claimed that 1) This stuff is dangerous, so he won’t go into details and 2) You really have to develop your own techniques. He seems quite sharp, so I’m inclined to trust his word, but that’s not much evidence for you. And I haven’t done much myself other than mess up my brain a few times, and practiced BWT-related focusing enough times that I started getting something out of it.
This post is quite strange and at odds with your first one. Your own point 5 contradicts your point 6. If they’re so good at taking ideas seriously, why wouldn’t they respond to coherent reasoning presented by a US president? Points 7 and 8 just read like hysterical Orientalist Twitter China Watcher nonsense, to be quite frank. There is absolutely nothing substantiating that China would recklessly pursue nothing but “superiority” in AI at all costs (up to and including national suicide) beyond simplistic narratives of the CCP being a cartoon evil force seeking world domination and such.
Instead of invoking tired tropes like the Century of Humiliation, I would mention the tech/economic restrictions recently levied by the US (which are, not inaccurately, broadly seen in China as an attempt to suppress its national development, with comments by Xi to that effect). Any negotiated slowdowns in AI would have to be demonstrated to China as not to be a component of that, which it shouldn’t be hard to if the US is also verifiably halting its own progress, and the AGI x-risk arguments can be clearly communicated.
I really don’t know what Beijing is going to do. Sometimes it makes really weird decisions, like not importing better COVID vaccines before the COVID restrictions were removed. There is no law of physics that says Beijing will take X seriously if Washington presents a good argument, or if Washington believes it hard enough. Beijing can be influenced by rational arguments, but mostly by Chinese experts. Right now, the Chinese space isn’t taking AI safety seriously. There is no Chinese Eliezer Yudkowsky. If the US in 2001 was approached by Beijing asking for AI disarmament, they would either assume it was either an attempt at manipulation or a special Chinese brand of crazy. Maybe both.
Put yourself in Beijing’s shoes for a moment. What would you do if you thought Washington seriously believed in AI ruin, but all of your experts told you it was just the West being crazy? I would squeeze as many concessions (especially Taiwan) as possible as the price of compliance before defecting and pursuing AI research, preferably in secret, safe in the knowledge that the US is unwilling to pursue this promising technology. What the hell are they going to do if they catch you? Start up their own AI research programs they believe are going to destroy the world? Maybe they’ll even send you their best AI safety research anyways, and you can use it to further your capabilities research.
I think (Cooperate, Cooperate) isn’t the obvious result here.
Why wouldn’t their leadership be capable of personally evaluating arguments that this community has repeatedly demonstrated can be compressed into sub 10 minute nontechnical talks? And why assume whichever experts they’re taking advice from would uniformly interpret it as “craziness” especially when surveys show most AI researchers in the west are now taking existential risk seriously? It’s really not such a difficult or unintuitive concept to grasp that building a more intelligent species could go badly.
My take is the lack of AI safety activity in China is effectively due almost entirely to the language barrier, I don’t see much reason they wouldn’t be about equally receptive to the fundamental arguments as a western audience once presented with them competently.
Honestly, I would probably be more concerned about convincing western leaders whose “being on board” this debate seems to take as an axiom.
The sub 10 minute arguments aren’t convincing. No sane politician would distrust their experts over online hysteria.
I have the experience of living in a strongly anti-West country ruled by the same guy for 10+ years (the Putin’s Russia). The list of similarities to Xi’s China includes the Shameful Period of Humiliation often employed by the state media to reinforce the anti-West narrative (in the case of Russia it’s the 1990s).
With this background, I see the points 7 and 8 as valid, and likely applicable to the majority of anti-West governments of the same nature.
Yep, same for Russia. Even if the Russian gov decides to make the impression of accepting such a treaty, or even if it starts enforcing the treaty among civilian companies, the Russian military will continue to secretly work on military AI anyway. As Putin himself said, “The country that secures a monopoly in the field of artificial intelligence will become the ruler of the world”.
Another of his famous sayings: “there is no value in a world where Russia doesn’t exist” (the context: a discussion about Russia destroying the world with nukes if the West attempts to subjugate Russia).
Again, same for Russia. Putin has the reputation of accepting any vaguely reasonable expert proposal, and even several contradicting proposals on the same topic, if the proposers are strongly loyal to Putin.
This sometimes results in the wildest shit becoming a law. For example, Russia banned exports of biological tissue samples, because someone told Putin that it could be used to develop a virus to exclusively kill Russians (which is a biological nonsense).
In general, Russia is way behind the US or China in the field of AI. But several major companies (Yandex, Sber) have demonstrated the ability to adapt and deploy some relatively recent open-source AI tech at scale.
Even with the severe hardware sanctions in place, maybe in 5 years or less there will be a Russian GPT4.
Eric Hoel has written about this and his(as well as mine) impression of the reigning attitude of the Party arguably is pro-coordination. This might be strange as I take it from internally knowing that there was discussion about what to do if an AGI seemed likely—one of the answer was to immediately launch nuclear attack and make sure no one(and no machine) survives.
It’s a very “brute” attitude toward alignment but there’s a clear preference for humanity in charge, and the idea of machines taking the “future” from the Chinese people is seen as unacceptable.
Conversely, without AI anywhere in the world, the Party seems to think they will “win.” So the game dynamics encourage them to agree to styming AI development everywhere, even if they do something more low-key in secret.
What’s intriguing to me is that while the OP disagrees on a lot of possibilities for agreement, the central dynamics of competition and “brute force” remain, as well as the higher technological savvy of the Party.
Tangential point: [epistemic status: armchair speculation without expertise]
Given that the US gov has started taking AI at least seriously enough to think about the underlying hardware as relevant to national security and banning the export to China, and also given that the US is significantly dependent on Taiwan for the hardware advantage is does have… Seems really unlikely that the US would give up Taiwan to China in a treaty. Probably the US would back Taiwan substantially more strongly than it has backed Ukraine, if there was a China-Taiwan active conflict initiated. Possibly maneuvering US battleships in Taiwanese waters and such, to play at brinkmanship. I’m no expert in this, just speculating.
I think it will help if you can just be clear on what you want for yourself, China, and the world. You’re worried about runaway AI, but is the answer (1) a licensing regime that makes very advanced AI simply illegal, or (2) theoretical and practical progress in “alignment” that can make even very advanced AI to be safe? Or do you just want there to be an intellectual culture that acknowledges the problem, and paves the way for all types of solutions to be pursued? If you can be clear in your own mind, about what your opinions are, then you can forthrightly express them, even as you develop a career in AI that might be more conventional to begin with.
Concerning the rivalry between China and America, (1) and (2) have different implications. If the answer is (1), then the strategy for China is to build up its own AI capabilities within the threshold of safety, while allying with those forces in America who also want to put a safe ceiling on American AI capabilities. If the answer is (2), then the priority is for all the most advanced research efforts, to also have the most advanced safety methodologies, which is potentially an even more cooperative situation.
Concerning the association in the West between AI safety and effective altruism, again, I think it helps to be clear in your own mind. If you perceive clearly why you reject EA but still see merit in AI safety, surely you can explain the logic of that position to someone else too.
I’m saying that funding technical alignment in China is importnat for 2 reasons: firstly, it helps build a community of people interested in the field, which helps sway elite opinion and ultimately policy. Secondly, it can contribute to overall progress in AI alignment. In my opinion, the former is more important and time-critical, as other efforts at community building have not been very successful thus far, and the process of fieldbuilding → elite opinion shift takes time.
I’m planning on making a detailed post about why EA/altruism-in-general is a bad match for China, with a lot of citations.
Some big names from China signed off on that recent one-sentence statement about extinction risk due to AI… Does that affect your sense of how receptive the Chinese establishment is, to AI safety as a field of research?