China-focused researcher thinking about China’s role in AI Governance, x-risk and other future scenarios
Jack_S
Karma: 148
You Can’t Really Bet on Doom
Thanks for the response. I’ve improved the signposting a little, and I’ve added an MIT open source license to the file.
I agree with the second recommendation, too. I was thinking of a more serialised alternative version, which actually tracked your choices month-by-month through an AI race scenario, like this FT browser game https://ig.ft.com/football-game/.
I might try that when I have more free time, but I’d love for someone to take the idea and make something like that!
AI Governance Strategy Builder: A Browser Game
The assumption of “US-OGI-1” works well, but I think it’s misleading to say that “the OGI model itself is geographically neutral—it could in principle be implemented by any technologically capable nation as host, or by multiple different countries as hosts for different AGI ventures.”
Part of the appeal of this proposal is that “norms and laws around property ownership, investor rights, and corporate governance are comparatively well established and integrated with civilian society”, but this clearly doesn’t hold everywhere.
The second-most-likely country for a leading AGI lab to emerge is China, and we can imagine a “China-OGI-1” scenario or a multipolar “US-OGI-n + China-OGI-n” scenario. In China, a few issues come to mind:
1. Weaker/more arbitrary corporate law and property rights, so no credible investor protections
2. More state/party interference in law, rights and governance
3. Weaker, less independent corporate governanceI’m curious if you think that this would still be a good model, assuming China-OGI-1 or a multipolar scenario. Or does it make more sense to say that OGI is only a good model where these norms and laws are actually well-established and integrated?
A few thoughts on this:
I feel that there’s a strong, slightly opposite bias of “picking winners”—I couldn’t find a specific term for it (there’s BIRGing (Basking in reflected glory) and bandwagon effects). It might be interesting to see when underdog bias vs. picking winners applies.
A quick example: sports fans famously love an underdog story, but when choosing a football team to support, people from around the world without a local team almost always choose one of the strongest few teams. If underdog bias ruled, you’d see football fans around the world with T-shirts of random mid-tier sides.
Evo-psych explanations (some listed in other comments) usually seem to explain “picking winners” more reliably. From siding with an alpha-male chimp to modern geopolitics, it can be dangerous to back a loser.
But these aren’t necessarily contradictory—the optimal strategy is often picking an underdog that’s actually got a secret weapon (e.g. David vs. Goliath). If you commit to siding with an underdog that wins, the payoffs are probably greater than siding with a winning overdog.
Slightly unrelated, there’s also a debate as to the universality of underdog bias. While the headline is that it’s somewhat universal, this paper (small sample, contrived experiment) finds that Israeli and Chinese respondents are slightly less underdog-prone than Japanese and American respondents. I sense there is something there—I often find “pro-overdog” narratives in Chinese media jarring. The theory might be that some cultures are more respecting of natural hierarchies or something
Local Detours On A Narrow Path: How might AI treaties fail in China?
I enjoyed this post.
I’m interested in interpersonal variation on these issues. Some people feel very strong “hangry” emotions and even physical pain, whereas some people (like myself) feel almost no negative emotions or sensations when they don’t eat for prolonged periods.
But some people have an insatiable appetite, and when they start eating, they can’t stop. If there’s limitless food of the right type available, I can easily put away 2000+ calories in a meal.
I wonder if these are anti-correlated in an interesting way. It seems that there could be an evolutionary logic that we need signals to eat, but these work through fairly distinct pathways, leading to subtly different “hunger phenotypes”.
The use of the Chinese study about healthy aging for elderly Chinese people is egregiously misleading. The OP uses it to make three separate points, about cognitive impairment, dose-response effects and lower overall odds of healthy aging. But it’s pretty clear that the study is basically showing the effects of poverty on health in old age!
Elderly Chinese people are mostly vegetarian or vegan because a) they can’t afford meat, or b) have stopped eating meat because they struggle with other health issues, both of which would massively bias the outcomes! So their poor outcomes might be partly through diet-related effects, like nutrient/protein deficiency, but could also be sanitation, malnutrition in earlier life (these are people brought up in extreme famines), education (particularly for the cognitive impairment test), and the health issues that cause them to reduce meat. The study fails to control for extreme poverty by grouping together everyone who earned <8000 Yuan a year (80% of the survey sample!), which is pretty ridiculous, because the original dataset should have continuous data.
But, the paper does control for diet quality, and it also makes it abundantly clear that diet quality is the real driver, and that healthy plant-based diets score similarly to omnivorous diets! “With vegetarians of higher diet quality not significantly differing in terms of overall healthy aging and individual outcomes when compared to omnivores”.
Probably less importantly, it conditions on survival to 80, which creates a case of survivorship bias/collider bias. So there could be a story where less healthy omnivores tend to die earlier, and the survivors appear healthier.