Would we be less likely to go extinct if OpenAI forcibly took over Anthropic? (a possible hypothetical: a combined effort of OpenAI and the US government forces a merger). My take: no. Main reasons:
I trust OpenAI leadership less than Anthropic leadership
Race dynamics would not improve due to the other players.
I judge the creation of Anthropic as retrospectively a good choice. At the time I was negative and have become less so.
Similarly, would we be less likely to go extinct if Alphabet forcibly took over OpenAI and Anthropic? My take: yes. Main reasons:
Substantial improvement in race dynamics.
Alphabet has less corporate need for urgent AI advances.
Alphabet has a better security posture.
I judge the creation of OpenAI as retrospectively a bad choice. At the time I was negative and have become more so.
Would we be less likely to go extinct if OpenAI forcibly took over Anthropic? (a possible hypothetical: a combined effort of OpenAI and the US government forces a merger). My take: no.
People keep making arguments shaped like this, and I feel that there’s some error here.
Currently, for race dynamics to dissolve, two LLM megacorps need to cease to be relevant. If Anthropic weren’t founded or were destroyed/absorbed, only one LLM megacorp would need to lose relevance. Destroying one megacorp is easier than destroying two. Therefore, in worlds in which Anthropic didn’t exist or were destroyed, it would be easier to dissolve the race dynamics altogether. Therefore, it’s bad that Anthropic exists and it would be better if it did not exist.
That seems straightforward to me? We are less likely to go extinct in Anthropic-less worlds because those worlds are more likely to also become OpenAI-less and/or DeepMind-less.
There seems to be some underlying assumption that this is a one-turn game, where you’re asked to choose between “implement Intervention X and then do nothing forever” and “do nothing forever”. I’m not sure why “do nothing forever” is implicitly tacked on.
(Arguments like “I, a capability researcher, don’t need to quit my AGI lab, because I will just be replaced by an identical capability researcher, for a net zero change in anything” seem from the same genre.)
Even if another megacorp doesn’t leave the race, companies have a much larger incentive to invest in safety in a 2-actor race than a 3-actor race, because they internalize more of the benefit. At a minimum, they’re 1.5x more likely to win, plus it’s easier to coordinate between 2 actors than 3.
I don’t think destroying one megacorp is easier than destroying two, each of half the size. My model: it’s easier to destroy a hundred companies with one employee than to destroy one company with a hundred employees, if one is coming in with an external force.
Conditional on OpenAI forcibly taking over Anthropic, it is more likely that Alphabet forcibly takes over OpenAI (or OpenAI forcibly takes over the AI portions of Alphabet). I was more thinking of “what if a freak event caused a forced takeover” rather than “what if corporate murder of AI labs became normal”.
Its not clear to me that shutting company 1 and then shutting company 2 later in time is easer than shutting both companies at the same time in the future. I dont really wanna argue for any conclusion though.
Some “intuition pump” of where this pattern dosent hold: if you want to stop 2 snowballs from rolling down the hill stopping only 1 will make the other grow faster and you wont be able to stop it at all. Snowballs collide with eachother and slow themselves down.
I genuinely don’t understand why you didn’t propose that anyone took over the [EXPLETIVES DELETED] from xAI, since I believe that it is xAI and China who motivate the Big Three to race hard and are far harder to make deals with. OpenAI, Anthropic and GDM, unlike xAI, agreed to condemn the efforts of Chinese distillators! What would have happened with the race ifthe world had only the American Big Three, not xAI, Meta AI research or China?
IMO xAI is no longer a frontier lab. All 11 of Musk’s cofounders have left, along with much of the technical talent, and they haven’t produced a frontier model in some time. If someone took over xAI, it might speed up the race, since you’d be giving over their vast compute to a more competent actor. That being said, I agree that they have historically been a worse actor for AI Safety than the big three labs.
I don’t think that xAI is THAT far from being a frontier lab. Ryan Shea estimates it to be three moths behind, not seven months behind as Zvi was informed by Wildeford. Additionally, Grok 4.20 was released in March. Had xAI been seven months behind, it would be equivalent to a model as capable as the leaders had in August 2025 with xAI having been a leader in early July. And yet we see Grok 4.20 close to the frontiers[1] of ARC-AGI-2 and ARC-AGI-1. I wish that someone actually checked every model on as many benchmarks as they can so that we would quickly learn the capabilities of neglected labs like xAI.
P.S. There are also Grok 5 and Claude Mythos, but they have yet to be released and benchmarked (by what benchmarks?) I am not even sure that Claude Mythos will emerge three months earlier than Grok 5.
For comparison, the performance of Grok 4 on the benchmark was replicated no later than by GPT-5.1 released on November 12 (or was it GPT-5-pro and Claude Sonnet 4.5 released in August and late September?) meaning that even gaming the benchmark was unlikely to cause xAI to obtain more than 4 months of a false lead.
To answer the question as posed: it’s very clear to me that xAI and Meta AI are bad for my species, I don’t need a hypothetical for that. And I can’t do a clean hypothetical where China doesn’t exist.
Would we be less likely to go extinct if OpenAI forcibly took over Anthropic? (a possible hypothetical: a combined effort of OpenAI and the US government forces a merger). My take: no. Main reasons:
I trust OpenAI leadership less than Anthropic leadership
Race dynamics would not improve due to the other players.
I judge the creation of Anthropic as retrospectively a good choice. At the time I was negative and have become less so.
Similarly, would we be less likely to go extinct if Alphabet forcibly took over OpenAI and Anthropic? My take: yes. Main reasons:
Substantial improvement in race dynamics.
Alphabet has less corporate need for urgent AI advances.
Alphabet has a better security posture.
I judge the creation of OpenAI as retrospectively a bad choice. At the time I was negative and have become more so.
People keep making arguments shaped like this, and I feel that there’s some error here.
Currently, for race dynamics to dissolve, two LLM megacorps need to cease to be relevant. If Anthropic weren’t founded or were destroyed/absorbed, only one LLM megacorp would need to lose relevance. Destroying one megacorp is easier than destroying two. Therefore, in worlds in which Anthropic didn’t exist or were destroyed, it would be easier to dissolve the race dynamics altogether. Therefore, it’s bad that Anthropic exists and it would be better if it did not exist.
That seems straightforward to me? We are less likely to go extinct in Anthropic-less worlds because those worlds are more likely to also become OpenAI-less and/or DeepMind-less.
There seems to be some underlying assumption that this is a one-turn game, where you’re asked to choose between “implement Intervention X and then do nothing forever” and “do nothing forever”. I’m not sure why “do nothing forever” is implicitly tacked on.
(Arguments like “I, a capability researcher, don’t need to quit my AGI lab, because I will just be replaced by an identical capability researcher, for a net zero change in anything” seem from the same genre.)
Even if another megacorp doesn’t leave the race, companies have a much larger incentive to invest in safety in a 2-actor race than a 3-actor race, because they internalize more of the benefit. At a minimum, they’re 1.5x more likely to win, plus it’s easier to coordinate between 2 actors than 3.
I don’t think destroying one megacorp is easier than destroying two, each of half the size. My model: it’s easier to destroy a hundred companies with one employee than to destroy one company with a hundred employees, if one is coming in with an external force.
Conditional on OpenAI forcibly taking over Anthropic, it is more likely that Alphabet forcibly takes over OpenAI (or OpenAI forcibly takes over the AI portions of Alphabet). I was more thinking of “what if a freak event caused a forced takeover” rather than “what if corporate murder of AI labs became normal”.
Thanks for the thoughtful response.
Its not clear to me that shutting company 1 and then shutting company 2 later in time is easer than shutting both companies at the same time in the future. I dont really wanna argue for any conclusion though.
Some “intuition pump” of where this pattern dosent hold: if you want to stop 2 snowballs from rolling down the hill stopping only 1 will make the other grow faster and you wont be able to stop it at all. Snowballs collide with eachother and slow themselves down.
I genuinely don’t understand why you didn’t propose that anyone took over the [EXPLETIVES DELETED] from xAI, since I believe that it is xAI and China who motivate the Big Three to race hard and are far harder to make deals with. OpenAI, Anthropic and GDM, unlike xAI, agreed to condemn the efforts of Chinese distillators! What would have happened with the race if the world had only the American Big Three, not xAI, Meta AI research or China?
IMO xAI is no longer a frontier lab. All 11 of Musk’s cofounders have left, along with much of the technical talent, and they haven’t produced a frontier model in some time. If someone took over xAI, it might speed up the race, since you’d be giving over their vast compute to a more competent actor. That being said, I agree that they have historically been a worse actor for AI Safety than the big three labs.
I don’t think that xAI is THAT far from being a frontier lab. Ryan Shea estimates it to be three moths behind, not seven months behind as Zvi was informed by Wildeford. Additionally, Grok 4.20 was released in March. Had xAI been seven months behind, it would be equivalent to a model as capable as the leaders had in August 2025 with xAI having been a leader in early July. And yet we see Grok 4.20 close to the frontiers[1] of ARC-AGI-2 and ARC-AGI-1. I wish that someone actually checked every model on as many benchmarks as they can so that we would quickly learn the capabilities of neglected labs like xAI.
P.S. There are also Grok 5 and Claude Mythos, but they have yet to be released and benchmarked (by what benchmarks?) I am not even sure that Claude Mythos will emerge three months earlier than Grok 5.
For comparison, the performance of Grok 4 on the benchmark was replicated no later than by GPT-5.1 released on November 12 (or was it GPT-5-pro and Claude Sonnet 4.5 released in August and late September?) meaning that even gaming the benchmark was unlikely to cause xAI to obtain more than 4 months of a false lead.
To answer the question as posed: it’s very clear to me that xAI and Meta AI are bad for my species, I don’t need a hypothetical for that. And I can’t do a clean hypothetical where China doesn’t exist.
I agree these are good things to think about.