There’s an approx analogy used that chimps can’t really control or contain humans due the intelligence difference.
But what if the chimps moved at 100x the speed of humans, and there were 100x more of them? Given time to prepare the humans could win, but if the game is already in motion, I think speed and numbers compensate for raw intelligence.
I think we’re on the cusp of that now with AI agents. There’s a sense in that they’re not as smart as smart humans on many tasks, and they’re missing some kind of cognitive skills (it still seems), but they are fast and it’s not hard for there to be a lot of them, and so even before they’re strictly smarter than humans in some full general quality sense, I think we lose the ability to control or contain them because they are too fast and too many.
In this scenario, I definitely wouldn’t worry about chimps seizing control of civilization and supplanting us—instead, I’d worry about them irrepsarably damaging industrial civilization.
I’m pretty sure a chimp moving at 2000 kph is a straight-up hypersonic missile, can run across the ocean, and can sink battleships through kamikaze attacks. I think the chances of defending against 800 billion of them are effectively zero. I think the chances of defending against 80 million of them is still below 50%. These things are literally faster than bullets. I think they outrun the pressure front of explosives.
If they can also throw rocks at 100x the speed of humans, that’s something that could disable a tank in a single hit. I don’t think you appreciate how insanely powerful a hyperchimp is.
Average Human Reading Speed:238–260 WPM (about 5 tokens per second), based on Marc Brysbaert’s reading meta-analysis. [1]
LLM Generation Speed (Output/Decoding):500 to 3,000+ WPM (20 to 100+ tokens per second), depending on hardware, model size, and hosting infrastructure. [1]
LLM Input Processing (Prefill):10,000+ WPM equivalent (hundreds to thousands of tokens per second) when ingesting long contexts into the KV cache. [1]
Depends. If they can literally run across the ocean, that’s too bad, but otherwise some humans could survive on an island and maybe prepare a counter-offensive. Or maybe wait for a few years, until most of the superchimp population starves to death.
An unworthy successor is a bad outcome which I feel is under-discussed.
I wonder if the talk about RSI which surpasses humanity precludes discussion of takeovers by spiky, locally effective but globally hopeless AI. If progress is continuous, and if AIs are better than humans at converting intelligence into power from speed and bulk, there will necessarily be a moment where we are smarter but less powerful. Most scenarios assume the moment will be brief, but maybe it will not be.
I don’t want superhuman AIs to leave humanity behind, but I strongly prefer that to a world of fine-tuned digital morlocks possessing the instruments to overthrow us, but without the general faculties that let us look up at the stars and hope for more.
I think the scenario “AI exterminates its creators, and then fails to colonize space (and either destroys itself clumsily or remains stuck on its planet of origin until it runs out of resources)” is a possible explanation for the lack or extraterrestrial intelligence.
Follow on to earlier quick take:
There’s an approx analogy used that chimps can’t really control or contain humans due the intelligence difference.
But what if the chimps moved at 100x the speed of humans, and there were 100x more of them? Given time to prepare the humans could win, but if the game is already in motion, I think speed and numbers compensate for raw intelligence.
I think we’re on the cusp of that now with AI agents. There’s a sense in that they’re not as smart as smart humans on many tasks, and they’re missing some kind of cognitive skills (it still seems), but they are fast and it’s not hard for there to be a lot of them, and so even before they’re strictly smarter than humans in some full general quality sense, I think we lose the ability to control or contain them because they are too fast and too many.
This graphic was present years ago on the landing page of the Center for Humane Technology.[1]
h/t TJ for bringing it to my attention
In this scenario, I definitely wouldn’t worry about chimps seizing control of civilization and supplanting us—instead, I’d worry about them irrepsarably damaging industrial civilization.
I’m pretty sure a chimp moving at 2000 kph is a straight-up hypersonic missile, can run across the ocean, and can sink battleships through kamikaze attacks. I think the chances of defending against 800 billion of them are effectively zero. I think the chances of defending against 80 million of them is still below 50%. These things are literally faster than bullets. I think they outrun the pressure front of explosives.
If they can also throw rocks at 100x the speed of humans, that’s something that could disable a tank in a single hit. I don’t think you appreciate how insanely powerful a hyperchimp is.
Human vs. LLM Speeds
Average Human Reading Speed: 238–260 WPM (about 5 tokens per second), based on Marc Brysbaert’s reading meta-analysis. [1]
LLM Generation Speed (Output/Decoding): 500 to 3,000+ WPM (20 to 100+ tokens per second), depending on hardware, model size, and hosting infrastructure. [1]
LLM Input Processing (Prefill): 10,000+ WPM equivalent (hundreds to thousands of tokens per second) when ingesting long contexts into the KV cache. [1]
From Google AI preview
Depends. If they can literally run across the ocean, that’s too bad, but otherwise some humans could survive on an island and maybe prepare a counter-offensive. Or maybe wait for a few years, until most of the superchimp population starves to death.
An unworthy successor is a bad outcome which I feel is under-discussed.
I wonder if the talk about RSI which surpasses humanity precludes discussion of takeovers by spiky, locally effective but globally hopeless AI. If progress is continuous, and if AIs are better than humans at converting intelligence into power from speed and bulk, there will necessarily be a moment where we are smarter but less powerful. Most scenarios assume the moment will be brief, but maybe it will not be.
I don’t want superhuman AIs to leave humanity behind, but I strongly prefer that to a world of fine-tuned digital morlocks possessing the instruments to overthrow us, but without the general faculties that let us look up at the stars and hope for more.
I think the scenario “AI exterminates its creators, and then fails to colonize space (and either destroys itself clumsily or remains stuck on its planet of origin until it runs out of resources)” is a possible explanation for the lack or extraterrestrial intelligence.
I think this is the worst non-contrived tragedy I can imagine.