Picture the scenario: A malicious superintelligent AI creates a weapon that wiped out all humans instantaneously. The AI has complete control over all network connected devices and computers.
Most manufacturing will stop working immediately without humans to complete that steps that can’t be automated. Within minutes the first power stations will stop working. Even if the AI can get them to work eventually they’d run out of fuel, which can’t be extracted, transported and used to refill the power stations without a lot of manual labour.
Solar panels/wind turbines should last longer, but even then at some point in the grid you generally need humans to carry out repairs. Repairs that are tricky without the dexterity and versatility that our fingers provide.
The AI is then on a clock to get the stage where it can use the existing infrastructure we left behind to create robots/nanobots that are able to bootstrap more powerful robots/nanobots etc. till it can achieve the level of sophistication required for it’s aims.
How could an AI go about that? Is this a hard enough step that it’s worth putting safeguards in place to keep it hard? What would make it easier for the AI?
Assume that software won’t be any issue—the AI can easily hack that. The question is what it can do with the hardware we leave behind.
I suspect if an AI has some particular goal that requires destroying humanity and manufacturing things in the aftermath, and is intelligent and capable enough to actually do it, then it will consider these things in advance, and set up whatever initial automation it needs to achieve this before destroying humanity. AI with enough planning capabilities to e.g. design a bioweapon or incite a nuclear war would probably be able to think ahead about what to do afterwards, would have its own contingencies in place, and would not need to rely on whatever tools humanity happens to leave lying around when it is gone.
You’ve touched on a point that many posts don’t address—the realities of the real world. So many AI is going to kill us posts start with AI is coming then ? and then we all die.
Look at something like Taiwan chip manufacture—it’s incredibly cutting edge and complicated and some of it isn’t written down! There are all kinds of processes that we don’t patent for various reasons. So much of our knowledge is in process rather than actually written anywhere.
And all of these processes are themselves the pinnacle of hundreds of other interlinked processes.
Not only does the malevolent AI need to take over chip manufacturing, but it needs to control the Australian lithium mine, plus the Saudi oilfields to run the ship to get the stuff to where it needs to go plus everything else. Hope it has rubber ring manufacture down for when an engine needs a part.
So many actual real world processes are incredibly complex involving webs of production which simply cannot be taken over by AI in any meaningful sense.
Not even with a hoard of dexterous robots.
Even if an AI had the ability to make dexterous robots, millions of them, and then scatter them all over the world, it would be required to take on almost the entirety of human manufacturing—and we’re goddamn lazy and don’t write so much of it down anywhere. Even with observation over time it wouldn’t pick up so many things.
There are machines that operate specific processes where the knowledge group is incredibly small. Whoops, you killed them AI, sorry, no more ultra-pure materials for those chips you need.
As for nanobots, it always reads a bit like a joke that an AI will have these incredible knowledge leaps that produce magical robots to do all the things.
I think most of the time it’s a handwavey attempt at addressing the real world realities of manufacturing and production.
We had one flood and it nearly wiped out the world supply of RAM years ago. The truth is that so many processes the AI would need are incredibly concentrated, run by a small number of people, not written down in clear enough detail for anyone to pick up easily, and contain all kinds of secret knowledge no AI can obtain.
Then for so long as it needs humans, it would act through humans.
This suggests than industrial espionage would be one of the AI’s priorities.