These are good thought-experiments, although, regarding the first scenario involving Algernon, I’d be much more worried about an AI that competently figures out a UBI scheme that keeps the unemployed out of poverty and combines that with social media influence to really mask the looming problem. That sort of AI would be much more likely to evade detection of malign intent, and could wait for just the right time to “flick the off-switch” and make all the humans who had become dependent on it even for basic survival (ideally for a generation or more) completely helpless and bewildered. Think of the TV series “Battlestar Galactica” and how the biggest Cylon trump card in the first episode is [SPOILER] being able to disable almost all of the enemy aircraft and defenses through prior hacking infiltration. I feel like, for a really competent malign AI, that is more how things would feel leading up to the AI takeover—utopia, utopia, utopia, until one day everything just stops working and AI machinery is doing its own thing.
These are good thought-experiments, although, regarding the first scenario involving Algernon, I’d be much more worried about an AI that competently figures out a UBI scheme that keeps the unemployed out of poverty and combines that with social media influence to really mask the looming problem. That sort of AI would be much more likely to evade detection of malign intent, and could wait for just the right time to “flick the off-switch” and make all the humans who had become dependent on it even for basic survival (ideally for a generation or more) completely helpless and bewildered. Think of the TV series “Battlestar Galactica” and how the biggest Cylon trump card in the first episode is [SPOILER] being able to disable almost all of the enemy aircraft and defenses through prior hacking infiltration. I feel like, for a really competent malign AI, that is more how things would feel leading up to the AI takeover—utopia, utopia, utopia, until one day everything just stops working and AI machinery is doing its own thing.