The problem, which various people have pointed out, is that this implies an intelligence capable of taking over the world, but not capable of working out that when a human says pursue a certain goal, they would not want this goal to be pursued in a way that leads to the destruction of the world.
The entity providing the goals for the AI wouldn’t have to be a human, it might instead be a corporation. A reasonable goal for such an AI might be to ‘maximize shareholder value’. The shareholders are not humans either, and what they value is only money.
The entity providing the goals for the AI wouldn’t have to be a human, it might instead be a corporation. A reasonable goal for such an AI might be to ‘maximize shareholder value’. The shareholders are not humans either, and what they value is only money.
Encouragingly, corporations seem to have am impetus to keep blue-sky thinking and direct execution somewhat separate.