Moreover, the nuclear weapons situation is much simpler than the AI situation. Pretty much everyone agrees that a nuclear weapon going off in an inhabited area is a big deal that can quickly make life worse for all involved. It is not the case that everyone agrees that general AI is a such a big deal. All the official nuclear powers know that there will be a significant negative response directed at them if they bomb anyone else. They do not know this about AI.
It will be probably much easier to use the AI against someone secretly.
You could try to drop an atomic bomb on someone without them knowing who dropped the bomb on them. But you cannot drop an atomic bomb on them without them knowing that someone dropped the bomb on them.
But you could give your AI a task to invent ways how to move things closer to your desired outcome without creating suspicion. The obvious options would be to make it happen as a “natural” outcome, or to cast the suspicion on someone else, or maybe try to reach the goal in a way that will make people believe it didn’t happen or that it wasn’t your goal at all. (A superhuman AI could find yet more options; some of them could be incomprehensive to humans. Also options like: the whole world turns into utter chaos; by the way your original goal is completed, but everyone is now too busy and too confused to even notice it or care about it.) How is anyone going to punish that?
Moreover, the nuclear weapons situation is much simpler than the AI situation. Pretty much everyone agrees that a nuclear weapon going off in an inhabited area is a big deal that can quickly make life worse for all involved. It is not the case that everyone agrees that general AI is a such a big deal. All the official nuclear powers know that there will be a significant negative response directed at them if they bomb anyone else. They do not know this about AI.
It will be probably much easier to use the AI against someone secretly.
You could try to drop an atomic bomb on someone without them knowing who dropped the bomb on them. But you cannot drop an atomic bomb on them without them knowing that someone dropped the bomb on them.
But you could give your AI a task to invent ways how to move things closer to your desired outcome without creating suspicion. The obvious options would be to make it happen as a “natural” outcome, or to cast the suspicion on someone else, or maybe try to reach the goal in a way that will make people believe it didn’t happen or that it wasn’t your goal at all. (A superhuman AI could find yet more options; some of them could be incomprehensive to humans. Also options like: the whole world turns into utter chaos; by the way your original goal is completed, but everyone is now too busy and too confused to even notice it or care about it.) How is anyone going to punish that?