What about “stop executing/writing code or sending signals?”
As a side note, I consider that we’re pretty much doomed anyways if the AI cannot conceive of a way to deposit 100 USD into a bank account without using nanotech because that’s made the goal hard for the AI, which will cause it to pose similar problems to that of an AI with an unbounded utility function. The task has to be easy for it to be an interesting problem.
Even if it can deposit $100 with 99.9% probability without doing anything fancy, maybe it can add another .099% by using nanotech. Or by starting a nuclear war to distract anything that might get in its way (destroying the bank five minutes later, but so what). (Credit to Carl Shulman for that suggestion.)
From my estimation, all it needs to do is find out how to hack a bank. If it can’t hack one bank, it can try to hack any other bank that it has access to, considering that almost all banks have more than 100 USD in them. It could even find and spread a keylogger to get someone’s credit card info.
Such techniques (which are repeatable within a very short timespan, faster than humans can react) seem much more sure than using nanotech or starting a nuclear war. I don’t think that distracting humans would really improve its chances of success because it’s incredibly doubtful that human’s could react so fast to so many different cyber-attacks.
Possible, true, but the chances of this happening seem uber-low.
How do we describe this shutdown command? “Shut down anything you have control over” sounds like the sort of event we’re trying to avoid.
What about “stop executing/writing code or sending signals?”
As a side note, I consider that we’re pretty much doomed anyways if the AI cannot conceive of a way to deposit 100 USD into a bank account without using nanotech because that’s made the goal hard for the AI, which will cause it to pose similar problems to that of an AI with an unbounded utility function. The task has to be easy for it to be an interesting problem.
Even if it can deposit $100 with 99.9% probability without doing anything fancy, maybe it can add another .099% by using nanotech. Or by starting a nuclear war to distract anything that might get in its way (destroying the bank five minutes later, but so what). (Credit to Carl Shulman for that suggestion.)
From my estimation, all it needs to do is find out how to hack a bank. If it can’t hack one bank, it can try to hack any other bank that it has access to, considering that almost all banks have more than 100 USD in them. It could even find and spread a keylogger to get someone’s credit card info.
Such techniques (which are repeatable within a very short timespan, faster than humans can react) seem much more sure than using nanotech or starting a nuclear war. I don’t think that distracting humans would really improve its chances of success because it’s incredibly doubtful that human’s could react so fast to so many different cyber-attacks.
Possible, true, but the chances of this happening seem uber-low.