It’s hard to predict the behavior of something smarter than you
Actually, predicting the behaviour of a superintelligence is a pretty trivial engineering feat—provided you are prepared to make it act a little bit more slowly.
Just get another agent to intercept all its motor outputs, delay them, and then print them all out all a little bit before they will be perfomed. Presto: a prediction of what the machine is about to do. Humans could use those predictions to veto the proposed actions—if they so chose.
You can’t control what you can’t predict,
Humans can’t predict what Deep Blue will do—but they can turn it off.
I think your argument collapses around about here.
Actually, predicting the behaviour of a superintelligence is a pretty trivial engineering feat—provided you are prepared to make it act a little bit more slowly.
Just get another agent to intercept all its motor outputs, delay them, and then print them all out all a little bit before they will be perfomed. Presto: a prediction of what the machine is about to do. Humans could use those predictions to veto the proposed actions—if they so chose.
Humans can’t predict what Deep Blue will do—but they can turn it off.
I think your argument collapses around about here.
If there is one thing Deep Blue is good at it is doing a deep search multiple moves ahead.
I think DB is going to see that one coming!
...but what’s it going to do? Castle? ;-) Unpredictable != Uncontrollable.