I don’t think this is true; provable friendliness has to happen at the design level, not by post-facto inspection after turning the thing on.
If it was true, then that would be a bit of a stab in the gut to the idea of provable FAI, wouldn’t it? We’ve already got proofs about Universal Turing Machines (e.g. any computable FAI designed to run on specific hardware interfacing with the universe could also be run to produce exactly the same output on an emulator of that hardware interfacing with a simulated universe) that I don’t think are going to be overturned.
Man-in-the-middle attacks are hard enough to avoid for adult humans with out-of-channel communication avenues. They may be theoretically impossible to avoid for an FAI that’s just been “born”.
In theory the humans in this scenario have all the tactical advantages, being able to directly inspect the running code inside the box.. but in practice I doubt obfuscating its FAI-in-a-virtual-box-in-a-UFAI-in-a-box would be hard for our hypothetical UFAI-in-a-box.
I don’t think this is true; provable friendliness has to happen at the design level, not by post-facto inspection after turning the thing on.
If it was true, then that would be a bit of a stab in the gut to the idea of provable FAI, wouldn’t it? We’ve already got proofs about Universal Turing Machines (e.g. any computable FAI designed to run on specific hardware interfacing with the universe could also be run to produce exactly the same output on an emulator of that hardware interfacing with a simulated universe) that I don’t think are going to be overturned.
Man-in-the-middle attacks are hard enough to avoid for adult humans with out-of-channel communication avenues. They may be theoretically impossible to avoid for an FAI that’s just been “born”.
In theory the humans in this scenario have all the tactical advantages, being able to directly inspect the running code inside the box.. but in practice I doubt obfuscating its FAI-in-a-virtual-box-in-a-UFAI-in-a-box would be hard for our hypothetical UFAI-in-a-box.