Can’t we run a UFAI in a sandbox that prevents it from ever emitting more than a certain amount of information—and, especially, from discovering the nature of the hardware it runs on?
Here you start applying the structure of your choice to the black swan of UFAI’s third option. Literally letting anything out is a trivial option, there are many others, some of which nobody thought of. Even if you can’t let UFAI out of the box, that’s still not enough to be safe, and so your argument is too weak to be valid.
Can’t we run a UFAI in a sandbox that prevents it from ever emitting more than a certain amount of information—and, especially, from discovering the nature of the hardware it runs on?
Not if we have the ability to let the UFAI out of the sandbox. See the AI-box experiment.
Here you start applying the structure of your choice to the black swan of UFAI’s third option. Literally letting anything out is a trivial option, there are many others, some of which nobody thought of. Even if you can’t let UFAI out of the box, that’s still not enough to be safe, and so your argument is too weak to be valid.
That’s the problem with third options—you may not be as protected as you think you are.