So we would need to figure out how to robustly get an honest signal from such an experiment, which still seems quite hard. But perhaps it’s easier than solving the full alignment problem before the first shot.
IMO this is a ‘additional line of defense’ boxing strategy instead of simplification.
Note that in the traditional version, the ‘dud’ bit of the bomb can only be the trigger; a bomb that absorbs the photon but then explodes isn’t distinguishable from a bomb that absorbs the photon and then doesn’t explode (because of an error deeper in the bomb).
But let’s suppose the quantum computing folks can come up with something like this, where we keep some branches entangled and run analysis of the AI code in only one branch, causing an explosion there but affecting the total outcome in all branches. [This seems pretty implausible to me that you manage to maintain entanglement despite that much impact on the external world, but maybe it’s possible.] Then 1) as you point out, we need to ensure that the AI doesn’t realize that what it needs to output in that branch and 2) need some sort of way to evaluate “did the AI pass our checks or not?”.
IMO this is a ‘additional line of defense’ boxing strategy instead of simplification.
Note that in the traditional version, the ‘dud’ bit of the bomb can only be the trigger; a bomb that absorbs the photon but then explodes isn’t distinguishable from a bomb that absorbs the photon and then doesn’t explode (because of an error deeper in the bomb).
But let’s suppose the quantum computing folks can come up with something like this, where we keep some branches entangled and run analysis of the AI code in only one branch, causing an explosion there but affecting the total outcome in all branches. [This seems pretty implausible to me that you manage to maintain entanglement despite that much impact on the external world, but maybe it’s possible.] Then 1) as you point out, we need to ensure that the AI doesn’t realize that what it needs to output in that branch and 2) need some sort of way to evaluate “did the AI pass our checks or not?”.
But, 2 is “the whole problem”!
Thanks!