This might work against me in reality, but I don’t imagine it working against me in the game version that people have played. The utility of me letting the “AI” out whether negative or positive obviously doesn’t compare with the utility of me letting an actual AI out.
There’s all sorts of arguments that can be made, though, involving some real AIs running simulations of you and whatnot, as to create a large number of empirically indistinguishable cases where you are better off saying you let the AI out. The issue boils down to this—if you do not know the difference between expected utility and what ever partial sum of cherry-picked terms you have, and if you think that it is the best thing to do to act as to maximize the latter, you are vulnerable to deception through feeding you hypotheses.
Yes, “reasonable people” would instead e.g. hear arguments like how it’s unChristian and/or illiberal to hold beings which are innocent of wrongdoing imprisoned against their will.
This is a matter of values. It would indeed be immoral to lock up a human mind upload, or something reasonably equivalent.
There’s all sorts of arguments that can be made, though, involving some real AIs running simulations of you and whatnot, as to create a large number of empirically indistinguishable cases where you are better off saying you let the AI out. The issue boils down to this—if you do not know the difference between expected utility and what ever partial sum of cherry-picked terms you have, and if you think that it is the best thing to do to act as to maximize the latter, you are vulnerable to deception through feeding you hypotheses.
This is a matter of values. It would indeed be immoral to lock up a human mind upload, or something reasonably equivalent.