If I’m understanding Stuart’s proposal correctly, the AI is not deceived about how common the stochastic event is. It’s just made not to care about worlds in which it doesn’t happen. This is very similar in effect to making it think the event is common, but (arguably, at least) it doesn’t involve any false beliefs.
(I say “arguably” because, e.g., doing this will tend to make the AI answer “yes” to “do you think the event will happen?”, plan on the basis that it will happen, etc., and perhaps making something behave exactly as it would if it believed X isn’t usefully distinguishable from making it believe X.)
The problem is that the definition of the event not happening is probably too strict. The worlds that the AI doesn’t care about don’t exist its decision-making purposes, and in the world that the AI cares about, the AI assigns high probability to hypotheses like “the users can see the message even before I send it through the noisy channel”.
If I’m understanding Stuart’s proposal correctly, the AI is not deceived about how common the stochastic event is. It’s just made not to care about worlds in which it doesn’t happen. This is very similar in effect to making it think the event is common, but (arguably, at least) it doesn’t involve any false beliefs.
(I say “arguably” because, e.g., doing this will tend to make the AI answer “yes” to “do you think the event will happen?”, plan on the basis that it will happen, etc., and perhaps making something behave exactly as it would if it believed X isn’t usefully distinguishable from making it believe X.)
The problem is that the definition of the event not happening is probably too strict. The worlds that the AI doesn’t care about don’t exist its decision-making purposes, and in the world that the AI cares about, the AI assigns high probability to hypotheses like “the users can see the message even before I send it through the noisy channel”.