There are Löbian situations and Gödelian situations. In the Löbian situation, the simulation makes no difference to the outcome. You’re hungry, you just made breakast, and the simulation predicts you will eat the breakfast because there’s no perverse impulse present in your psyche right now, that would drive you to go hungry just to contradict the prediction. The Gödelian situations arise when knowing the prediction does matter, factually or motivationally. If an agent has the intention and the power to always do A if you say B and B if you say A, then there’s no way to say A or B and get it right. But you can “predict”, for example, that it will listen to your prediction and … become annoyed, because you didn’t say A or B.
There are Löbian situations and Gödelian situations. In the Löbian situation, the simulation makes no difference to the outcome. You’re hungry, you just made breakast, and the simulation predicts you will eat the breakfast because there’s no perverse impulse present in your psyche right now, that would drive you to go hungry just to contradict the prediction. The Gödelian situations arise when knowing the prediction does matter, factually or motivationally. If an agent has the intention and the power to always do A if you say B and B if you say A, then there’s no way to say A or B and get it right. But you can “predict”, for example, that it will listen to your prediction and … become annoyed, because you didn’t say A or B.