Why would it backtrack (or what do you mean by backtrack)? Eventually, it observes that w = false (that “ON” went through unchanged) and that its actions are no longer beneficial, so it just stops doing anything, right? The process terminates or it goes to standby?
I think the presumption is that the case where the “ON” signal goes thru normally and the case where the “ON” signal is overwritten by a thermodynamic miracle… into exactly the same “ON” signal are equivalent. That is that after the “ON” signal has gone though the AI would behave identically to an AI that was not indifferent to worlds where the thermodynamic miracle did not occur.
The reason for this is that although the chance that the “ON” signal was overwritten into exactly the same “ON” signal is tiny, it is the only remaining possible world that the AI cares about so it will act as if that is what it believes.
I think the presumption is that the case where the “ON” signal goes thru normally and the case where the “ON” signal is overwritten by a thermodynamic miracle… into exactly the same “ON” signal are equivalent. That is that after the “ON” signal has gone though the AI would behave identically to an AI that was not indifferent to worlds where the thermodynamic miracle did not occur.
The reason for this is that although the chance that the “ON” signal was overwritten into exactly the same “ON” signal is tiny, it is the only remaining possible world that the AI cares about so it will act as if that is what it believes.