It’s interesting not being my past self and being able to understand that problem.
Because strategies based on simulation of the predictor are opaque to the predictor, while strategies based on high-level reasoning are transparent to the predictor, the problem is no longer just determined by the agent’s final decisions—it’s not in the same class as Newcomb’s problem anymore. It’s a computation-dependent problem, but it’s not quite in the same class as a two box problem that rewards you for picking options alphabetically (the AlphaBeta problem :D).
I agree with Vladimir’s idea that the UDT agent formalized in your original post might still be able to handle it without any extensions, if it finds a short proof that includes some gnarly self-reference (See note). The AlphaBeta problem, on the other hand, is unwinnable for any utility-maximizer without the ability to suspend its own utility-maximizing. This is interesting, because it seems like the ASP problem is also more “reasonable” than the AlphaBeta problem.
(note): As a sketch: The existence of a proof that one-boxing means maximum utility that is less than N is equivalent to both boxes being filled, and if no such proof exists, only one box is filled. If the proven-maximum-utility-meaning action is always taken, then the maximum available utility is when one box is taken and both boxes are full. The optimal action is always This proof is less than N. By the power vested in me by Loeb’s theorem...
the problem is no longer just determined by the agent’s final decisions
Right.
It’s interesting not being my past self and being able to understand that problem.
Congratulations :-) Now I’ll do the thing that Wei usually does, and ask you if something specific in the problem description was tripping you up? How would you rephrase it to make your past self understand it faster?
How would you rephrase it to make your past self understand it faster?
Include a link to Wei Dai’s analysis of the absentminded driver problem, with a short blurb explaining why your theorem-proving agent is like that and not like CDT, maybe. But that would have had only a faint hope of success :P
It’s interesting not being my past self and being able to understand that problem.
Because strategies based on simulation of the predictor are opaque to the predictor, while strategies based on high-level reasoning are transparent to the predictor, the problem is no longer just determined by the agent’s final decisions—it’s not in the same class as Newcomb’s problem anymore. It’s a computation-dependent problem, but it’s not quite in the same class as a two box problem that rewards you for picking options alphabetically (the AlphaBeta problem :D).
I agree with Vladimir’s idea that the UDT agent formalized in your original post might still be able to handle it without any extensions, if it finds a short proof that includes some gnarly self-reference (See note). The AlphaBeta problem, on the other hand, is unwinnable for any utility-maximizer without the ability to suspend its own utility-maximizing. This is interesting, because it seems like the ASP problem is also more “reasonable” than the AlphaBeta problem.
(note): As a sketch: The existence of a proof that one-boxing means maximum utility that is less than N is equivalent to both boxes being filled, and if no such proof exists, only one box is filled. If the proven-maximum-utility-meaning action is always taken, then the maximum available utility is when one box is taken and both boxes are full. The optimal action is always This proof is less than N. By the power vested in me by Loeb’s theorem...
Right.
Congratulations :-) Now I’ll do the thing that Wei usually does, and ask you if something specific in the problem description was tripping you up? How would you rephrase it to make your past self understand it faster?
Include a link to Wei Dai’s analysis of the absentminded driver problem, with a short blurb explaining why your theorem-proving agent is like that and not like CDT, maybe. But that would have had only a faint hope of success :P