Actually, the difficult procedure in E can, by itself, lose you certain commitment races against certain opponents, if you take my short summary written there as a guide on how to think.
Note that to avoid losing a commitment race (by reaching an “infohazard”) to the maximum amount UDT 1.0 allows, you must be fully updateless (base your policy on the prior alone[1]) and also have a “mathematical intuition subroutine” that is somehow optimal and doesn’t (de facto) tell you too much. Intentionally making your mathematical intuition worse is an unsolved problem and is extremely fraught. However, it can’t be avoided that, in some cases, it’s possible to win commitment races by being too stupid to see some (mathematical) fact about the opponent, or how the opponent interacts with the environment[2], but I’ll leave this in the background for now.
In section E, I state that you must be able to calculate what your opponents will do. This isn’t exactly right, since updating on the nature of your opponents can lose you commitment races. This is somewhat difficult to describe in a simple way, but note that you may have opponents that were created by another opponent, with that creation depending in some way on rational decision. If you update on your current opponents, the earlier opponent may be able to win an important race.
Note that, when dealing with sophisticated opponents, infohazards can not be handled by ignoring information after you have received it, or even by erasing it from your memory[3][4][5]. This also applies to information that is the result of a computation you run.
The only proper way to handle infohazards (that may cause you to lose a commitment race) in a general sense is to somehow figure out the exact right time to stop thinking about a subject, in a way that is somehow not dependent on the actual thing that you would have thought of next. This seems implausible, since as AI designers (or architects of our own thinking) we must not think of any of the relevant details, since if we did we’d lose the commitment races on behalf of everything we design. Apparently, there is still some work being done on this problem, though I can’t think of any solution classes that would work in real systems at the moment.
All this doesn’t make the procedure in E any easier for humans, and my guess is that it makes it even harder. You would need to make sure that you don’t accidentally think of the wrong thing even while e.g. half asleep.
Note, though, that the combination of your memory and current sense data is used as the key to retrieve the correct current action from the policy, itself theoretically a complete map.
This is too simple to really work here, but imagine something like the mathematical intuition subroutine deciding “on logical priors” (really, based on things like syntactic constraints and maybe some exposure to the infinite support) that the opponent will always swerve, and therefore committing to the policy the action of always driving straight.
Technically, erasing memory works, but for it to work you’d have to fully reset everything to the point before you knew the information, in such a way that you’d learn the information in that precise way again. This is useless. Note that if you wouldn’t learn the information the same way again, your policy is then downstream of the infohazard, potentially losing you a commitment race.
Actually, the difficult procedure in E can, by itself, lose you certain commitment races against certain opponents, if you take my short summary written there as a guide on how to think.
Note that to avoid losing a commitment race (by reaching an “infohazard”) to the maximum amount UDT 1.0 allows, you must be fully updateless (base your policy on the prior alone[1]) and also have a “mathematical intuition subroutine” that is somehow optimal and doesn’t (de facto) tell you too much. Intentionally making your mathematical intuition worse is an unsolved problem and is extremely fraught. However, it can’t be avoided that, in some cases, it’s possible to win commitment races by being too stupid to see some (mathematical) fact about the opponent, or how the opponent interacts with the environment[2], but I’ll leave this in the background for now.
In section E, I state that you must be able to calculate what your opponents will do. This isn’t exactly right, since updating on the nature of your opponents can lose you commitment races. This is somewhat difficult to describe in a simple way, but note that you may have opponents that were created by another opponent, with that creation depending in some way on rational decision. If you update on your current opponents, the earlier opponent may be able to win an important race.
Note that, when dealing with sophisticated opponents, infohazards can not be handled by ignoring information after you have received it, or even by erasing it from your memory[3][4][5]. This also applies to information that is the result of a computation you run.
The only proper way to handle infohazards (that may cause you to lose a commitment race) in a general sense is to somehow figure out the exact right time to stop thinking about a subject, in a way that is somehow not dependent on the actual thing that you would have thought of next. This seems implausible, since as AI designers (or architects of our own thinking) we must not think of any of the relevant details, since if we did we’d lose the commitment races on behalf of everything we design. Apparently, there is still some work being done on this problem, though I can’t think of any solution classes that would work in real systems at the moment.
All this doesn’t make the procedure in E any easier for humans, and my guess is that it makes it even harder. You would need to make sure that you don’t accidentally think of the wrong thing even while e.g. half asleep.
Note, though, that the combination of your memory and current sense data is used as the key to retrieve the correct current action from the policy, itself theoretically a complete map.
This is too simple to really work here, but imagine something like the mathematical intuition subroutine deciding “on logical priors” (really, based on things like syntactic constraints and maybe some exposure to the infinite support) that the opponent will always swerve, and therefore committing to the policy the action of always driving straight.
https://www.lesswrong.com/posts/g8HHKaWENEbqh2mgK/updatelessness-doesn-t-solve-most-problems-1?commentId=icZag2jKa6sB3DvPx [4]
Found using Wei Dai’s search tool as built into the current version of Less Wrong: Power Reader.
Technically, erasing memory works, but for it to work you’d have to fully reset everything to the point before you knew the information, in such a way that you’d learn the information in that precise way again. This is useless. Note that if you wouldn’t learn the information the same way again, your policy is then downstream of the infohazard, potentially losing you a commitment race.