Predictors exist: CDT going bonkers… forever

I’ve been want­ing to get a bet­ter ex­am­ple of CDT (causal de­ci­sion the­ory) mis­be­hav­ing, where the be­havi­our is more clearly sub­op­ti­mal than it is in the New­comb prob­lem (which many peo­ple don’t seem to ac­cept as CDT be­ing sub­op­ti­mal), and sim­pler to grasp than Death in Da­m­as­cus.

The “pre­dic­tors ex­ist” problem

So con­sider this sim­ple ex­am­ple: the player is play­ing against Omega, who will pre­dict their ac­tions[1]. The player can take three ac­tions: “zero”, “one”, or “leave”.

If ever they do “leave”, then the ex­per­i­ment is over and they leave. If they choose “zero” or “one”, then Omega will pre­dict their ac­tion, and com­pare this to their ac­tual ac­tion. If the two match, then the player loses util­ity and the game re­peats; if the ac­tion and the pre­dic­tion differs, then the player gains util­ity and the ex­per­i­ment ends.

As­sume that ac­tu­ally Omega is a perfect or quasi-perfect pre­dic­tor, with a good model of the player. An FDT or EDT agent would soon re­al­ise that they couldn’t trick Omega, af­ter a few tries, and would quickly end the game.

But the CDT player would be in­ca­pable of reach­ing this rea­son­ing. What­ever dis­tri­bu­tion they com­pute over Omega’s pre­dic­tion, they will always es­ti­mate that they (the CDT player) have at least a chance of choos­ing the other op­tion[2], for an ex­pected util­ity gain of at least .

Ba­si­cally, the CDT agent can never learn that Omega is a good pre­dic­tor of them­selves[3]. And so they will con­tinue play­ing, and con­tinue los­ing… for ever.

  1. Omega will make this pre­dic­tion not nec­es­sar­ily be­fore the player takes their ac­tion, not even nec­es­sar­ily with­out see­ing this ac­tion, but still makes the pre­dic­tion in­de­pen­dently of this knowl­edge. And that’s enough for CDT. ↩︎

  2. For ex­am­ple, sup­pose the CDT agent es­ti­mates the pre­dic­tion will be “zero” with prob­a­bil­ity , and “one” with prob­a­bil­ity 1-p. Then if , they can say “one”, and have a prob­a­bil­ity of win­ning, in their own view. If , they can say “zero”, and have a sub­jec­tive prob­a­bil­ity of win­ning. ↩︎

  3. The CDT agent has no prob­lem be­liev­ing that Omega is a perfect pre­dic­tor of other agents, how­ever. ↩︎