The Counterfactual Mugging is my favorite decision theory problem, and it’s the problem that got me started reading LessWrong in the first place. In short,
Imagine that one day, Omega comes to you and says that it has just tossed a fair coin, and given that the coin came up tails, it decided to ask you to give it $100. Whatever you do in this situation, nothing else will happen differently in reality as a result. Naturally you don’t want to give up your $100. But see, Omega tells you that if the coin came up heads instead of tails, it’d give you $10000, but only if you’d agree to give it $100 if the coin came up tails.
Since hearing the problem for the first time, I have flip-flopped on what should be done many times. I won’t go into the details, but the general consensus on this forum (as far as I can tell) is that you should pay the $100 and that UDT tells you to pay the $100.
While I admit I found some of these arguments (especially MBlume’s) quite persuasive, and my position for a good while was that one should pay, I still had this intuition in the back of my mind telling me that rational agents should win. Giving Omega $100 for no real gain sure doesn’t sound like winning to me, and if UDT says to pay the $100, that means that UDT is wrong, not that we should change our preferences to including paying the $100 in this scenario.
But there is a third option, one that allows you to save your $100 while still following UDT: show that UDT doesn’t tell you to pay.
When the Counterfactual Mugging is usually presented, it would appear that there are two possible scenarios, each with probability 0.5: Omega exists and the coin landed heads, and Omega exists and the coin landed tails. Thus, by UDT, we would want to precommit to paying should the coin land tails, and so when the coin lands tails, we pay.
However, those are not the only two scenarios. Before we learn about the counterfactual mugging, there is a third option: Nomega exists, a being who will pay $10000 to anyone who doesn’t pay Omega when counterfactually mugged, and gives no money to someone who does. Our new view of the world:
Scenario
Probability
U(Precommit)
U(Don’t Preccomit)
Omega, Heads
0.25
10000
0
Omega, Tails
0.25
-100
0
Nomega
0.5
0
10000
Expected Value
2475
5000
Thus a rational agent running UDT should NOT precommit to paying a counterfactual mugger. Once we learn that we live in a universe where Omega, rather than Nomega, exists, it may look tempting to pay. But at that point, we have also leaned that we live a universe in which the coin is a tail, rather than a head, and so we still should not pay.
Some caveats:
Firstly, as always, I may be completely overlooking something, in which case this entire arguments is flawed.
Secondly, there is some intuition that Omega seems more real / more likely to exists then Nomega does, which may through the calculus off. Considering Omega and Nomega as equally likely options seems to open you up to getting Pascal Mugged and Wagered all over the place. However, I have no real way to formalizing why Omega might be more real then Nomega. (And, in fact, Pascals Wager is about not making decisions, precommitments, etc., on the basis that some God may exist, because its equally likely that some God with opposite incentives exists. Actually, the counterfactual mugging is starting to smell a lot like Pascal’s Wager.)
Irrespective of point 2, even if we decide there is a reason to believe in Omega more than Nomega, I still feel like this idea makes the case for UDT telling us to pay a lot more shaky, and relies on multiplying lots of numbers, which makes me nervous.
Finally, this same argument might be able to be applied to the Counterfactual Prisoner’s Dilemma to tell you not to pay, even though I relatively certain that one should pay in that scenario.
UDT might not pay a Counterfactual Mugger
The Counterfactual Mugging is my favorite decision theory problem, and it’s the problem that got me started reading LessWrong in the first place. In short,
Since hearing the problem for the first time, I have flip-flopped on what should be done many times. I won’t go into the details, but the general consensus on this forum (as far as I can tell) is that you should pay the $100 and that UDT tells you to pay the $100.
While I admit I found some of these arguments (especially MBlume’s) quite persuasive, and my position for a good while was that one should pay, I still had this intuition in the back of my mind telling me that rational agents should win. Giving Omega $100 for no real gain sure doesn’t sound like winning to me, and if UDT says to pay the $100, that means that UDT is wrong, not that we should change our preferences to including paying the $100 in this scenario.
But there is a third option, one that allows you to save your $100 while still following UDT: show that UDT doesn’t tell you to pay.
When the Counterfactual Mugging is usually presented, it would appear that there are two possible scenarios, each with probability 0.5: Omega exists and the coin landed heads, and Omega exists and the coin landed tails. Thus, by UDT, we would want to precommit to paying should the coin land tails, and so when the coin lands tails, we pay.
However, those are not the only two scenarios. Before we learn about the counterfactual mugging, there is a third option: Nomega exists, a being who will pay $10000 to anyone who doesn’t pay Omega when counterfactually mugged, and gives no money to someone who does. Our new view of the world:
Thus a rational agent running UDT should NOT precommit to paying a counterfactual mugger. Once we learn that we live in a universe where Omega, rather than Nomega, exists, it may look tempting to pay. But at that point, we have also leaned that we live a universe in which the coin is a tail, rather than a head, and so we still should not pay.
Some caveats:
Firstly, as always, I may be completely overlooking something, in which case this entire arguments is flawed.
Secondly, there is some intuition that Omega seems more real / more likely to exists then Nomega does, which may through the calculus off. Considering Omega and Nomega as equally likely options seems to open you up to getting Pascal Mugged and Wagered all over the place. However, I have no real way to formalizing why Omega might be more real then Nomega. (And, in fact, Pascals Wager is about not making decisions, precommitments, etc., on the basis that some God may exist, because its equally likely that some God with opposite incentives exists. Actually, the counterfactual mugging is starting to smell a lot like Pascal’s Wager.)
Irrespective of point 2, even if we decide there is a reason to believe in Omega more than Nomega, I still feel like this idea makes the case for UDT telling us to pay a lot more shaky, and relies on multiplying lots of numbers, which makes me nervous.
Finally, this same argument might be able to be applied to the Counterfactual Prisoner’s Dilemma to tell you not to pay, even though I relatively certain that one should pay in that scenario.