When you consider expected utility in a broader context, expected utility in a smaller event (a particular scenario) only contributes proportionally to the probability of that event. The usual heuristic of locally (independently) maximizing expected utility of particular scenarios stops working if your decisions within the scenario are able to control the probability of the scenario. And this happens naturally when the decision as to whether to set up the scenario is based on predictions of your hypothetical decisions within the scenario.
One way of influencing whether a hypothetical scenario is implemented is by making it impossible. When faced with the hypothetical scenario, make a decision that contradicts the assumptions of the hypothetical (such as a prediction of your decision, or something based on that prediction). That would make the hypothetical self-contradictory, impossible to implement.
If you find yourself in an “independent hypothetical”, the prior probability of this event is not apparent, but it matters if it can be controlled, and it can be controlled by for example making decisions that contradict the assumptions of the hypothetical. If a certain decision increases expected utility of a game conditional on the game being played, but decreases the probability that the game gets played, this can well make the outcome worse, if the alternative to the game being played is less lucrative.
In our case, if you make a decision that implies that a prime number is composite, given the assumption that the prediction is accurate, this makes the scenario (in the case where the number is prime) impossible, its measure zero, and its conditional expected utility irrelevant (it doesn’t contribute to the overall expected utility). Alternatively, if we permit some probability of error in the prediction, this significantly reduces the probability of the scenario, as it now requires the prediction to be in error. (By eliminating the case where the Lottery number is prime, you increase the conditional probability of the number being composite, given the assumption that the game gets played, but you don’t change the absolute probability of the number being composite, outside the assumption that the game gets played.)
It seems like that Omega’s actions make the primality of 1033 a constant you can ambiently control. (Though, to be fair, I don’t understand that post very well, and it’s probably true that if I did I wouldn’t have this question.)
You control the posterior probability of 1033 being composite, conditional on the event of playing the game where both numbers are 1033. Seen from outside that event (i.e. without conditioning on it), what you are controlling is the probability of Omega’s number being composite, but not the probability of Lottery number being composite. The expected value of composite 1033 comes from the event of Lottery number being composite, but you don’t control the probability of this event. Instead, you control the conditional probability of this event given another event (the game with both 1033). This conditional probability is therefore misleading for the purposes of overall expected utility maximization, where outcomes are weighed by their absolute (prior) probability, not conditional probability on arbitrary sub-events.
So, I can definitely see why this applies to the Ultimate Newcomb’s Problem. As a contrast to help me understand it, I’ve adjusted this problem so that P(playing the game, both numbers 1033|playing the game) = ~1. See my response to Manfred here.
It is, of course, possible that your algorithm results in you not playing the game at all—but if Omega does this every year, say, then the winners will be the ones who make the most when the numbers are the same, since no other option exists.
(If Omega can choose whether to let you play the game, which is the only game available to you, and the game has the rule that the numbers must be equal, then you should two-box to improve your chances of being allowed to play when the Lottery number is composite, and thus capture more of the composite Lottery outcomes. This works not because you are increasing the conditional probability of the number you get being composite (though you do), but because you are increasing the prior probability of playing the game with a composite Lottery number.)
Okay. I feel much more confident in my answer, then :P.
Double-checking: What if the lottery picks primes and composites with equal frequency?
… then, on average, you’ll get into half the games and make twice the money, so you should still two-box. I think.
So, assuming you/Manfred don’t poke holes into this, I’ll edit the original post. Thanks—having a clear, alternative two-box problem makes understanding the original much easier.
When you consider expected utility in a broader context, expected utility in a smaller event (a particular scenario) only contributes proportionally to the probability of that event. The usual heuristic of locally (independently) maximizing expected utility of particular scenarios stops working if your decisions within the scenario are able to control the probability of the scenario. And this happens naturally when the decision as to whether to set up the scenario is based on predictions of your hypothetical decisions within the scenario.
One way of influencing whether a hypothetical scenario is implemented is by making it impossible. When faced with the hypothetical scenario, make a decision that contradicts the assumptions of the hypothetical (such as a prediction of your decision, or something based on that prediction). That would make the hypothetical self-contradictory, impossible to implement.
If you find yourself in an “independent hypothetical”, the prior probability of this event is not apparent, but it matters if it can be controlled, and it can be controlled by for example making decisions that contradict the assumptions of the hypothetical. If a certain decision increases expected utility of a game conditional on the game being played, but decreases the probability that the game gets played, this can well make the outcome worse, if the alternative to the game being played is less lucrative.
In our case, if you make a decision that implies that a prime number is composite, given the assumption that the prediction is accurate, this makes the scenario (in the case where the number is prime) impossible, its measure zero, and its conditional expected utility irrelevant (it doesn’t contribute to the overall expected utility). Alternatively, if we permit some probability of error in the prediction, this significantly reduces the probability of the scenario, as it now requires the prediction to be in error. (By eliminating the case where the Lottery number is prime, you increase the conditional probability of the number being composite, given the assumption that the game gets played, but you don’t change the absolute probability of the number being composite, outside the assumption that the game gets played.)
Pardon me, how does this differ from the examples in this post?
What analogy are you thinking about (between which points, more specifically)? These discussions don’t seem particularly close.
It seems like that Omega’s actions make the primality of 1033 a constant you can ambiently control. (Though, to be fair, I don’t understand that post very well, and it’s probably true that if I did I wouldn’t have this question.)
You control the posterior probability of 1033 being composite, conditional on the event of playing the game where both numbers are 1033. Seen from outside that event (i.e. without conditioning on it), what you are controlling is the probability of Omega’s number being composite, but not the probability of Lottery number being composite. The expected value of composite 1033 comes from the event of Lottery number being composite, but you don’t control the probability of this event. Instead, you control the conditional probability of this event given another event (the game with both 1033). This conditional probability is therefore misleading for the purposes of overall expected utility maximization, where outcomes are weighed by their absolute (prior) probability, not conditional probability on arbitrary sub-events.
Ah, I see. (Thanks! Some rigour helps a lot.)
So, I can definitely see why this applies to the Ultimate Newcomb’s Problem. As a contrast to help me understand it, I’ve adjusted this problem so that P(playing the game, both numbers 1033|playing the game) = ~1. See my response to Manfred here.
It is, of course, possible that your algorithm results in you not playing the game at all—but if Omega does this every year, say, then the winners will be the ones who make the most when the numbers are the same, since no other option exists.
(If Omega can choose whether to let you play the game, which is the only game available to you, and the game has the rule that the numbers must be equal, then you should two-box to improve your chances of being allowed to play when the Lottery number is composite, and thus capture more of the composite Lottery outcomes. This works not because you are increasing the conditional probability of the number you get being composite (though you do), but because you are increasing the prior probability of playing the game with a composite Lottery number.)
(Responded before your edit, so doubly-curious about your answer.)
Okay. I feel much more confident in my answer, then :P.
Double-checking: What if the lottery picks primes and composites with equal frequency?
… then, on average, you’ll get into half the games and make twice the money, so you should still two-box. I think.
So, assuming you/Manfred don’t poke holes into this, I’ll edit the original post. Thanks—having a clear, alternative two-box problem makes understanding the original much easier.