Sometime ago I figured out a refutation of this kind of reasoning in Counterfactual Mugging, and it seems to apply in Newcomb’s Problem too. It goes as follows:
Imagine another god, Upsilon, that offers you a similar two-box setup—except to get the $2M in the box B, you must be a one-boxer with regard to Upsilon and a two-boxer with regard to Omega. (Upsilon predicts your counterfactual behavior if you’d met Omega instead.) Now you must choose your dispositions wisely because you can’t win money from both gods. The right disposition depends on your priors for encountering Omega or Upsilon, which is a “bead jar guess” because both gods are very improbable. In other words, to win in such problems, you can’t just look at each problem individually as it arises—you need to have the correct prior/predisposition over all possible predictors of your actions, before you actually meet any of them. Obtaining such a prior is difficult, so I don’t really know what I’m predisposed to do in Newcomb’s Problem if I’m faced with it someday.
Omega lets me decide to take only one box after meeting Omega, when I have already updated on the fact that Omega exists, and so I have much better knowledge about which sort of god I’m likely to encounter. Upsilon treats me on the basis of a guess I would subjunctively make without knowledge of Upsilon. It is therefore not surprising that I tend to do much better with Omega than with Upsilon, because the relevant choices being made by me are being made with much better knowledge. To put it another way, when Omega offers me a Newcomb’s Problem, I will condition my choice on the known existence of Omega, and all the Upsilon-like gods will tend to cancel out into Pascal’s Wagers. If I run into an Upsilon-like god, then, I am not overly worried about my poor performance—it’s like running into the Christian God, you’re screwed, but so what, you won’t actually run into one. Even the best rational agents cannot perform well on this sort of subjunctive hypothesis without much better knowledge while making the relevant choices than you are offering them. For every rational agent who performs well with respect to Upsilon there is one who performs poorly with respect to anti-Upsilon.
On the other hand, beating Newcomb’s Problem is easy, once you let go of the idea that to be “rational” means performing a strange ritual cognition in which you must only choose on the basis of physical consequences and not on the basis of correct predictions that other agents reliably make about you, so that (if you choose using this bizarre ritual) you go around regretting how terribly “rational” you are because of the correct predictions that others make about you. I simply choose on the basis of the correct predictions that others make about me, and so I do not regret being rational.
And these questions are highly relevant and realistic, unlike Upsilon; in the future we can expect there to be lots of rational agents that make good predictions about each other.
Omega lets me decide to take only one box after meeting Omega, when I have already updated on the fact that Omega exists, and so I have much better knowledge about which sort of god I’m likely to encounter.
In what sense can you update? Updating is about following a plan, not about deciding on a plan. You already know that it’s possible to observe anything, you don’t learn anything new about environment by observing any given thing. There could be a deep connection between updating and logical uncertainty that makes it a good plan to update, but it’s not obvious what it is.
Intuitively, the notion of updating a map of fixed reality makes sense, but in the context of decision-making, formalization in full generality proves elusive, even unnecessary, so far.
By making a choice, you control the truth value of certain statements—statements about your decision-making algorithm and about mathematical objects depending on your algorithm. Only some of these mathematical objects are part of the “real world”. Observations affect what choices you make (“updating is about following a plan”), but you must have decided beforehand what consequences you want to establish (“[updating is] not about deciding on a plan”). You could have decided beforehand to care only about mathematical structures that are “real”, but what characterizes those structures apart from the fact that you care about them?
This is not a refutation, because what you describe is not about the thought experiment. In the thought experiment, there are no Upsilons, and so nothing to worry about. It is if you face this scenario in real life, where you can’t be given guarantees about the absence of Upsilons, that your reasoning becomes valid. But it doesn’t refute the reasoning about the thought experiment where it’s postulated that there are no Upsilons.
Thanks for dropping the links here. FWIW, I agree with your objection. But at the very least, the people claiming they’re “one-boxers” should also make the distinction you make.
Also, user Nisan tried to argue that various Upsilons and other fauna must balance themselves out if we use the universal prior. We eventually took this argument to email, but failed to move each other’s positions.
OK. I assume the usual (Omega and Upsilon are both reliable and sincere, I can reliably distinguish one from the other, etc.)
Then I can’t see how the game doesn’t reduce to standard Newcomb, modulo a simple probability calculation, mostly based on “when I encounter one of them, what’s my probability of meeting the other during my lifetime?” (plus various “actuarial” calculations).
If I have no information about the probability of encountering either, then my decision may be incorrect—but there’s nothing paradoxical or surprising about this, it’s just a normal, “boring” example of an incomplete information problem.
you need to have the correct prior/predisposition over all possible predictors of
your actions, before you actually meet any of them.
I can’t see why that is—again, assuming that the full problem is explained to you on encountering either Upsilon or Omega, both are truhful, etc. Why can I not perform the appropriate calculations and make an expectation-maximising decision even after Upsilon-Omega has left? Surely Omega-Upsilon can predict that I’m going to do just that and act accordingly, right?
Yes, this is a standard incomplete information problem. Yes, you can do the calculations at any convenient time, not necessarily before meeting Omega. (These calculations can’t use the information that Omega exists, though.) No, it isn’t quite as simple as you state: when you meet Omega, you have to calculate the counterfactual probability of you having met Upsilon instead, and so on.
I’m pretty sure the logic is correct. I do make silly math mistakes sometimes, but I’ve tested this one on Vladimir Nesov and he agrees. No comment from Eliezer yet (this scenario was first posted to decision-theory-workshop).
Then I think the original Newcomb’s Problem should remind you of Pascal’s Wager just as much, and my scenario should be analogous to the refutation thereof. (Thereunto? :-)
Sometime ago I figured out a refutation of this kind of reasoning in Counterfactual Mugging, and it seems to apply in Newcomb’s Problem too. It goes as follows:
Imagine another god, Upsilon, that offers you a similar two-box setup—except to get the $2M in the box B, you must be a one-boxer with regard to Upsilon and a two-boxer with regard to Omega. (Upsilon predicts your counterfactual behavior if you’d met Omega instead.) Now you must choose your dispositions wisely because you can’t win money from both gods. The right disposition depends on your priors for encountering Omega or Upsilon, which is a “bead jar guess” because both gods are very improbable. In other words, to win in such problems, you can’t just look at each problem individually as it arises—you need to have the correct prior/predisposition over all possible predictors of your actions, before you actually meet any of them. Obtaining such a prior is difficult, so I don’t really know what I’m predisposed to do in Newcomb’s Problem if I’m faced with it someday.
Omega lets me decide to take only one box after meeting Omega, when I have already updated on the fact that Omega exists, and so I have much better knowledge about which sort of god I’m likely to encounter. Upsilon treats me on the basis of a guess I would subjunctively make without knowledge of Upsilon. It is therefore not surprising that I tend to do much better with Omega than with Upsilon, because the relevant choices being made by me are being made with much better knowledge. To put it another way, when Omega offers me a Newcomb’s Problem, I will condition my choice on the known existence of Omega, and all the Upsilon-like gods will tend to cancel out into Pascal’s Wagers. If I run into an Upsilon-like god, then, I am not overly worried about my poor performance—it’s like running into the Christian God, you’re screwed, but so what, you won’t actually run into one. Even the best rational agents cannot perform well on this sort of subjunctive hypothesis without much better knowledge while making the relevant choices than you are offering them. For every rational agent who performs well with respect to Upsilon there is one who performs poorly with respect to anti-Upsilon.
On the other hand, beating Newcomb’s Problem is easy, once you let go of the idea that to be “rational” means performing a strange ritual cognition in which you must only choose on the basis of physical consequences and not on the basis of correct predictions that other agents reliably make about you, so that (if you choose using this bizarre ritual) you go around regretting how terribly “rational” you are because of the correct predictions that others make about you. I simply choose on the basis of the correct predictions that others make about me, and so I do not regret being rational.
And these questions are highly relevant and realistic, unlike Upsilon; in the future we can expect there to be lots of rational agents that make good predictions about each other.
In what sense can you update? Updating is about following a plan, not about deciding on a plan. You already know that it’s possible to observe anything, you don’t learn anything new about environment by observing any given thing. There could be a deep connection between updating and logical uncertainty that makes it a good plan to update, but it’s not obvious what it is.
Huh? Updating is just about updating your map. (?) The next sentence I didn’t understand the reasoning of, could you expand?
Intuitively, the notion of updating a map of fixed reality makes sense, but in the context of decision-making, formalization in full generality proves elusive, even unnecessary, so far.
By making a choice, you control the truth value of certain statements—statements about your decision-making algorithm and about mathematical objects depending on your algorithm. Only some of these mathematical objects are part of the “real world”. Observations affect what choices you make (“updating is about following a plan”), but you must have decided beforehand what consequences you want to establish (“[updating is] not about deciding on a plan”). You could have decided beforehand to care only about mathematical structures that are “real”, but what characterizes those structures apart from the fact that you care about them?
Vladimir talks more about his crazy idea in this comment.
Pascal’s Wagers, huh. So your decision theory requires a specific prior?
This is not a refutation, because what you describe is not about the thought experiment. In the thought experiment, there are no Upsilons, and so nothing to worry about. It is if you face this scenario in real life, where you can’t be given guarantees about the absence of Upsilons, that your reasoning becomes valid. But it doesn’t refute the reasoning about the thought experiment where it’s postulated that there are no Upsilons.
(Original thread, my discussion.)
Thanks for dropping the links here. FWIW, I agree with your objection. But at the very least, the people claiming they’re “one-boxers” should also make the distinction you make.
Also, user Nisan tried to argue that various Upsilons and other fauna must balance themselves out if we use the universal prior. We eventually took this argument to email, but failed to move each other’s positions.
Just didn’t want you confusing people or misrepresenting my opinion, so made everything clear. :-)
OK. I assume the usual (Omega and Upsilon are both reliable and sincere, I can reliably distinguish one from the other, etc.)
Then I can’t see how the game doesn’t reduce to standard Newcomb, modulo a simple probability calculation, mostly based on “when I encounter one of them, what’s my probability of meeting the other during my lifetime?” (plus various “actuarial” calculations).
If I have no information about the probability of encountering either, then my decision may be incorrect—but there’s nothing paradoxical or surprising about this, it’s just a normal, “boring” example of an incomplete information problem.
I can’t see why that is—again, assuming that the full problem is explained to you on encountering either Upsilon or Omega, both are truhful, etc. Why can I not perform the appropriate calculations and make an expectation-maximising decision even after Upsilon-Omega has left? Surely Omega-Upsilon can predict that I’m going to do just that and act accordingly, right?
Yes, this is a standard incomplete information problem. Yes, you can do the calculations at any convenient time, not necessarily before meeting Omega. (These calculations can’t use the information that Omega exists, though.) No, it isn’t quite as simple as you state: when you meet Omega, you have to calculate the counterfactual probability of you having met Upsilon instead, and so on.
Something seems off about this, but I’m not sure what.
I’m pretty sure the logic is correct. I do make silly math mistakes sometimes, but I’ve tested this one on Vladimir Nesov and he agrees. No comment from Eliezer yet (this scenario was first posted to decision-theory-workshop).
It reminds me vaguely of Pascal’s Wager, but my cached responses thereunto are not translating informatively.
Then I think the original Newcomb’s Problem should remind you of Pascal’s Wager just as much, and my scenario should be analogous to the refutation thereof. (Thereunto? :-)