I already understood what you meant by “rationalists should win”, Eliezer, but I don’t find Newcomb’s problem very convincing as an example. The way I see it, if you one-box you’ve lost. You could have gotten an extra $1000 but you chose not to.
And yet those who one-box get $999000 more than those who don’t. What gives? If there is a systematic, predictable thing that offers one-boxers $1000000 and offers two-boxers $1000, and there is not a systematic, predictable thing that provides some sort of countering offer to two-boxers, by one-boxing you still get more money.
I can’t think of something of equal power level (examining your decisions, not your method of arriving at those decisions) which would be able to provide the countering offer to two-boxers.
Simple: most situations in real life aren’t like this. If you believe Omega and one-box, you’ll lose when he’s lying. If your decision theory works better in hypothetical situations and worse in real life, then it doesn’t make you win.
Also, I don’t think Eliezer keeps harping on Newcomb’s problem because he anticipates experiencing precisely that scenario. I see several important points that I don’t think have been clearly made (not that I’m the one to do so):
We can choose whether and when to implement certain decision algorithms, including classical causal decision theory (CCDT). This choice may in fact be trivial, or it may be subtle, but it is a worthy question for a rationalist.
Although, for any fixed set of options, implementing CCDT maximizes your return, there are in fact cases where the options you have depend on the outcome of a model of your decision algorithm. I’m not talking about Omega, I’m talking about human social life. We base a large portion of our interactions with others on our anticipations of how they might respond. (This isn’t often done rationally by anyone’s standards, but it can be.)
It gets confusing (in particular, Hofstadterian) here, but a plausibly better outcome might be reached in the Prisoner’s Dilemma by selfish non-strangers mutually modeling the other’s likely decision process, and recognizing that only C-C and D-D are stable outcomes under mutual modeling.
Of course, I still feel a bit uncomfortable with this line of reasoning.
Of course the one-boxers get more money: They were put in a situation in which they could either get $1 000 000 or $1 001 000, whereas the two-boxers were put in a situation in which they could get $0 or $1000.
It makes no sense to compare the two decisions the way you and Eliezer do. It’s like organizing a swimming competition between an Olympic athlete who has to swim ten kilometers to win and an untrained fatass who only has to swim a hundred meters to win, and concluding that because the fatass wins more often than the athlete, therefore fatasses clearly make better swimmers than athletes.
Of course the one-boxers get more money: They were put in a situation in which they could either get $1 000 000 or $1 001 000, whereas the two-boxers were put in a situation in which they could get $0 or $1000.
When faced with this decision, you are either in the real world, in which case you can get an extra $1000 by two boxing, or you are in a simulation, in which case you can arrange so your self in the real world gets and extra $1,000,000 by one boxing. Given that you can’t tell which of these is the case, and that you are deterministic, you will make the same decision in both situations. So your choice is to either one box and gain $1,000,000 or two box and gain $1000. If you like having more money, it seems clear which of those choices is more rational.
But if you were put into said hypothetical competition, and could somehow decide just before the contest began whether to be an Olympic athlete or an untrained fatass, which would you choose?
I think you’re getting overly distracted by the details of the problem construction and missing the point.
If my only goal were to win that particular competition (and not to be a good swimmer), of course I’d choose to turn into a fatass and lose all my training. Likewise, if I could choose to precommit to one-boxing in Newcomb-like problems, I would, because pre-commitment has an effect on what will be in box B (whereas the actual decision does not).
The details are what makes Newcomb’s problem what it is, so I don’t see how it’s possible to get “overly distracted” by them. Correct me if I’m wrong, but pre-commitment isn’t an option in Newcomb’s problem, so the best, the most rational, the winning decision is two-boxing.
Correct me if I’m wrong, but pre-commitment isn’t an option in Newcomb’s problem, so the best, the most rational, the winning decision is two-boxing.
By construction, Omega’s predictions are known to be essentially infallible. Given that, whatever you choose, you can safely assume Omega will have correctly predicted that choice. To what extent, then, is pre-commitment distinguishable from deciding on the spot?
In a sense there is an implicit pre-commitment in the structure of the problem; while you have not pre-committed to a choice on this specific problem, you are essentially pre-committed to a decision-making algorithm.
Eliezer’s argument, if I understand it, is that any decision-making algorithm that results in two-boxing is by definition irrational due to giving a predictably bad outcome.
In a sense there is an implicit pre-commitment in the structure of the problem; while you have not pre-committed to a choice on this specific problem, you are essentially pre-committed to a decision-making algorithm.
That’s an interesting, and possibly fruitful, way of looking at the problem.
Pre-commitment is different from deciding on the spot because once you’re on the spot, there is nothing, absolutely nothing you can do to change what’s in box B. It’s over. It’s a done deal. It’s beyond your control.
My understanding of Eliezer’s argument is the same as yours. My objection is that two-boxing doesn’t actually give a bad outcome. It gives the best outcome you can get given the situation you’re in. That you don’t know what situation you’re in until after you’ve opened box B doesn’t change that fact. As Eliezer is so fond of saying, the map isn’t the territory.
Pre-commitment is different from deciding on the spot because once you’re on the spot, there is nothing, absolutely nothing you can do to change what’s in box B.
If your decision on the spot is 100 percent predictable ahead of time, as is explicitly assumed in the problem construction, you are effectively pre-committed as far as Omega is concerned. You, apparently, have essentially pre-committed to opening two boxes.
My objection is that two-boxing doesn’t actually give a bad outcome. It gives the best outcome you can get given the situation you’re in.
And yet, everyone who opens one box does better than the people who open two boxes.
You seem to have a very peculiar definition of “best outcome”.
If your decision on the spot is 100 percent predictable ahead of time, as is explicitly assumed in the problem construction, you are effectively pre-committed as far as Omega is concerned. You, apparently, have essentially pre-committed to opening two boxes.
What I meant by ‘pre-commitment’ is a decision that we can make if and only if we know about Newcomb-like problems before being faced with one. In other words, it’s a decision that can affect what Omega will put in box B. That Omega can deduce what my decision will be doesn’t mean that the decision is already taken.
And yet, everyone who opens one box does better than the people who open two boxes.
And every fatass who competes against an Olympic athlete in the scenario I described above does ‘better’ than the athlete. So what? Unless the athlete knows about the competition’s rules ahead of time and eats non-stop to turn himself into a fatass, there’s not a damn thing he can do about it, except try his best once the competition starts.
You seem to have a very peculiar definition of “best outcome”.
It seems too obvious to say, but I guess I have to say it. “The best outcome” in this context is “the best outcome that it is possible to achieve by making a decision”.
If box B contains nothing, then the best outcome that it is possible to achieve by making a decision is to win a thousand dollars. If box B contains a million dollars, then the best outcome that it is possible to achieve by making a decision is to win one million and one thousand dollars.
Well, I don’t see how I can explain myself more clearly than this, so this will be my last comment on this subject. In this thread. This week. ;)
This exchange has finally imparted a better understanding of this problem for me.
If you pre-commit now to always one-box – and you believe that about yourself – then deciding to one-box when Omega asks you is the best decision.
If you are uncertain of your commitment then you probably haven’t really pre-committed! I haven’t tried to math it, but I think your actual decision when Omega arrives would depend on the strength of your belief about your own pre-commitment. [Though a more-inconvenient possible world is the one in which you’ve never heard of this, or similar, puzzles!]
Now I grok why rationality should be self-consistent under reflection.
Small nitpick: If you’ve really pre-committed to one-boxing, there is no decision to be made once Omega has set up the boxes. In fact, the thought of making a decision won’t even cross your mind. If it does cross your mind, you should two-box. But if you two-box, you now know that you haven’t really pre-committed to one-boxing. Actually, even if you decide to (mistakenly) one-box, you’ll still know you haven’t really pre-committed, or you wouldn’t have had to decide anything on the spot.
In other words, Newcomb’s problem can only ever involve a single true decision. If you’re capable of pre-commitment (that is, if you know about Newcomb-like problems in advance and if you have the means to really pre-commit), it’s the decision to pre-commit, which precludes any ulterior decision. If you aren’t capable of pre-commitment (that is, if at least one of the above conditions is false), it’s the on-the-spot decision.
Eliezer’s argument, if I understand it, is that any decision-making algorithm that results in two-boxing is by definition irrational due to giving a predictably bad outcome.
So he’s assuming the conclusion that you get a bad outcome? Golly.
The result of two-boxing is a thousand dollars. The result of one-boxing is a million dollars. By definition, a mind that always one-boxes receives a better payout than one that always two-boxes, and therefore one-boxing is more rational, by definition.
The result of two-boxing is a thousand dollars more than you would have gotten otherwise. The result of one-boxing is a thousand dollars less than you would have gotten otherwise. Therefore two-boxing is more rational, by definition.
What determines whether you’ll be in a 1M/1M+1K situation or in a 0/1K situation is the kind of mind you have, but in Newcomb’s problem you’re not given the opportunity to affect what kind of mind you have (by pre-commiting to one-boxing, for example), you can only decide whether to get X or X+1K, regardless of X’s value.
Suppose for a moment that one-boxing is the Foo thing to do. Two-boxing is the expected-utility-maximizing thing to do. Omega decided to try to reward those minds which it predicts will choose to do the Foo thing with a decision between doing the Foo thing and gaining $1000000, and doing the unFoo thing and gaining $1001000, while giving those minds which will choose to do the unFoo thing a decision between doing the Foo thing and gaining $0 and doing the unFoo thing and gaining $1000.
The relevant question is whether there is a generalization of the computation Foo which we can implement that doesn’t screw us over on all sorts of non-Newcomb problems. Drescher for instance claims that acting ethically implies, among other things, doing the Foo thing, even when it is obviously not the expected-utility-maximizing thing.
You’re assuming that you can just choose how you go about making decisions every time you make a decision. If you’re not granted that assumption, Furcas’s analysis is spot on. Two-boxers succeed in other places and also on Newcomb; one-boxers fail in many situations that are similar to Newcomb but not as nice. So you need to decide what sort of decisions you’ll make in general, and that will (arguably) determine how much money is in the boxes in this particular experiment.
I already understood what you meant by “rationalists should win”, Eliezer, but I don’t find Newcomb’s problem very convincing as an example. The way I see it, if you one-box you’ve lost. You could have gotten an extra $1000 but you chose not to.
And yet those who one-box get $999000 more than those who don’t. What gives? If there is a systematic, predictable thing that offers one-boxers $1000000 and offers two-boxers $1000, and there is not a systematic, predictable thing that provides some sort of countering offer to two-boxers, by one-boxing you still get more money.
I can’t think of something of equal power level (examining your decisions, not your method of arriving at those decisions) which would be able to provide the countering offer to two-boxers.
Simple: most situations in real life aren’t like this. If you believe Omega and one-box, you’ll lose when he’s lying. If your decision theory works better in hypothetical situations and worse in real life, then it doesn’t make you win.
Apply the Least Convenient Possible World principle.
Also, I don’t think Eliezer keeps harping on Newcomb’s problem because he anticipates experiencing precisely that scenario. I see several important points that I don’t think have been clearly made (not that I’m the one to do so):
We can choose whether and when to implement certain decision algorithms, including classical causal decision theory (CCDT). This choice may in fact be trivial, or it may be subtle, but it is a worthy question for a rationalist.
Although, for any fixed set of options, implementing CCDT maximizes your return, there are in fact cases where the options you have depend on the outcome of a model of your decision algorithm. I’m not talking about Omega, I’m talking about human social life. We base a large portion of our interactions with others on our anticipations of how they might respond. (This isn’t often done rationally by anyone’s standards, but it can be.)
It gets confusing (in particular, Hofstadterian) here, but a plausibly better outcome might be reached in the Prisoner’s Dilemma by selfish non-strangers mutually modeling the other’s likely decision process, and recognizing that only C-C and D-D are stable outcomes under mutual modeling.
Of course, I still feel a bit uncomfortable with this line of reasoning.
Of course the one-boxers get more money: They were put in a situation in which they could either get $1 000 000 or $1 001 000, whereas the two-boxers were put in a situation in which they could get $0 or $1000.
It makes no sense to compare the two decisions the way you and Eliezer do. It’s like organizing a swimming competition between an Olympic athlete who has to swim ten kilometers to win and an untrained fatass who only has to swim a hundred meters to win, and concluding that because the fatass wins more often than the athlete, therefore fatasses clearly make better swimmers than athletes.
When faced with this decision, you are either in the real world, in which case you can get an extra $1000 by two boxing, or you are in a simulation, in which case you can arrange so your self in the real world gets and extra $1,000,000 by one boxing. Given that you can’t tell which of these is the case, and that you are deterministic, you will make the same decision in both situations. So your choice is to either one box and gain $1,000,000 or two box and gain $1000. If you like having more money, it seems clear which of those choices is more rational.
But if you were put into said hypothetical competition, and could somehow decide just before the contest began whether to be an Olympic athlete or an untrained fatass, which would you choose?
I think you’re getting overly distracted by the details of the problem construction and missing the point.
If my only goal were to win that particular competition (and not to be a good swimmer), of course I’d choose to turn into a fatass and lose all my training. Likewise, if I could choose to precommit to one-boxing in Newcomb-like problems, I would, because pre-commitment has an effect on what will be in box B (whereas the actual decision does not).
The details are what makes Newcomb’s problem what it is, so I don’t see how it’s possible to get “overly distracted” by them. Correct me if I’m wrong, but pre-commitment isn’t an option in Newcomb’s problem, so the best, the most rational, the winning decision is two-boxing.
By construction, Omega’s predictions are known to be essentially infallible. Given that, whatever you choose, you can safely assume Omega will have correctly predicted that choice. To what extent, then, is pre-commitment distinguishable from deciding on the spot?
In a sense there is an implicit pre-commitment in the structure of the problem; while you have not pre-committed to a choice on this specific problem, you are essentially pre-committed to a decision-making algorithm.
Eliezer’s argument, if I understand it, is that any decision-making algorithm that results in two-boxing is by definition irrational due to giving a predictably bad outcome.
That’s an interesting, and possibly fruitful, way of looking at the problem.
Pre-commitment is different from deciding on the spot because once you’re on the spot, there is nothing, absolutely nothing you can do to change what’s in box B. It’s over. It’s a done deal. It’s beyond your control.
My understanding of Eliezer’s argument is the same as yours. My objection is that two-boxing doesn’t actually give a bad outcome. It gives the best outcome you can get given the situation you’re in. That you don’t know what situation you’re in until after you’ve opened box B doesn’t change that fact. As Eliezer is so fond of saying, the map isn’t the territory.
If your decision on the spot is 100 percent predictable ahead of time, as is explicitly assumed in the problem construction, you are effectively pre-committed as far as Omega is concerned. You, apparently, have essentially pre-committed to opening two boxes.
And yet, everyone who opens one box does better than the people who open two boxes.
You seem to have a very peculiar definition of “best outcome”.
What I meant by ‘pre-commitment’ is a decision that we can make if and only if we know about Newcomb-like problems before being faced with one. In other words, it’s a decision that can affect what Omega will put in box B. That Omega can deduce what my decision will be doesn’t mean that the decision is already taken.
And every fatass who competes against an Olympic athlete in the scenario I described above does ‘better’ than the athlete. So what? Unless the athlete knows about the competition’s rules ahead of time and eats non-stop to turn himself into a fatass, there’s not a damn thing he can do about it, except try his best once the competition starts.
It seems too obvious to say, but I guess I have to say it. “The best outcome” in this context is “the best outcome that it is possible to achieve by making a decision”. If box B contains nothing, then the best outcome that it is possible to achieve by making a decision is to win a thousand dollars. If box B contains a million dollars, then the best outcome that it is possible to achieve by making a decision is to win one million and one thousand dollars.
Well, I don’t see how I can explain myself more clearly than this, so this will be my last comment on this subject. In this thread. This week. ;)
This exchange has finally imparted a better understanding of this problem for me.
If you pre-commit now to always one-box – and you believe that about yourself – then deciding to one-box when Omega asks you is the best decision.
If you are uncertain of your commitment then you probably haven’t really pre-committed! I haven’t tried to math it, but I think your actual decision when Omega arrives would depend on the strength of your belief about your own pre-commitment. [Though a more-inconvenient possible world is the one in which you’ve never heard of this, or similar, puzzles!]
Now I grok why rationality should be self-consistent under reflection.
Small nitpick: If you’ve really pre-committed to one-boxing, there is no decision to be made once Omega has set up the boxes. In fact, the thought of making a decision won’t even cross your mind. If it does cross your mind, you should two-box. But if you two-box, you now know that you haven’t really pre-committed to one-boxing. Actually, even if you decide to (mistakenly) one-box, you’ll still know you haven’t really pre-committed, or you wouldn’t have had to decide anything on the spot.
In other words, Newcomb’s problem can only ever involve a single true decision. If you’re capable of pre-commitment (that is, if you know about Newcomb-like problems in advance and if you have the means to really pre-commit), it’s the decision to pre-commit, which precludes any ulterior decision. If you aren’t capable of pre-commitment (that is, if at least one of the above conditions is false), it’s the on-the-spot decision.
So he’s assuming the conclusion that you get a bad outcome? Golly.
True, we don’t know the outcome. But we should still predict that it will be bad, due to Omega’s 99% accuracy rate.
Don’t mess with Omega.
The result of two-boxing is a thousand dollars. The result of one-boxing is a million dollars. By definition, a mind that always one-boxes receives a better payout than one that always two-boxes, and therefore one-boxing is more rational, by definition.
See Arguing “By Definition”. It’s particularly problematic when the definition of “rational” is precisely what’s in dispute.
The result of two-boxing is a thousand dollars more than you would have gotten otherwise. The result of one-boxing is a thousand dollars less than you would have gotten otherwise. Therefore two-boxing is more rational, by definition.
What determines whether you’ll be in a 1M/1M+1K situation or in a 0/1K situation is the kind of mind you have, but in Newcomb’s problem you’re not given the opportunity to affect what kind of mind you have (by pre-commiting to one-boxing, for example), you can only decide whether to get X or X+1K, regardless of X’s value.
Suppose for a moment that one-boxing is the Foo thing to do. Two-boxing is the expected-utility-maximizing thing to do. Omega decided to try to reward those minds which it predicts will choose to do the Foo thing with a decision between doing the Foo thing and gaining $1000000, and doing the unFoo thing and gaining $1001000, while giving those minds which will choose to do the unFoo thing a decision between doing the Foo thing and gaining $0 and doing the unFoo thing and gaining $1000.
The relevant question is whether there is a generalization of the computation Foo which we can implement that doesn’t screw us over on all sorts of non-Newcomb problems. Drescher for instance claims that acting ethically implies, among other things, doing the Foo thing, even when it is obviously not the expected-utility-maximizing thing.
You’re assuming that you can just choose how you go about making decisions every time you make a decision. If you’re not granted that assumption, Furcas’s analysis is spot on. Two-boxers succeed in other places and also on Newcomb; one-boxers fail in many situations that are similar to Newcomb but not as nice. So you need to decide what sort of decisions you’ll make in general, and that will (arguably) determine how much money is in the boxes in this particular experiment.
Such as?
(Is this meant to refer to failures of evidential decision theory? There are other options.)
This premise is not accepted by the 1-box contingent. Occasionally they claim there’s a reason.
Can you please elaborate? I’m trying to catch up!
You mean they don’t accept that the decision doesn’t affect what’s in box B?