This is a global response to several replies within my little thread here, so I’ve put it at nearly the top level. Hopefully that works out OK.
I’m glad that FAWS brought up the probabilistic version. That’s because the greater the probability that Omega makes mistakes, the more inclined I am to take two boxes. I once read the claim that 70% of people, when told Newcomb’s Paradox in an experiment, claim to choose to take only one box. If this is accurate, then Omega can achieve a 70% level of accuracy by predicting that everybody is a one-boxer. Even if 70% is not accurate, you can still make the paradox work by adjusting the dollar amounts, as long as the bias is great enough that Omega can be confident that it will show up at all in the records of its past predictions. (To be fair, the proportion of two-boxers will probably rise as Omega’s accuracy falls, and changing the stakes should also affect people’s choices; there may not be a fixed point, although I expect that there is.)
If, in addition to the problem as stated (but with only 70% probability of success), I know that Omega always predicts one-boxing, then (hopefully) everybody agrees that I should take both boxes. There needs to some correlation between Omega’s predictions and the actual outcomes, not just a high proportion of past successes.
FAWS also writes:
You yourself claim to know what you would do in the boxing experiment
Actually, I don’t really want to make that claim. Although I’ve written things like ‘I would take both boxes’, I really should have written ‘I should take both boxes’. I’m stating a correct decision, not making a prediction about my actual actions. Right now, I predict about a 70% chance of two-boxing given the situation as stated in the original post, although I’ve never tried to calculate my estimates of probabilities, so who knows what that really means. (H’m, 70% again? Nope, I don’t trust that calibration at all!)
FAWS writes elsewhere:
Making a choice between two options […] just means that attributing the reason for your taking whatever option you take is most usefully attributed to you (and not e.g. gravity, government, the person holding a gun to you head etc.).
I don’t see what the gun has to do with it; this is a perfectly good problem in decision theory:
Suppose that you have a button that, if pressed, will trigger a bomb that kills two strangers on the other side of the world. I hold a gun to your head and threaten to shoot you if you don’t press the button. Should you press it?
A person who presses the button in that situation can reasonably say afterwards ‘I had no choice! Toby held a gun to my head!’, but that doesn’t invalidate the question. Such a person might even panic and make the question irrelevant, but it’s still a good question.
If it is a fact about you that you will leave the choice up to chance then Omega probably doesn’t offer you to take part in the first place.
So that’s how Omega gets such a good record! (^_^)
Understanding the question really is important. I’ve been interpreting it something along these lines: you interrupt your normal thought processes to go through a complete evaluation of the situation before you, then see what you do. (This is exactly what you cannot do if you panic in the gun problem above.) So perhaps we can predict with certain accuracy that an utter bigot will take one course of action, but that is not what the bigot should do, nor is it what they will do if they discard their prejudices and decide afresh.
Now that I think about it, I see some problems with this interpretation, and also some refinements that might fix it. (The first thing to do is to make it less dependent on the specific person making the decision.) But I’ll skip the refinements. It’s enough to notice that Omega might very well predict that a person will not take the time to think things through, so there is poor correlation between what one should do and what Omega will predict, even though the decision is based on what the world would be like if one did take the time.
I still think that (modulo refinements) this is a good interpretation of what most people would mean if they tell a story and then ask ‘What should this person do?’. (I can try to defend that claim if anybody still wants me to after they finish this comment.) In that case, I stand by my decision that one should take both boxes, at least if there is no good evidence of new physics.
However, I now realise that there is another interpretation, which is more practical, however much the ordinary person might not interpret things this way. That is: sit down and think through the whole situation now, long before you are ever faced with it in real life, and decide what to do. One obvious benefit of this is that when I hold a gun to your head, you won’t panic, because you will be prepared. More generally, this is what we are all actually doing right now! So as we make these idle philosophical musings, let’s be practical, and decide what we’ll do if Omega ever offers us this deal.
In this case, I agree that I will be better off (given the extremely unlikely but possible assumption that I am ever in this situation) if I have decided now to take only Box B. As RobinZ points out, I might change my mind later, but that can’t be helped (and to a certain extent shouldn’t be helped, since it’s best if I take two boxes after Omega predicts that I’ll only take one, but we can’t judge that extent if Omega is smarter than us, so really there’s no benefit to holding back at all).
If Omega is fallible, then the value of one-boxing falls drastically, and even adjusting the amount of money doesn’t help in the end; once Omega’s proportion of past success matches the observed proportion in experiments (or whatever our best guess of the actual proportion of real people is), then I’m back to two-boxing, since I expect that Omega simply always predicts one-boxing.
In hindsight, it’s obvious that the the original post was about decision in this sense, since Eliezer was talking about an AI that modifies its decision procedures in anticipation of facing Omega in the future. Similarly, we humans modify our decision procedures by making commitments and letting ourselves invent rationalisations for them afterwards (although the problem with this is that it makes it hard to change our minds when we receive new information). So obviously Eliezer wants us to decide now (or at least well ahead of time) and use our leet Methods of Rationality to keep the rationalisations in check.
So I hereby decide that I will pick only one box. (You hear that, Omega!?) Since I am honest (and strongly doubt that Omega exists), I’ll add that I may very well change my mind if this ever really happens, but that’s about what I would do, not what I should do. And in a certain sense, I should change my mind … then. But in another sense, I should (and do!) choose to be a one-boxer now.
(Thanks also to CarlShulman, whom I haven’t quoted, but whose comment was a big help in drawing my attention to the different senses of ‘should’, even though I didn’t really adopt his analysis of them.)
If Omega is fallible, then the value of one-boxing falls drastically, and even adjusting the amount of money doesn’t help in the end;
Assume Omega has a probability X of correctly predicting your decision:
If you choose to two-box:
X chance of getting $1000
(1-X) chance of getting $1,001,000
If you choose to take box B only:
X chance of getting $1,000,000
(1-X) chance of getting $0
Your expected utilities for two-boxing and one-boxing are (respectively):
E2 = 1000X + (1-X)1001000 E1 = 1000000X
For E2 > E1, we must have 1000X + 1,001,000 − 1,001,000X − 1,000,000X > 0, or 1,001,000 > 2,000,000X, or
X < 0.5005
So as long as Omega can maintain a greater than 50% accuracy, you should expect to earn more money by one-boxing. Since the solution seems so simple, and since I’m a total novice at decision theory, it’s possible I’m missing something here, so please let me know.
So as long as Omega can maintain a greater than 50% accuracy, you should expect to earn more money by one-boxing. Since the solution seems so simple, and since I’m a total novice at decision theory, it’s possible I’m missing something here, so please let me know.
Your caclulation is fine. What you’re missing is that Omega has a record of 70% accuracy because Omega always predicts that a person will one-box and 70% of people are one-boxers. So Omega always puts the million dollars in Box B, and I will always get $1,001,000$ if I’m one of the 30% of people who two-box.
At least, that is a possibility, which your calculation doesn’t take into account. I need evidence of a correlation between Omega’s predictions and the participants’ actual behaviour, not just evidence of correct predictions. My prior probability distribution for how often people one-box isn’t even concentrated very tightly around 70% (which is just a number that I remember reading once as the result of one survey), so anything short of a long run of predictions with very high proportion of correct ones will make me suspect that Omega is pulling a trick like this.
So the problem is much cleaner as Eliezer states it, with a perfect record. (But if even that record is short, I won’t buy it.)
Oops, I see that RobinZ already replied, and with calculations. This shows that I should still remove the word ‘drastically’ from the bit that nhamann quoted.
Wait—we can’t assume that the probability of being correct is the same for two-boxing and one-boxing. Suppose Omega has a probability X of predicting one when you choose one and Y of predicting one when you choose two.
E1 = E($1 000 000) * X
E2 = E($1 000) + E($1 000 000) * Y
The special case you list corresponds to Y = 1 - X, but in the general case, we can derive that E1 > E2 implies
X > Y + E($1 000) / E($1 000 000)
If we assume linear utility in wealth, this corresponds to a difference of 0.001. If, alternately, we choose a median net wealth of $93 100 (the U.S. figure) and use log-wealth as the measure of utility, the required difference increases to 0.004 or so. Either way, unless you’re dead broke (e.g. net wealth $1), you had better be extremely confident that you can fool the interrogator before you two-box.
Thanks for the replies, everybody!
This is a global response to several replies within my little thread here, so I’ve put it at nearly the top level. Hopefully that works out OK.
I’m glad that FAWS brought up the probabilistic version. That’s because the greater the probability that Omega makes mistakes, the more inclined I am to take two boxes. I once read the claim that 70% of people, when told Newcomb’s Paradox in an experiment, claim to choose to take only one box. If this is accurate, then Omega can achieve a 70% level of accuracy by predicting that everybody is a one-boxer. Even if 70% is not accurate, you can still make the paradox work by adjusting the dollar amounts, as long as the bias is great enough that Omega can be confident that it will show up at all in the records of its past predictions. (To be fair, the proportion of two-boxers will probably rise as Omega’s accuracy falls, and changing the stakes should also affect people’s choices; there may not be a fixed point, although I expect that there is.)
If, in addition to the problem as stated (but with only 70% probability of success), I know that Omega always predicts one-boxing, then (hopefully) everybody agrees that I should take both boxes. There needs to some correlation between Omega’s predictions and the actual outcomes, not just a high proportion of past successes.
FAWS also writes:
Actually, I don’t really want to make that claim. Although I’ve written things like ‘I would take both boxes’, I really should have written ‘I should take both boxes’. I’m stating a correct decision, not making a prediction about my actual actions. Right now, I predict about a 70% chance of two-boxing given the situation as stated in the original post, although I’ve never tried to calculate my estimates of probabilities, so who knows what that really means. (H’m, 70% again? Nope, I don’t trust that calibration at all!)
FAWS writes elsewhere:
I don’t see what the gun has to do with it; this is a perfectly good problem in decision theory:
Suppose that you have a button that, if pressed, will trigger a bomb that kills two strangers on the other side of the world. I hold a gun to your head and threaten to shoot you if you don’t press the button. Should you press it?
A person who presses the button in that situation can reasonably say afterwards ‘I had no choice! Toby held a gun to my head!’, but that doesn’t invalidate the question. Such a person might even panic and make the question irrelevant, but it’s still a good question.
So that’s how Omega gets such a good record! (^_^)
Understanding the question really is important. I’ve been interpreting it something along these lines: you interrupt your normal thought processes to go through a complete evaluation of the situation before you, then see what you do. (This is exactly what you cannot do if you panic in the gun problem above.) So perhaps we can predict with certain accuracy that an utter bigot will take one course of action, but that is not what the bigot should do, nor is it what they will do if they discard their prejudices and decide afresh.
Now that I think about it, I see some problems with this interpretation, and also some refinements that might fix it. (The first thing to do is to make it less dependent on the specific person making the decision.) But I’ll skip the refinements. It’s enough to notice that Omega might very well predict that a person will not take the time to think things through, so there is poor correlation between what one should do and what Omega will predict, even though the decision is based on what the world would be like if one did take the time.
I still think that (modulo refinements) this is a good interpretation of what most people would mean if they tell a story and then ask ‘What should this person do?’. (I can try to defend that claim if anybody still wants me to after they finish this comment.) In that case, I stand by my decision that one should take both boxes, at least if there is no good evidence of new physics.
However, I now realise that there is another interpretation, which is more practical, however much the ordinary person might not interpret things this way. That is: sit down and think through the whole situation now, long before you are ever faced with it in real life, and decide what to do. One obvious benefit of this is that when I hold a gun to your head, you won’t panic, because you will be prepared. More generally, this is what we are all actually doing right now! So as we make these idle philosophical musings, let’s be practical, and decide what we’ll do if Omega ever offers us this deal.
In this case, I agree that I will be better off (given the extremely unlikely but possible assumption that I am ever in this situation) if I have decided now to take only Box B. As RobinZ points out, I might change my mind later, but that can’t be helped (and to a certain extent shouldn’t be helped, since it’s best if I take two boxes after Omega predicts that I’ll only take one, but we can’t judge that extent if Omega is smarter than us, so really there’s no benefit to holding back at all).
If Omega is fallible, then the value of one-boxing falls drastically, and even adjusting the amount of money doesn’t help in the end; once Omega’s proportion of past success matches the observed proportion in experiments (or whatever our best guess of the actual proportion of real people is), then I’m back to two-boxing, since I expect that Omega simply always predicts one-boxing.
In hindsight, it’s obvious that the the original post was about decision in this sense, since Eliezer was talking about an AI that modifies its decision procedures in anticipation of facing Omega in the future. Similarly, we humans modify our decision procedures by making commitments and letting ourselves invent rationalisations for them afterwards (although the problem with this is that it makes it hard to change our minds when we receive new information). So obviously Eliezer wants us to decide now (or at least well ahead of time) and use our leet Methods of Rationality to keep the rationalisations in check.
So I hereby decide that I will pick only one box. (You hear that, Omega!?) Since I am honest (and strongly doubt that Omega exists), I’ll add that I may very well change my mind if this ever really happens, but that’s about what I would do, not what I should do. And in a certain sense, I should change my mind … then. But in another sense, I should (and do!) choose to be a one-boxer now.
(Thanks also to CarlShulman, whom I haven’t quoted, but whose comment was a big help in drawing my attention to the different senses of ‘should’, even though I didn’t really adopt his analysis of them.)
Assume Omega has a probability X of correctly predicting your decision:
If you choose to two-box:
X chance of getting $1000
(1-X) chance of getting $1,001,000
If you choose to take box B only:
X chance of getting $1,000,000
(1-X) chance of getting $0
Your expected utilities for two-boxing and one-boxing are (respectively):
E2 = 1000X + (1-X)1001000
E1 = 1000000X
For E2 > E1, we must have 1000X + 1,001,000 − 1,001,000X − 1,000,000X > 0, or 1,001,000 > 2,000,000X, or
X < 0.5005
So as long as Omega can maintain a greater than 50% accuracy, you should expect to earn more money by one-boxing. Since the solution seems so simple, and since I’m a total novice at decision theory, it’s possible I’m missing something here, so please let me know.
Your caclulation is fine. What you’re missing is that Omega has a record of 70% accuracy because Omega always predicts that a person will one-box and 70% of people are one-boxers. So Omega always puts the million dollars in Box B, and I will always get $1,001,000$ if I’m one of the 30% of people who two-box.
At least, that is a possibility, which your calculation doesn’t take into account. I need evidence of a correlation between Omega’s predictions and the participants’ actual behaviour, not just evidence of correct predictions. My prior probability distribution for how often people one-box isn’t even concentrated very tightly around 70% (which is just a number that I remember reading once as the result of one survey), so anything short of a long run of predictions with very high proportion of correct ones will make me suspect that Omega is pulling a trick like this.
So the problem is much cleaner as Eliezer states it, with a perfect record. (But if even that record is short, I won’t buy it.)
Oops, I see that RobinZ already replied, and with calculations. This shows that I should still remove the word ‘drastically’ from the bit that nhamann quoted.
Wait—we can’t assume that the probability of being correct is the same for two-boxing and one-boxing. Suppose Omega has a probability X of predicting one when you choose one and Y of predicting one when you choose two.
The special case you list corresponds to Y = 1 - X, but in the general case, we can derive that E1 > E2 implies
If we assume linear utility in wealth, this corresponds to a difference of 0.001. If, alternately, we choose a median net wealth of $93 100 (the U.S. figure) and use log-wealth as the measure of utility, the required difference increases to 0.004 or so. Either way, unless you’re dead broke (e.g. net wealth $1), you had better be extremely confident that you can fool the interrogator before you two-box.