In order to make inferences about the other players decision, you have to have a probability distribution for them, and then do a complicated integral.
The point is that there is no strictly optimal strategy, which means that your “generally optimal” strategy can do really abysmally against the wrong probability distribution. Call that probability distribution “Newcomb’s problem” and the point that I see is that you might have an “optimal decision theory” which fails to be “rational.”
If you know all of the rules, as in Newcomb’s problem, then you can know how to react optimally. If you fail to incorporate your knowledge of Omega’s capabilities, you’re not acting optimally, and you can do better.
But you don’t have absolute knowledge of Omega, you have a probability estimate of whether ve is, say, omniscient versus stalking you on the internet, or on the other hand if ve even has a million dollars to put in the other box.
The sort of newcomb- (or kavka-) like problem you might expect to run into on the street hinges almost entirely on the probability that there’s a million dollars in the box. So if you’re trying to create a decision theory that gives you optimal action assuming the probability distributions of real life I don’t see how optimizing for a particular, uncommon, problem (where other problems are more common and might be decided differently!) will help out with being rational on hostile hardware.
On this point, I’ve going to have to agree with what EY said here (which I repeated here).
In short: Omega’s strategy and its consequences for you are not, in any sense, atypical. Omega is treating you based upon what you would do, given full (or approximate) knowledge of the situation. This is quite normal: people do in fact treat you differently based upon estimation of “what you would do”, which is also known as your “character”.
Your point would be valid if Omega were basing the reward profile on your genetics, or how you got to your decision, or some other strange factor. But here, Omega is someone who just bases its treatment of you on things that are normal to care about in normal problems.
You’re just emphasizing the fact that you have full knowledge of the situation.
I currently believe, that if I ever am in a position where I believe myself to be confronted with Newcomb’s problem, no matter how convinced I am at that time, it will be a hoax in some way; for example, Omega has limited prediction capability or there isn’t actually $1 million in the box.
I’m not saying “you should two-box because the money is already in there” I’m saying “maybe you should JUST take the $1000 box because you’ve seen that money and if you don’t think ve’s lying you’re probably hallucinating.”
True: you will probably never be in the epistemic state in which you will justifiably believe you are in Newcomb’s problem. Nevertheless, you will frequently be in probabilistic variants of the problem, and a sane decision theory that wins on those cases will have the implication that it should one-box when you take the limit of all variables as they go to what they need to be to make it the literal Newcomb’s problem.
I got wrapped up in writing this comment and forgot about the larger context; my point is that it may be necessary (in the least convenient possible world) to choose a decision theory that does poorly on Newcomb’s problem but better elsewhere, given that Newcomb’s problem is unlikely to occur and similar-seeming but more common problems give better results with a different strategy.
So like the original post, I ask why Newcomb’s problem seems to be (have been?) driving discussions of decision theory? Is it because this is the easiest place to make improvements, or because it’s fun to think about?
In order to make inferences about the other players decision, you have to have a probability distribution for them, and then do a complicated integral.
The point is that there is no strictly optimal strategy, which means that your “generally optimal” strategy can do really abysmally against the wrong probability distribution. Call that probability distribution “Newcomb’s problem” and the point that I see is that you might have an “optimal decision theory” which fails to be “rational.”
If you know all of the rules, as in Newcomb’s problem, then you can know how to react optimally. If you fail to incorporate your knowledge of Omega’s capabilities, you’re not acting optimally, and you can do better.
But you don’t have absolute knowledge of Omega, you have a probability estimate of whether ve is, say, omniscient versus stalking you on the internet, or on the other hand if ve even has a million dollars to put in the other box.
The sort of newcomb- (or kavka-) like problem you might expect to run into on the street hinges almost entirely on the probability that there’s a million dollars in the box. So if you’re trying to create a decision theory that gives you optimal action assuming the probability distributions of real life I don’t see how optimizing for a particular, uncommon, problem (where other problems are more common and might be decided differently!) will help out with being rational on hostile hardware.
If we spend TOO MUCH time preparing for the least convenient possible world we may miss out on the real world.
On this point, I’ve going to have to agree with what EY said here (which I repeated here).
In short: Omega’s strategy and its consequences for you are not, in any sense, atypical. Omega is treating you based upon what you would do, given full (or approximate) knowledge of the situation. This is quite normal: people do in fact treat you differently based upon estimation of “what you would do”, which is also known as your “character”.
Your point would be valid if Omega were basing the reward profile on your genetics, or how you got to your decision, or some other strange factor. But here, Omega is someone who just bases its treatment of you on things that are normal to care about in normal problems.
You’re just emphasizing the fact that you have full knowledge of the situation.
I currently believe, that if I ever am in a position where I believe myself to be confronted with Newcomb’s problem, no matter how convinced I am at that time, it will be a hoax in some way; for example, Omega has limited prediction capability or there isn’t actually $1 million in the box.
I’m not saying “you should two-box because the money is already in there” I’m saying “maybe you should JUST take the $1000 box because you’ve seen that money and if you don’t think ve’s lying you’re probably hallucinating.”
True: you will probably never be in the epistemic state in which you will justifiably believe you are in Newcomb’s problem. Nevertheless, you will frequently be in probabilistic variants of the problem, and a sane decision theory that wins on those cases will have the implication that it should one-box when you take the limit of all variables as they go to what they need to be to make it the literal Newcomb’s problem.
I got wrapped up in writing this comment and forgot about the larger context; my point is that it may be necessary (in the least convenient possible world) to choose a decision theory that does poorly on Newcomb’s problem but better elsewhere, given that Newcomb’s problem is unlikely to occur and similar-seeming but more common problems give better results with a different strategy.
So like the original post, I ask why Newcomb’s problem seems to be (have been?) driving discussions of decision theory? Is it because this is the easiest place to make improvements, or because it’s fun to think about?