Wow, it must be frustrating arguing with people that say $1,000 is the rational choice over $1,000,000. I did not expect such a failure mode to exist, what is even the point of decision theory if it doesn’t help us make good choices.
On another note, this article introduces Newcomb’s problem but doesn’t explain why we should care about it. The Prisoner’s Dilemma has obvious applications for real life, does Newcomb’s problem comes up often too or is it mostly a theoretical issue for now?
And of course I can’t resist providing my own take on Newcomb’s:
What matters is not which box I choose, but which box Omega predicts I will choose. So I can restate the problem like so:
“Can I deceive a super-intelligence into predicting I will One Box?”
Payoff matrix:
Can deceive? \ Choice
One Box
Two Box
No
$1,000,000
$ 1,000
Yes
$1,000,000
$1,001,000
So to maximize the payoff:
No ⇒ One Box
Yes ⇒ Two Box
On priors humans should go with No and thus One Box. Taking into account the track record of the super-intelligence mentioned in the article (100 correct predictions out of 100) only reinforces that No.
Unfortunately this doesn’t pass the Ideological Turing Test. Like, your argument would work on people that already agree with your frame, and wouldn’t on people with different frame who two box. It’s really kind of hard to step in into that mindset, and I agree that it’s ridiculously wrong, but you are not targeting it.
If I was a two boxer I would say something like:
In each situation that Omega could have put me in, two boxing gives me more money than one boxing. Omega is in my past, and my actions can’t control it. Therefore I should two box. Your payoff matrix is irrelevant.
Are you sure you’ve chosen the right payoff matrix? All you know about Omega is that it is a superintelligence from another galaxy that has a very good track record of correct predictions. You don’t actually know whether your ability (or not) to deceive Omega is relevant.
Wow, it must be frustrating arguing with people that say $1,000 is the rational choice over $1,000,000. I did not expect such a failure mode to exist, what is even the point of decision theory if it doesn’t help us make good choices.
On another note, this article introduces Newcomb’s problem but doesn’t explain why we should care about it. The Prisoner’s Dilemma has obvious applications for real life, does Newcomb’s problem comes up often too or is it mostly a theoretical issue for now?
And of course I can’t resist providing my own take on Newcomb’s:
What matters is not which box I choose, but which box Omega predicts I will choose. So I can restate the problem like so:
“Can I deceive a super-intelligence into predicting I will One Box?”
Payoff matrix:
So to maximize the payoff:
No ⇒ One Box
Yes ⇒ Two Box
On priors humans should go with No and thus One Box. Taking into account the track record of the super-intelligence mentioned in the article (100 correct predictions out of 100) only reinforces that No.
Unfortunately this doesn’t pass the Ideological Turing Test. Like, your argument would work on people that already agree with your frame, and wouldn’t on people with different frame who two box. It’s really kind of hard to step in into that mindset, and I agree that it’s ridiculously wrong, but you are not targeting it.
If I was a two boxer I would say something like:
In each situation that Omega could have put me in, two boxing gives me more money than one boxing. Omega is in my past, and my actions can’t control it. Therefore I should two box. Your payoff matrix is irrelevant.
Are you sure you’ve chosen the right payoff matrix? All you know about Omega is that it is a superintelligence from another galaxy that has a very good track record of correct predictions. You don’t actually know whether your ability (or not) to deceive Omega is relevant.