More people will think like this if there is a large body of stories involving great criminals with an unlikely frequency of minor harms and inconveniences in their past.
In this hypothetical society, how would/should you react if you were the target of an unlikely frequency of minor harms?
Consider Newcomb’s Problem with transparent boxes. Even if you see that box B is empty, you should still one-box. For the same reason, even if you’re getting punished, you should still not become a criminal—and not out of moral concerns but for your own benefit.
Depends on what about it seems wrong. Do you disagree that you should one-box if you see that box B is empty? It’s unintuitive, but it’s the strategy that UDT stratightforwardly yields and that you would want to precommit to. Here’s one bit of intuition: by being the kind of person that one-boxes even with an empty box, you force Omega to never give you an empty box, on pain of being wrong. Maybe these are relevant too.
Ok, it’s like CM: right now (before Omega shows up) I want to be the kind of person who always one-boxes even if the box is empty, so that I’ll never get an empty box. That is the rational and correct choice now.
This is not, however, the same thing as saying that the rational choice for someone staring at an empty B box is to one-box. It’s a scenario that will never materialise if you don’t screw up, but if you take it as the hypothesis that you do find yourself in that scenario (because, for example, you weren’t rational before meeting Omega, but became perfectly rational afterwards), the rational answer for that scenario is to two-box. Yes, it does mean you screwed up by not wanting it sincerely enough, but it’s the question that assumes you’ve already screwed up.
Translating this to the pre-punishment scenario, what this means is that—assuming a sufficient severity of average pre-punishment—a rational person will not want to ever become a criminal. So a rational person will never be pre-punished anyway. But if Normal_Anomaly asks: “Assume that you’ve been pre-punished; should you then commit crimes?” the answer is “Yes, but note that your hypothesis can only be true if I hadn’t been perfectly rational in the past”.
(Separate observation: Omega puts a million dollars in B iff you will one-box. Omega then reads my strategy: “if box B is opaque or transparent and full, I will one-box; if it is transparent and empty, I two-box”. If box B is opaque, this forces Omega to put the money there. But if B is transparent, Omega will be right no matter what. Are we authorised to assume that Omega will choose to flip a coin in this scenario, or should we just say that the problem isn’t well-posed for a transparent box? I’m leaning towards the latter. If the box is transparent and your choice is conditional on its content, you’ve effectively turned Omega’s predictive ability against itself. You’ll one-box iff Omega puts a million dollar there iff you one-box, loop.)
Transparent Newcomb is well-posed but, I admit, underspecified. So add this rule:
Omega fills box B if you would one-box no matter what, leaves box B empty if you would two-box no matter what, flips a coin if you would one-box given a full box B and two-box given an empty box B, and doesn’t invite you to his games in the first place you if you would two-box given a full box B and one-box given an empty box B.
Indeed, you could have any mapping from pairs of (probability distributions over) actions to box-states, where the first element of the pair is what you would do if you saw a filled box B, and the second element is what you would do if you saw an empty box B. But I’m trying to preserve the spirit of the original Newcomb.
Can you explain and/or link this analysis of transparent Newcomb? It looks very wrong to me.
It’s only wrong if you are the kind of person who doesn’t like getting $1,000,000.
If only all our knowledge of our trading partners and environment was as reliable as ‘fundamentally included in the very nature of the problem specification’. You have to think a lot harder when you are only kind of confident and know the limits of your own mind reading capabilities.
If only all our knowledge of our trading partners and environment was as reliable as ‘fundamentally included in the very nature of the problem specification’.
If you’re going to make that kind of argument, you’re dismissing pretty much all LW-style thought experiments.
If you’re going to make that kind of argument, you’re dismissing pretty much all LW-style thought experiments.
I think you’re reading in an argument that isn’t there. I was explaining the most common reason why human intuitions fail so blatantly when encountering transparent Newcomb. If anything that is more reason to formalise it as a thought experiment.
In this hypothetical society, how would/should you react if you were the target of an unlikely frequency of minor harms?
Consider Newcomb’s Problem with transparent boxes. Even if you see that box B is empty, you should still one-box. For the same reason, even if you’re getting punished, you should still not become a criminal—and not out of moral concerns but for your own benefit.
Can you explain and/or link this analysis of transparent Newcomb? It looks very wrong to me.
Depends on what about it seems wrong. Do you disagree that you should one-box if you see that box B is empty? It’s unintuitive, but it’s the strategy that UDT stratightforwardly yields and that you would want to precommit to. Here’s one bit of intuition: by being the kind of person that one-boxes even with an empty box, you force Omega to never give you an empty box, on pain of being wrong. Maybe these are relevant too.
Ok, it’s like CM: right now (before Omega shows up) I want to be the kind of person who always one-boxes even if the box is empty, so that I’ll never get an empty box. That is the rational and correct choice now.
This is not, however, the same thing as saying that the rational choice for someone staring at an empty B box is to one-box. It’s a scenario that will never materialise if you don’t screw up, but if you take it as the hypothesis that you do find yourself in that scenario (because, for example, you weren’t rational before meeting Omega, but became perfectly rational afterwards), the rational answer for that scenario is to two-box. Yes, it does mean you screwed up by not wanting it sincerely enough, but it’s the question that assumes you’ve already screwed up.
Translating this to the pre-punishment scenario, what this means is that—assuming a sufficient severity of average pre-punishment—a rational person will not want to ever become a criminal. So a rational person will never be pre-punished anyway. But if Normal_Anomaly asks: “Assume that you’ve been pre-punished; should you then commit crimes?” the answer is “Yes, but note that your hypothesis can only be true if I hadn’t been perfectly rational in the past”.
(Separate observation: Omega puts a million dollars in B iff you will one-box. Omega then reads my strategy: “if box B is opaque or transparent and full, I will one-box; if it is transparent and empty, I two-box”. If box B is opaque, this forces Omega to put the money there. But if B is transparent, Omega will be right no matter what. Are we authorised to assume that Omega will choose to flip a coin in this scenario, or should we just say that the problem isn’t well-posed for a transparent box? I’m leaning towards the latter. If the box is transparent and your choice is conditional on its content, you’ve effectively turned Omega’s predictive ability against itself. You’ll one-box iff Omega puts a million dollar there iff you one-box, loop.)
Transparent Newcomb is well-posed but, I admit, underspecified. So add this rule:
Omega fills box B if you would one-box no matter what, leaves box B empty if you would two-box no matter what, flips a coin if you would one-box given a full box B and two-box given an empty box B, and doesn’t invite you to his games in the first place you if you would two-box given a full box B and one-box given an empty box B.
Wrong rules, corrected here.
...or an Omega that fills box B unless you would two-box no matter what.
Indeed, you could have any mapping from pairs of (probability distributions over) actions to box-states, where the first element of the pair is what you would do if you saw a filled box B, and the second element is what you would do if you saw an empty box B. But I’m trying to preserve the spirit of the original Newcomb.
Sorry, decided that comment wasn’t ready and deleted it, but you managed to see it. See my other comment.
It’s only wrong if you are the kind of person who doesn’t like getting $1,000,000.
If only all our knowledge of our trading partners and environment was as reliable as ‘fundamentally included in the very nature of the problem specification’. You have to think a lot harder when you are only kind of confident and know the limits of your own mind reading capabilities.
If you’re going to make that kind of argument, you’re dismissing pretty much all LW-style thought experiments.
I think you’re reading in an argument that isn’t there. I was explaining the most common reason why human intuitions fail so blatantly when encountering transparent Newcomb. If anything that is more reason to formalise it as a thought experiment.