So he’s using a counterexample that’s predicated on a logical inconsistency and could not happen. If a decision theory fails in situations that couldn’t really happen, that’s actually not a problem.
but that counterexample isn’t predicated on a logical inconsistency. An FDT agent can still get blackmailed, by someone who doesn’t know it’s an FDT agent or who doesn’t care that the outcome will be bad or who is lousy at predicting the behaviour of FDT agents.
What prevents the FDT agent from getting blackmailed is not that he is known to be an FDT agent. Rather, it’s simply that he’s known to be the sort of person who will blow the whistle on blackmailers. Potential blackmailers need not know or care why the FDT agent behaves like this (indeed, it does not matter at all, for the purpose of each individual case, why the agent behaves like this—whether it’s because the dictates of FDT so direct him, or because he’s naturally stubborn, or whatever; it suffices that blackmailers expect the agent to blow the whistle).
So if we stipulate an FDT agent, we also, thereby, stipulate that he is known to be a whistle-blower. (We’re assuming, in every version of this scenario, that, regardless of the agent’s motivations, his propensity toward such behavior is known.) And that does indeed make the “he’s blackmailed anyway” scenario logically inconsistent.
Even if you are known to be the sort of person who will blow the whistle on blackmailers it is still possible that someone will try to blackmail you. How could that possibly involve a logical inconsistency? (People do stupid things all the time.)
I’ll say it again in different words, I did not understand the paper (and consequently, the blog) to be talking about actual blackmail in a big messy physical world. I understood them to be talking about a specific, formalized blackmail scenario, in which the blackmailer’s decision to blackmail is entirely contingent on the victim’s counterfactional behaviour, in which case resolving to never pay and still being blackmailed isn’t possible- in full context, it’s logically inconsistent.
Different formalisation are possible, but I’d guess the strict one is what was used. In the softer ones you still generally wont pay.
The paper discusses two specific blackmail scenarios. One (“XOR Blackmail”) is a weirdly contrived situation and I don’t think any of what wo says is talking about it. The other (“Mechanical Blackmail”) is a sort of stylized version of real blackmail scenarios, and does assume that the blackmailer is a perfect predictor. The paper’s discussion of Mechanical Blackmail then considers the case where the blackmailer is an imperfect (but still very reliable) predictor, and says that there too an FDT agent should refuse to pay.
wo’s discussion of blackmail doesn’t directly address either of the specific scenarios discussed in the FDT paper. The first blackmail scenario wo discusses (before saying much about FDT) is a generic case of blackmail (the participants being labelled “Donald” and “Stormy”, perhaps suggesting that we are not supposed to imagine either of them as any sort of perfectly rational perfect predictor...). Then, when specifically discussing FDT, wo considers a slightly different scenario, which again is clearly not meant to involve perfect prediction, simulation, etc., because he says things like ” FDT says you should not pay because, if you were the kind of person who doesn’t pay, you likely wouldn’t have been blackmailed” and “FDT agents who are known as FDT agents have a lower chance of getting blackmailed” (emphasis mine, in both cases).
So. The paper considers a formalized scenario in which the blackmailer’s decision is made on the basis of a perfectly accurate prediction of the victim, but then is at pains to say that it all works just the same if the prediction is imperfect. The blog considers only imperfect-prediction scenarios. Real-world blackmail, of course, never involves anything close to the sort of perfect prediction that would make it take a logical inconsistency for an FDT agent to get blackmailed.
So taking the paper and the blogpost to be talking only about provably-perfect-prediction scenarios seems to me to require (1) reading the paper oddly selectively, (2) interpreting the blogpost very differently from me, and (3) not caring about situations that could ever occur in the actual world, even though wo is clearly concerned with real-world applicability and the paper makes at least some gestures towards such applicability.
For the avoidance of doubt, I don’t think there’s necessarily anything wrong with being interested primarily in such scenarios: the best path to understanding how a theory works in practice might well go via highly simplified scenarios. But it seems to me simply untrue that the paper, still more the blogpost, considers (or should have considered) only such scenarios when discussing blackmail.
It might. It’s pretty long. I don’t suppose you’d like to be more specific?
(If the answer is no, you wouldn’t, because all you mean is that it seems like the sort of discussion that might have useful things in it, then fair enough of course.)
I read that under a subtext where we were talking about the same blackmail scenario, but, okay, others are possible.
In cases where the blackmail truly seems not to be contingent on its policy, (and in many current real-world cases) the FDT agent will pay.
The only cases when an FDT agent actually will get blackmailed and refuse to pay are cases where being committed to not paying shifts the probabilities enough to make that profitable on average.
It is possible to construct obstinate kinds of agents that aren’t sensitive to FDT’s acausal dealmaking faculties. Evolution might produce them often. They will not be seen as friends. As an LDT-like human, my feelings towards those sorts of blackmailers is that we should destroy all of them as quickly as we can, because their existence is a blight to ours. In light of that, I’m not sure they have a winning strategy. When you start to imagine the directed ocean of coordinated violence that an LDT-aligned faction (so, literally any real-world state with laws against blackmail) points in your direction as soon as they can tell what you are, you may start to wonder if pretending you can’t understand their source code is really a good idea.
I imagine a time when the distinction between CDT and LDT is widely essentially understood, by this time, the very idea of blackmailing will come to seem very strange, we will wonder how there was this era when a person could just say “If you don’t do X, then I will do the fairly self-destructive action Y which I gain nothing from doing” and have everyone just believe them unconditionally, just believe this unqualified statement about their mechanism. Wasn’t that stupid? To lie like that? And even stupider for their victims to pretend that they believe the lie? We will not be able to understand it any more.
Imagine that you see an agnostic community head walking through the park at night. You know it’s a long shot, but you amble towards him, point your gun at him and say “give me your wallet.” He looks back at you and says, “I don’t understand the deal. You’ll shoot me? How does that help you? Because you want my wallet? I don’t understand the logic there, why are those two things related? That doesn’t get you my wallet.”
Only it does, because when you shoot someone you can loot their corpse, so it occurs to me that muggers are a bad example of blackmail. I imagine they’ve got to have a certain amount of comfort with actually killing people, to do that. It’s not really threatening to do something self-destructive, in their view, they still benefit a little from killing you. They still get to empty your pockets. To an extent, mugging is often just a display of a power imbalance and consequent negotiation of a mutually beneficial alternative to violence.
The gang can profit from robbing your store at gunpoint, but you and them both will profit more if you just pay them protection money. LDT only refuses to pay protection money if it realises that having all of the other entangled LDT store owners paying protection money as well would make the gang profitable enough to grow, and that having a grown gang around would have been, on the whole, worse than the amortised risk of being robbed.
You say
but that counterexample isn’t predicated on a logical inconsistency. An FDT agent can still get blackmailed, by someone who doesn’t know it’s an FDT agent or who doesn’t care that the outcome will be bad or who is lousy at predicting the behaviour of FDT agents.
No, this is wrong.
What prevents the FDT agent from getting blackmailed is not that he is known to be an FDT agent. Rather, it’s simply that he’s known to be the sort of person who will blow the whistle on blackmailers. Potential blackmailers need not know or care why the FDT agent behaves like this (indeed, it does not matter at all, for the purpose of each individual case, why the agent behaves like this—whether it’s because the dictates of FDT so direct him, or because he’s naturally stubborn, or whatever; it suffices that blackmailers expect the agent to blow the whistle).
So if we stipulate an FDT agent, we also, thereby, stipulate that he is known to be a whistle-blower. (We’re assuming, in every version of this scenario, that, regardless of the agent’s motivations, his propensity toward such behavior is known.) And that does indeed make the “he’s blackmailed anyway” scenario logically inconsistent.
Even if you are known to be the sort of person who will blow the whistle on blackmailers it is still possible that someone will try to blackmail you. How could that possibly involve a logical inconsistency? (People do stupid things all the time.)
I’ll say it again in different words, I did not understand the paper (and consequently, the blog) to be talking about actual blackmail in a big messy physical world. I understood them to be talking about a specific, formalized blackmail scenario, in which the blackmailer’s decision to blackmail is entirely contingent on the victim’s counterfactional behaviour, in which case resolving to never pay and still being blackmailed isn’t possible- in full context, it’s logically inconsistent.
Different formalisation are possible, but I’d guess the strict one is what was used. In the softer ones you still generally wont pay.
The paper discusses two specific blackmail scenarios. One (“XOR Blackmail”) is a weirdly contrived situation and I don’t think any of what wo says is talking about it. The other (“Mechanical Blackmail”) is a sort of stylized version of real blackmail scenarios, and does assume that the blackmailer is a perfect predictor. The paper’s discussion of Mechanical Blackmail then considers the case where the blackmailer is an imperfect (but still very reliable) predictor, and says that there too an FDT agent should refuse to pay.
wo’s discussion of blackmail doesn’t directly address either of the specific scenarios discussed in the FDT paper. The first blackmail scenario wo discusses (before saying much about FDT) is a generic case of blackmail (the participants being labelled “Donald” and “Stormy”, perhaps suggesting that we are not supposed to imagine either of them as any sort of perfectly rational perfect predictor...). Then, when specifically discussing FDT, wo considers a slightly different scenario, which again is clearly not meant to involve perfect prediction, simulation, etc., because he says things like ” FDT says you should not pay because, if you were the kind of person who doesn’t pay, you likely wouldn’t have been blackmailed” and “FDT agents who are known as FDT agents have a lower chance of getting blackmailed” (emphasis mine, in both cases).
So. The paper considers a formalized scenario in which the blackmailer’s decision is made on the basis of a perfectly accurate prediction of the victim, but then is at pains to say that it all works just the same if the prediction is imperfect. The blog considers only imperfect-prediction scenarios. Real-world blackmail, of course, never involves anything close to the sort of perfect prediction that would make it take a logical inconsistency for an FDT agent to get blackmailed.
So taking the paper and the blogpost to be talking only about provably-perfect-prediction scenarios seems to me to require (1) reading the paper oddly selectively, (2) interpreting the blogpost very differently from me, and (3) not caring about situations that could ever occur in the actual world, even though wo is clearly concerned with real-world applicability and the paper makes at least some gestures towards such applicability.
For the avoidance of doubt, I don’t think there’s necessarily anything wrong with being interested primarily in such scenarios: the best path to understanding how a theory works in practice might well go via highly simplified scenarios. But it seems to me simply untrue that the paper, still more the blogpost, considers (or should have considered) only such scenarios when discussing blackmail.
This thread might be relevant to your question.
It might. It’s pretty long. I don’t suppose you’d like to be more specific?
(If the answer is no, you wouldn’t, because all you mean is that it seems like the sort of discussion that might have useful things in it, then fair enough of course.)
I read that under a subtext where we were talking about the same blackmail scenario, but, okay, others are possible.
In cases where the blackmail truly seems not to be contingent on its policy, (and in many current real-world cases) the FDT agent will pay.
The only cases when an FDT agent actually will get blackmailed and refuse to pay are cases where being committed to not paying shifts the probabilities enough to make that profitable on average.
It is possible to construct obstinate kinds of agents that aren’t sensitive to FDT’s acausal dealmaking faculties. Evolution might produce them often. They will not be seen as friends. As an LDT-like human, my feelings towards those sorts of blackmailers is that we should destroy all of them as quickly as we can, because their existence is a blight to ours. In light of that, I’m not sure they have a winning strategy. When you start to imagine the directed ocean of coordinated violence that an LDT-aligned faction (so, literally any real-world state with laws against blackmail) points in your direction as soon as they can tell what you are, you may start to wonder if pretending you can’t understand their source code is really a good idea.
I imagine a time when the distinction between CDT and LDT is widely essentially understood, by this time, the very idea of blackmailing will come to seem very strange, we will wonder how there was this era when a person could just say “If you don’t do X, then I will do the fairly self-destructive action Y which I gain nothing from doing” and have everyone just believe them unconditionally, just believe this unqualified statement about their mechanism. Wasn’t that stupid? To lie like that? And even stupider for their victims to pretend that they believe the lie? We will not be able to understand it any more.
Imagine that you see an agnostic community head walking through the park at night. You know it’s a long shot, but you amble towards him, point your gun at him and say “give me your wallet.” He looks back at you and says, “I don’t understand the deal. You’ll shoot me? How does that help you? Because you want my wallet? I don’t understand the logic there, why are those two things related? That doesn’t get you my wallet.”
Only it does, because when you shoot someone you can loot their corpse, so it occurs to me that muggers are a bad example of blackmail. I imagine they’ve got to have a certain amount of comfort with actually killing people, to do that. It’s not really threatening to do something self-destructive, in their view, they still benefit a little from killing you. They still get to empty your pockets. To an extent, mugging is often just a display of a power imbalance and consequent negotiation of a mutually beneficial alternative to violence.
The gang can profit from robbing your store at gunpoint, but you and them both will profit more if you just pay them protection money. LDT only refuses to pay protection money if it realises that having all of the other entangled LDT store owners paying protection money as well would make the gang profitable enough to grow, and that having a grown gang around would have been, on the whole, worse than the amortised risk of being robbed.