Well, that’s more “what laws should there be/what sort of enforcement ought there to be?” I was more asking with regards to “what underlying rule is the Rational Way”? :)
ie, some level of being willing to do what you’re told even if it’s not optimal is a consequence of the need to coordinate groups and stuff. ie, the various newcomb arguments and so on.. I’m just trying to figure out where that breaks down, how stupid the orders have to seem before that implodes, if ever.
The morality one was easier, since the “obey even though you think you know better” thing is based on one boxing and having the goal of winning that battle/whatever. If losing would actually be preferable to obeying, then the single iteration PD/newcomb problem type stuff doesn’t seem to come up as strongly.
Any idea what an explicit rule a rationalist should follow with regards to this might look like? (not necesarally “how do we enforce it” though. Separate question)
Even an upper limit criteria would be okay. ie, something of the form “I don’t know the exact dividing line, but I think I can argue that at least if it gets to this point, then disobeying is rational”
(which is what I did for the morality one, with the “better to lose than obey” criteria.)
No, I don’t have a boiled down answer. When I try to think about it, rational/right includes not just the outcome of the current engagement, but the incentives and lessons left behind for the next incident.
Okay, here’s one example I’ve used before: torture. It’s somewhat orthogonal to the question of following orders, but it bears on the issue of setting up incentives for how often breaking the rules is acceptable. I think the law and the practice should be that torture is illegal and punished strictly. If some person is convinced that imminent harm will result if information isn’t extracted from a suspect, and that it’s worth going to jail for a long time in order to prevent the harm, then they are able to (which is not the same as authorized) torture. But it’s always at the cost of personal sacrifice. So, if you think a million people will die from a nuke, and you’re convinced you can actually get information out of someone by immoral and prohibited means (which I think is usually the weakest link in the chain) and you’re willing to give up your life or your liberty in order to prevent it, then go for it.
But don’t ever expect a hero’s welcome for your sacrifice. It’s a bad choice that’s (conceivably) sometimes necessary. The idea that any moral society would authorize the use of torture in routine situations makes me sick.
I think people exist who will make the personal sacrifice of going to jail for a long time to prevent the nuke from going off. But I do not think people exist who will also sacrifice a friend. But under American law that is what a person would have to do to consult with a friend on the decision of whether to torture: American law punishes people who have foreknowledge of certain crimes but do not convey their foreknowledge to the authorities. So the person is faced with making what may well be the most important decision of their lives without help from any friend or conspiring somehow to keep the authorities from learning about the friend’s foreknowledge of the crime. Although I believe that lying is sometimes justified, this particular lie must be planned out simultaneously with the deliberations over the important decision—potentially undermining those deliberations if the person is unused to high-stakes lies—and the person probably is unused to high-stakes lies if he is the kind of person seriously considering such a large personal sacrifice.
Yes. I am sympathetic to that view of “how to deal with stuff like torture/etc”, but that doesn’t answer “when to do it”.
ie, I wasn’t saying “when should it be ‘officially permitted’?” but rather at what point should a rationalist do so? how convinced does a rationalist need to be, if ever?
Or did I completely misunderstand what you were saying?
No, you understood me. I sidestepped the heart of the question.
This is an example where I believe I know what the right incentives structure of the answer is. But I can’t give any guidance on the root question, since in my example case, (torture) I don’t believe in the efficacy of the immoral act. I don’t think you can procure useful information by torturing someone when time is short. And when time isn’t short, there are better choices.
I guess the big question here is why do you not believe it. Since you (and I!) would prefer to live in a world where torture is not effective, we must be aware that our biases is to believe it is not effective—it makes the world nicer. Hence, we must conciously shift up our belief in the effectiveness of torture from our “gut feeling.” Given that, what evidence have you seen that for the purposes of solving NP-like problems (meaning, a problem where a solution is hard to find but easy to verify like “where is the bomb hidden”) is not effective. I would say that for me personally, the amount that my preferences shift in the presence of relatively mild pain (“I prefer not to medicate myself” vs. “Gimme that goddamn pill”) is at least cause to suspect that someone who is an expert at causing vast amounts of pain would be able to make me do things I would normally prefer not to do (like tell them where I hid the bomb) to stop that pain.
Of course, torture used for unverifiable information is completely useless for exactly the same reason—the prisoner will say anything they can get away with to make the pain stop.
Maybe my previous answer would have been cleaner if I had said “I don’t think I can procure useful information by torturing someone when time is short.” It’s a relatively easy choice for me, since I doubt that even with proper tools, that I could appropriately gauge the level of pain to the necessary calibration in order to get detailed information in a few minutes or hours.
When I think about other people who might have more experience, it’s hard to imagine someone who had repeatedly fallen into the situation where they were the right person to perform the torture so they had enough experience to both make the call, and effectively extract information. Do you want to argue that they could have gotten to that point without violating our sense of morality?
Since my question is “What should the law be?”, not “is it ever conceivable that torture could be effective?” I still have to say that the law should forbid torture, and people should expect to be punished if they torture. There may be cases where you or I would agree that in that circumstance it was the necessary thing to do, but I still believe that the system should never condone it.
You talked about two issues that have little to do with each other:
What should the law be? (I didn’t argue with your point here, so re-iterating it is useless?)
A statement that was misleading: apparently you meant that you’re not a good torturer. That is not impossible. I think that given a short amount of time, with someone who knows something specific (where the bomb is hidden), my best chance (in effective, not moral, ordering) is to torture them. I’m not a professional torturer, I luckily never had to torture anyone, but like any human, I have an understanding in pain. I’ve watched movies about torture, and I’ve heard about waterboarding. If I decided that this was the ethical thing to do (which be both agree, in some cases is possible), and I was the only one around, I’d probably try waterboarding. It’s risky, there’s a chance the prisoner might die, but if I have one hour, and 50 million people will die otherwise, I don’t see any better way. So let me ask you flat out—I’m assuming you also read about waterboarding, and that when you need to, you have access to the WP article about waterboarding. What would you do in that situation? Ask nicely?
All that does not go to condone torture. I’m just saying, if a nation of Rationalists is fighting with the Barbarians, then it’s not necessarily in their best interests to decide they will never torture no matter what.
My point wasn’t just that I wouldn’t make a good torturer. It seems to me that ordinary circumstances don’t provide many opportunities for anyone to learn much about torture, (other than from fictional sources). I have little reason to believe that inexperienced torturers would be effective in the time-critical circumstances that seem necessary for any convincing justification of torture. You may believe it, but it’s not convincing to me. So it would be hard to ethically produce trained torturers, and there’s a dearth of evidence on the effectiveness of inexperienced torturers in the circumstances necessary to justify it.
Given that, I think it’s better to take the stance that torture is always unethical. There are conceivable circumstances when it would be the only way to prevent a cataclysm, but they’re neither common, nor easy to prepare for.
And I don’t think I’ve said that it would be ethical, just that individuals would sometimes think it was necessary. I think we are all better off if they have to make that choice without any expectation that we will condone their actions. Otherwise, some will argue that it’s useful to have a course of training in how to perform torture, which would encourage its use even though we don’t have evidence of its usefulness. It seems difficult to produce evidence one way or another on the efficacy of torture without violating the spirit of the Nuremberg Code. I don’t see an ethical way to add to the evidence.
You seem to believe that sufficient evidence exists. Can you point to any?
You wanted an explicit answer to your question. My response is that I would be unhappy that I didn’t have effective tools for finding out the truth. But my unhappiness doesn’t change the facts of the situation. There isn’t always something useful that you can do. When I generalize over all the fictional evidence I’ve been exposed to, it’s too likely that my evidence is wrong as to the identity of the suspect, or he doesn’t have the info I want, or the bomb can’t be disabled anyway. When I try to think of actual circumstances, I don’t come up with examples in which time was short and the information produced was useful. I also can’t imagine myself personally punching, pistol-whipping, pulling fingernails, waterboarding, etc, nor ordering the experienced torturer (who you want me to imagine is under my command) to do so.
Sorry to disappoint you, but I don’t believe the arguments I’ve heard for effectiveness or morality of torture.
Yeah, the “do it, but keep it illegal and be punished for it even if it was needed” is a possible solution given “in principle it may be useful”, which is a whole other question.
But anyways, I was talking about “when should a rationalist soldier be willing to disobey in the name of ‘I think my CO is giving really stupid orders’?”, since I believe I already have a partial solution to the “I think my CO is giving really immoral orders” case (as described above)
As far as when torture would even be plausibly useful (especially plausibly optimal) for obtaining info? I can’t really currently think of any non-contrived situations.
Well, that’s more “what laws should there be/what sort of enforcement ought there to be?” I was more asking with regards to “what underlying rule is the Rational Way”? :)
ie, some level of being willing to do what you’re told even if it’s not optimal is a consequence of the need to coordinate groups and stuff. ie, the various newcomb arguments and so on.. I’m just trying to figure out where that breaks down, how stupid the orders have to seem before that implodes, if ever.
The morality one was easier, since the “obey even though you think you know better” thing is based on one boxing and having the goal of winning that battle/whatever. If losing would actually be preferable to obeying, then the single iteration PD/newcomb problem type stuff doesn’t seem to come up as strongly.
Any idea what an explicit rule a rationalist should follow with regards to this might look like? (not necesarally “how do we enforce it” though. Separate question)
Even an upper limit criteria would be okay. ie, something of the form “I don’t know the exact dividing line, but I think I can argue that at least if it gets to this point, then disobeying is rational”
(which is what I did for the morality one, with the “better to lose than obey” criteria.)
No, I don’t have a boiled down answer. When I try to think about it, rational/right includes not just the outcome of the current engagement, but the incentives and lessons left behind for the next incident.
Okay, here’s one example I’ve used before: torture. It’s somewhat orthogonal to the question of following orders, but it bears on the issue of setting up incentives for how often breaking the rules is acceptable. I think the law and the practice should be that torture is illegal and punished strictly. If some person is convinced that imminent harm will result if information isn’t extracted from a suspect, and that it’s worth going to jail for a long time in order to prevent the harm, then they are able to (which is not the same as authorized) torture. But it’s always at the cost of personal sacrifice. So, if you think a million people will die from a nuke, and you’re convinced you can actually get information out of someone by immoral and prohibited means (which I think is usually the weakest link in the chain) and you’re willing to give up your life or your liberty in order to prevent it, then go for it.
But don’t ever expect a hero’s welcome for your sacrifice. It’s a bad choice that’s (conceivably) sometimes necessary. The idea that any moral society would authorize the use of torture in routine situations makes me sick.
I think people exist who will make the personal sacrifice of going to jail for a long time to prevent the nuke from going off. But I do not think people exist who will also sacrifice a friend. But under American law that is what a person would have to do to consult with a friend on the decision of whether to torture: American law punishes people who have foreknowledge of certain crimes but do not convey their foreknowledge to the authorities. So the person is faced with making what may well be the most important decision of their lives without help from any friend or conspiring somehow to keep the authorities from learning about the friend’s foreknowledge of the crime. Although I believe that lying is sometimes justified, this particular lie must be planned out simultaneously with the deliberations over the important decision—potentially undermining those deliberations if the person is unused to high-stakes lies—and the person probably is unused to high-stakes lies if he is the kind of person seriously considering such a large personal sacrifice.
Any suggestions for the person?
Discuss a hypothetical situation with your friend that happens to match up in all particulars with the real-world situation, which you do not discuss.
It isn’t actually important here that your friend be fooled, the goal is to give your friend plausible deniability to protect her from litigation.
Yes. I am sympathetic to that view of “how to deal with stuff like torture/etc”, but that doesn’t answer “when to do it”.
ie, I wasn’t saying “when should it be ‘officially permitted’?” but rather at what point should a rationalist do so? how convinced does a rationalist need to be, if ever?
Or did I completely misunderstand what you were saying?
No, you understood me. I sidestepped the heart of the question.
This is an example where I believe I know what the right incentives structure of the answer is. But I can’t give any guidance on the root question, since in my example case, (torture) I don’t believe in the efficacy of the immoral act. I don’t think you can procure useful information by torturing someone when time is short. And when time isn’t short, there are better choices.
I guess the big question here is why do you not believe it. Since you (and I!) would prefer to live in a world where torture is not effective, we must be aware that our biases is to believe it is not effective—it makes the world nicer. Hence, we must conciously shift up our belief in the effectiveness of torture from our “gut feeling.” Given that, what evidence have you seen that for the purposes of solving NP-like problems (meaning, a problem where a solution is hard to find but easy to verify like “where is the bomb hidden”) is not effective. I would say that for me personally, the amount that my preferences shift in the presence of relatively mild pain (“I prefer not to medicate myself” vs. “Gimme that goddamn pill”) is at least cause to suspect that someone who is an expert at causing vast amounts of pain would be able to make me do things I would normally prefer not to do (like tell them where I hid the bomb) to stop that pain.
Of course, torture used for unverifiable information is completely useless for exactly the same reason—the prisoner will say anything they can get away with to make the pain stop.
Maybe my previous answer would have been cleaner if I had said “I don’t think I can procure useful information by torturing someone when time is short.” It’s a relatively easy choice for me, since I doubt that even with proper tools, that I could appropriately gauge the level of pain to the necessary calibration in order to get detailed information in a few minutes or hours.
When I think about other people who might have more experience, it’s hard to imagine someone who had repeatedly fallen into the situation where they were the right person to perform the torture so they had enough experience to both make the call, and effectively extract information. Do you want to argue that they could have gotten to that point without violating our sense of morality?
Since my question is “What should the law be?”, not “is it ever conceivable that torture could be effective?” I still have to say that the law should forbid torture, and people should expect to be punished if they torture. There may be cases where you or I would agree that in that circumstance it was the necessary thing to do, but I still believe that the system should never condone it.
You talked about two issues that have little to do with each other:
What should the law be? (I didn’t argue with your point here, so re-iterating it is useless?)
A statement that was misleading: apparently you meant that you’re not a good torturer. That is not impossible. I think that given a short amount of time, with someone who knows something specific (where the bomb is hidden), my best chance (in effective, not moral, ordering) is to torture them. I’m not a professional torturer, I luckily never had to torture anyone, but like any human, I have an understanding in pain. I’ve watched movies about torture, and I’ve heard about waterboarding. If I decided that this was the ethical thing to do (which be both agree, in some cases is possible), and I was the only one around, I’d probably try waterboarding. It’s risky, there’s a chance the prisoner might die, but if I have one hour, and 50 million people will die otherwise, I don’t see any better way. So let me ask you flat out—I’m assuming you also read about waterboarding, and that when you need to, you have access to the WP article about waterboarding. What would you do in that situation? Ask nicely?
All that does not go to condone torture. I’m just saying, if a nation of Rationalists is fighting with the Barbarians, then it’s not necessarily in their best interests to decide they will never torture no matter what.
My point wasn’t just that I wouldn’t make a good torturer. It seems to me that ordinary circumstances don’t provide many opportunities for anyone to learn much about torture, (other than from fictional sources). I have little reason to believe that inexperienced torturers would be effective in the time-critical circumstances that seem necessary for any convincing justification of torture. You may believe it, but it’s not convincing to me. So it would be hard to ethically produce trained torturers, and there’s a dearth of evidence on the effectiveness of inexperienced torturers in the circumstances necessary to justify it.
Given that, I think it’s better to take the stance that torture is always unethical. There are conceivable circumstances when it would be the only way to prevent a cataclysm, but they’re neither common, nor easy to prepare for.
And I don’t think I’ve said that it would be ethical, just that individuals would sometimes think it was necessary. I think we are all better off if they have to make that choice without any expectation that we will condone their actions. Otherwise, some will argue that it’s useful to have a course of training in how to perform torture, which would encourage its use even though we don’t have evidence of its usefulness. It seems difficult to produce evidence one way or another on the efficacy of torture without violating the spirit of the Nuremberg Code. I don’t see an ethical way to add to the evidence.
You seem to believe that sufficient evidence exists. Can you point to any?
You wanted an explicit answer to your question. My response is that I would be unhappy that I didn’t have effective tools for finding out the truth. But my unhappiness doesn’t change the facts of the situation. There isn’t always something useful that you can do. When I generalize over all the fictional evidence I’ve been exposed to, it’s too likely that my evidence is wrong as to the identity of the suspect, or he doesn’t have the info I want, or the bomb can’t be disabled anyway. When I try to think of actual circumstances, I don’t come up with examples in which time was short and the information produced was useful. I also can’t imagine myself personally punching, pistol-whipping, pulling fingernails, waterboarding, etc, nor ordering the experienced torturer (who you want me to imagine is under my command) to do so.
Sorry to disappoint you, but I don’t believe the arguments I’ve heard for effectiveness or morality of torture.
Yeah, the “do it, but keep it illegal and be punished for it even if it was needed” is a possible solution given “in principle it may be useful”, which is a whole other question.
But anyways, I was talking about “when should a rationalist soldier be willing to disobey in the name of ‘I think my CO is giving really stupid orders’?”, since I believe I already have a partial solution to the “I think my CO is giving really immoral orders” case (as described above)
As far as when torture would even be plausibly useful (especially plausibly optimal) for obtaining info? I can’t really currently think of any non-contrived situations.
How about this upper limit: when the outcome of (everyone) following orders would be worse than everyone doing their own thing, disobey.