Dialogue on Appeals to Consequences

Link post

[note: the fol­low­ing is es­sen­tially an ex­panded ver­sion of this LessWrong com­ment on whether ap­peals to con­se­quences are nor­ma­tive in dis­course. I am ex­as­per­ated that this is even up for de­bate, but I figure that mak­ing the ar­gu­men­ta­tion here ex­plicit is helpful]

Carter and Quinn are dis­cussing char­i­ta­ble mat­ters in the town square, with a few on­look­ers.

Carter: “So, this lo­cal char­ity, Peo­ple Against Drown­ing Pup­pies (PADP), is nom­i­nally op­posed to drown­ing pup­pies.”

Quinn: “Of course.”

Carter: “And they said they’d saved 2170 pup­pies last year, whereas their to­tal spend­ing was $1.2 mil­lion, so they es­ti­mate they save one puppy per $553.”

Quinn: “Sounds about right.”

Carter: “So, I ac­tu­ally checked with some of their former em­ploy­ees, and if what they say and my cor­re­spond­ing calcu­la­tions are right, they ac­tu­ally only saved 138 pup­pies.”

Quinn: “Hold it right there. Re­gard­less of whether that’s true, it’s bad to say that.”

Carter: “That’s an ap­peal to con­se­quences, well-known to be a log­i­cal fal­lacy.”

Quinn: “Is that re­ally a fal­lacy, though? If say­ing some­thing has bad con­se­quences, isn’t it nor­ma­tive not to say it?”

Carter: “Well, for my own per­sonal de­ci­sion­mak­ing, I’m broadly a con­se­quen­tial­ist, so, yes.”

Quinn: “Well, it fol­lows that ap­peals to con­se­quences are valid.”

Carter: “It isn’t log­i­cally valid. If say­ing some­thing has bad con­se­quences, that doesn’t make it false.”

Quinn: “But it is de­ci­sion-the­o­ret­i­cally com­pel­ling, right?”

Carter: “In the­ory, if it could be proven, yes. But, you haven’t offered any proof, just a state­ment that it’s bad.”

Quinn: “Okay, let’s dis­cuss that. My ar­gu­ment is: PADP is a good char­ity. There­fore, they should be get­ting more dona­tions. Say­ing that they didn’t save as many pup­pies as they claimed they did, in pub­lic (as you just did), is go­ing to re­sult in them get­ting fewer dona­tions. There­fore, your say­ing that they didn’t save as many pup­pies as they claimed to is bad, and is caus­ing more pup­pies to drown.”

Carter: “While I could spend more effort to re­fute that ar­gu­ment, I’ll ini­tially note that you only took into ac­count a sin­gle effect (peo­ple donat­ing less to PADP) and ne­glected other effects (such as peo­ple hav­ing more ac­cu­rate be­liefs about how char­i­ties work).”

Quinn: “Still, you have to ad­mit that my case is plau­si­ble, and that some on­look­ers are con­vinced.”

Carter: “Yes, it’s plau­si­ble, in that I don’t have a full re­fu­ta­tion, and my mod­els have a lot of un­cer­tainty. This gets into some com­pli­cated de­ci­sion the­ory and so­ciolog­i­cal mod­el­ing. I’m afraid we’ve got­ten side­tracked from the rel­a­tively clear con­ver­sa­tion, about how many pup­pies PADP saved, to a rel­a­tively un­clear one, about the de­ci­sion the­ory of mak­ing ac­tual char­ity effec­tive­ness clear to the pub­lic.”

Quinn: “Well, sure, we’re into the weeds now, but this is im­por­tant! If it’s ac­tu­ally bad to say what you said, it’s im­por­tant that this is widely rec­og­nized, so that we can have fewer… mis­takes like that.”

Carter: “That’s cor­rect, but I feel like I might be get­ting trol­led. Any­way, I think you’re shoot­ing the mes­sen­ger: when I started crit­i­ciz­ing PADP, you turned around and made the crit­i­cism about me say­ing that, di­rect­ing at­ten­tion against PADP’s pos­si­ble fraud­u­lent ac­tivity.”

Quinn: “You still haven’t re­futed my ar­gu­ment. If you don’t do so, I win by de­fault.”

Carter: “I’d re­ally rather that we just out­law ap­peals to con­se­quences, but, fine, as long as we’re here, I’m go­ing to do this, and it’ll be a learn­ing ex­pe­rience for ev­ery­one in­volved. First, you said that PADP is a good char­ity. Why do you think this?”

Quinn: “Well, I know the peo­ple there and they seem nice and hard­work­ing.”

Carter: “But, they said they saved over 2000 pup­pies last year, when they ac­tu­ally only saved 138, in­di­cat­ing some im­por­tant dishon­esty and in­effec­tive­ness go­ing on.”

Quinn: “Allegedly, ac­cord­ing to your calcu­la­tions. Any­way, say­ing that is bad, as I’ve already ar­gued.”

Carter: “Hold up! We’re in the mid­dle of eval­u­at­ing your ar­gu­ment that say­ing that is bad! You can’t use the con­clu­sion of this ar­gu­ment in the course of prov­ing it! That’s cir­cu­lar rea­son­ing!”

Quinn: “Fine. Let’s try some­thing else. You said they’re be­ing dishon­est. But, I know them, and they wouldn’t tell a lie, con­sciously, al­though it’s pos­si­ble that they might have some mo­ti­vated rea­son­ing, which is to­tally differ­ent. It’s re­ally un­civil to call them dishon­est like that. If ev­ery­one did that with the will­ing­ness you had to do so, that would lead to an all-out rhetor­i­cal war...”

Carter: “God damn it. You’re mak­ing an­other ap­peal to con­se­quences.”

Quinn: “Yes, be­cause I think ap­peals to con­se­quences are nor­ma­tive.”

Carter: “Look, at the start of this con­ver­sa­tion, your ar­gu­ment was that say­ing PADP only saved 138 pup­pies is bad.”

Quinn: “Yes.”

Carter: “And now you’re in the course of ar­gu­ing that it’s bad.”

Quinn: “Yes.”

Carter: “Whether it’s bad is a mat­ter of fact.”

Quinn: “Yes.”

Carter: “So we have to be try­ing to get the right an­swer, when we’re de­ter­min­ing whether it’s bad.”

Quinn: “Yes.”

Carter: “And, while ap­peals to con­se­quences may be de­ci­sion the­o­ret­i­cally com­pel­ling, they don’t di­rectly bear on the facts.”

Quinn: “Yes.”

Carter: “So we shouldn’t have ap­peals to con­se­quences in con­ver­sa­tions about whether the con­se­quences of say­ing some­thing is bad.”

Quinn: “Why not?”

Carter: “Be­cause we’re try­ing to get to the truth.”

Quinn: “But aren’t we also try­ing to avoid all-out rhetor­i­cal wars, and pup­pies drown­ing?”

Carter: “If we want to do those things, we have to do them by get­ting to the truth.”

Quinn: “The truth, ac­cord­ing to your opinion-

Carter: “God damn it, you just keep trol­ling me, so we never get to dis­cuss the ac­tual facts. God damn it. Fuck you.”

Quinn: “Now you’re just spout­ing in­sults. That’s re­ally ir­re­spon­si­ble, given that I just ac­cused you of do­ing some­thing bad, and caus­ing more pup­pies to drown.”

Carter: “You just keep con­trol­ling the con­ver­sa­tion by OODA loop­ing faster than me, though. I can’t re­fute your ar­gu­ment, be­cause you ap­peal to con­se­quences again in the mid­dle of the re­fu­ta­tion. And then we go an­other step down the lad­der, and never get to the truth.”

Quinn: “So what do you ex­pect me to do? Let you in­sult well-re­puted an­i­mal welfare work­ers by call­ing them dishon­est?”

Carter: “Yes! I’m mod­el­ing the PADP situ­a­tion us­ing de­ci­sion-the­o­retic mod­els, which re­quire me to rep­re­sent the knowl­edge states and op­ti­miza­tion pres­sures ex­erted by differ­ent agents (both con­scious and un­con­scious), in­clud­ing when these op­ti­miza­tion pres­sures are to­wards de­cep­tion, and even when this de­cep­tion is un­con­scious!”

Quinn: “Sounds like a bunch of nerd talk. Can you speak more plainly?”

Carter: “I’m mod­el­ing the ac­tual facts of how PADP op­er­ates and how effec­tive they are, not just how well-liked the peo­ple are.”

Quinn: “Wow, that’s a straw­man.”

Carter: “Look, how do you think ar­gu­ments are sup­posed to work, ex­actly? Who­ever is best at claiming that their op­po­nent’s ar­gu­men­ta­tion is evil wins?”

Quinn: “Sure, isn’t that the same thing as who’s mak­ing bet­ter ar­gu­ments?”

Carter: “If we ar­gue by prov­ing our state­ments are true, we reach the truth, and thereby reach the good. If we ar­gue by prov­ing each other are be­ing evil, we don’t reach the truth, nor the good.”

Quinn: “In this case, though, we’re talk­ing about drown­ing pup­pies. Surely, the good in this case is caus­ing fewer pup­pies to drown, and di­rect­ing more re­sources to the peo­ple sav­ing them.”

Carter: “That’s un­der con­tention, though! If PADP is ly­ing about how many pup­pies they’re sav­ing, they’re mak­ing the episte­mol­ogy of the puppy-sav­ing field worse, lead­ing to fewer pup­pies be­ing saved. And, they’re tak­ing money away from the next-best-look­ing char­ity, which is prob­a­bly more effec­tive if, un­like PADP, they’re not ly­ing.”

Quinn: “How do you know that, though? How do you know the money wouldn’t go to things other than sav­ing drown­ing pup­pies if it weren’t for PADP?”

Carter: “I don’t know that. My guess is that the money might go to other an­i­mal welfare char­i­ties that claim high cost-effec­tive­ness.”

Quinn: “PADP is quite effec­tive, though. Even if your calcu­la­tions are right, they save about one puppy per $10,000. That’s pretty good.”

Carter: “That’s not even that im­pres­sive, but even if their di­rect work is rel­a­tively effec­tive, they’re de­stroy­ing the episte­mol­ogy of the puppy-sav­ing field by ly­ing. So effec­tive­ness ba­si­cally caps out there in­stead of get­ting bet­ter due to bet­ter episte­mol­ogy.”

Quinn: “What an ex­ag­ger­a­tion. There are lots of other char­i­ties that have mis­lead­ing mar­ket­ing (which is to­tally not the same thing as ly­ing). PADP isn’t sin­gle­hand­edly de­stroy­ing any­thing, ex­cept in­stances of pup­pies drown­ing.”

Carter: “I’m be­gin­ning to think that the differ­ence be­tween us is that I’m anti-ly­ing, whereas you’re pro-ly­ing.”

Quinn: “Look, I’m only in fa­vor of ly­ing when it has good con­se­quences. That makes me differ­ent from pro-ly­ing scoundrels.”

Carter: “But you have re­ally sloppy rea­son­ing about whether ly­ing, in fact, has good con­se­quences. Your ar­gu­ments for do­ing so, when you lie, are made of Swiss cheese.”

Quinn: “Well, I can’t de­duc­tively prove any­thing about the real world, so I’m us­ing the most rele­vant con­sid­er­a­tions I can.”

Carter: “But you’re us­ing rea­son­ing pro­cesses that sys­tem­at­i­cally pro­tect cer­tain cached facts from up­dates, and use these cached facts to jus­tify not up­dat­ing. This was very clear when you used out­right cir­cu­lar rea­son­ing, to use the cached fact that den­i­grat­ing PADP is bad, to jus­tify ter­mi­nat­ing my ar­gu­ment that it wasn’t bad to den­i­grate them. Also, you said the PADP peo­ple were nice and hard­work­ing as a rea­son I shouldn’t ac­cuse them of dishon­esty… but, the fact that PADP saved far fewer pup­pies than they claimed ac­tu­ally casts doubt on those facts, and the rele­vance of them to PADP’s effec­tive­ness. You didn’t up­date when I first told you that fact, you in­stead started com­mit­ting rhetor­i­cal vi­o­lence against me.”

Quinn: “Hmm. Let me see if I’m get­ting this right. So, you think I have false cached facts in my mind, such as PADP be­ing a good char­ity.”

Carter: “Cor­rect.”

Quinn: “And you think those cached facts tend to pro­tect them­selves from be­ing up­dated.”

Carter: “Cor­rect.”

Quinn: “And you think they pro­tect them­selves from up­dates by gen­er­at­ing bad con­se­quences of mak­ing the up­date, such as fewer peo­ple donat­ing to PADP.”

Carter: “Cor­rect.”

Quinn: “So you want to out­law ap­peals to con­se­quences, so facts have to get ac­knowl­edged, and these self-re­in­forc­ing loops go away.”

Carter: “Cor­rect.”

Quinn: “That makes sense from your per­spec­tive. But, why should I think my be­liefs are wrong, and that I have lots of bad self-pro­tect­ing cached facts?”

Carter: “If ev­ery­one were as will­ing as you to lie, the his­tory books would be full of con­ve­nient sto­ries, the news­pa­pers would be parts of the ma­trix, the schools would be teach­ing pro­pa­ganda, and so on. You’d have no rea­son to trust your own ar­gu­ments that speak­ing the truth is bad.”

Quinn: “Well, I guess that makes sense. Even though I lie in the name of good val­ues, not ev­ery­one agrees on val­ues or be­liefs, so they’ll lie to pro­mote their own val­ues ac­cord­ing to their own be­liefs.”

Carter: “Ex­actly. So you should ex­pect that, as a re­flec­tion to your ly­ing to the world, the world lies back to you. So your head is full of lies, like the ‘PADP is effec­tive and run by good peo­ple’ one.”

Quinn: “Even if that’s true, what could I pos­si­bly do about it?”

Carter: “You could start by not mak­ing ap­peals to con­se­quences. When some­one is ar­gu­ing that a be­lief of yours is wrong, listen to the ar­gu­ment at the ob­ject level, in­stead of jump­ing to the ques­tion of whether say­ing the rele­vant ar­gu­ments out loud is a good idea, which is a much harder ques­tion.”

Quinn: “But how do I pre­vent ac­tu­ally bad con­se­quences from hap­pen­ing?”

Carter: “If your head is full of lies, you can’t re­ally trust ad-hoc ob­ject-level ar­gu­ments against speech, like ‘say­ing PADP didn’t save very many pup­pies is bad be­cause PADP is a good char­ity’. You can in­stead think about what dis­course norms lead to the truth be­ing re­vealed, and which lead to it be­ing ob­scured. We’ve seen, dur­ing this con­ver­sa­tion, that ap­peals to con­se­quences tend to ob­scure the truth. And so, if we share the goal of reach­ing the truth to­gether, we can agree not to do those.”

Quinn: “That still doesn’t an­swer my ques­tion. What about things that are ac­tu­ally bad, like pri­vacy vi­o­la­tions?”

Carter: “It does seem plau­si­ble that there should be some dis­course norms that pro­tect pri­vacy, so that some facts aren’t re­vealed, if such norms have good con­se­quences over­all. Per­haps some top­ics, such as in­di­vi­d­ual peo­ple’s sex lives, are con­sid­ered to be banned top­ics (in at least some spaces), un­less the per­son con­sents.”

Quinn: “Isn’t that an ap­peal to con­se­quences, though?”

Carter: “Not re­ally. De­cid­ing what pri­vacy norms are best re­quires think­ing about con­se­quences. But, once those norms have been de­cided on, it is no longer nec­es­sary to prove that pri­vacy vi­o­la­tions are bad dur­ing dis­cus­sions. There’s a sim­ple norm to ap­peal to, which says some things are out of bounds for dis­cus­sion. And, these ex­cep­tions can be made with­out al­low­ing ap­peals to con­se­quences in full gen­er­al­ity.”

Quinn: “Okay, so we still have some­thing like ap­peals to con­se­quences at the level of norms, but not at the level of in­di­vi­d­ual ar­gu­ments.”

Carter: “Ex­actly.”

Quinn: “Does this mean I have to say a rele­vant true fact, even if I think it’s bad to say it?”

Carter: “No. Those situ­a­tions hap­pen fre­quently, and while some rad­i­cal hon­esty prac­ti­tion­ers try not to sup­press any im­pulse to say some­thing true, this prac­tice is prob­a­bly a bad idea for a lot of peo­ple. So, of course you can eval­u­ate con­se­quences in your head be­fore de­cid­ing to say some­thing.”

Quinn: “So, in sum­mary: if we’re go­ing to have sup­pres­sion of some facts be­ing said out loud, we should have that through ei­ther clear norms de­signed with con­se­quences (in­clud­ing con­se­quences for episte­mol­ogy) in mind, or in­di­vi­d­u­als de­cid­ing not to say things, but oth­er­wise our norms should be pro­tect­ing true speech, and out­law­ing ap­peals to con­se­quences.”

Carter: “Yes, that’s ex­actly right! I’m glad we came to agree­ment on this.”