Appeal to Consequence, Value Tensions, And Robust Organizations

Epistemic Sta­tus: Strong opinions weakly held. Mostly try­ing to bring some things into the dis­course that I think are too of­ten ig­nored.

Some up­dates I’ve made based on the dis­cus­sion in this post are here .


Jes­si­cata’s Dialogue on Ap­peals to Con­se­quences is an ex­pan­sion of a re­sponse that she wrote to me a few months ago, ar­gu­ing a par­tic­u­lar point that I agree with: Namely, if you have an ob­ject level thing you want in the world, it’s al­most never worth ly­ing or with­hold­ing in­for­ma­tion about that thing, be­cause it breaks meta level norms about truth­seek­ing that are much more im­por­tant to ac­com­plish­ing ob­ject level goals in gen­eral. How­ever, there’s a slightly more in­ter­est­ing case that I think is quite murk­ier, that the origi­nal com­ment was point­ing to. That is, what if your truth­seek­ing norms are in ten­sion with OTHER meta level norms that are im­por­tant? In gen­eral, how do you deal with in­stances where ten­sions be­tween two im­por­tant val­ues cause you to not know what to do?


Let’s imag­ine John and Jill are dis­cussing John’s be­hav­ior in a pri­vate space. Jill is a leader of the space, and John is some­one who fre­quently at­tends the space and has lively dis­cus­sions try­ing to get to the truth.

Jill: John, I’ve had sev­eral com­plaints about your ten­dency to steer con­ver­sa­tions to­wards the di­vi­sive topic that ev­ery­one should be a Ve­gan, and I’m go­ing to ask you to tone it down a bit when you’re in our main space.

John: Are peo­ple say­ing that I’m mak­ing ar­gu­ments that are false?

Jill: No, no one is say­ing that you’re mak­ing false ar­gu­ments. John: Are peo­ple say­ing that I’m de­railing the con­ver­sa­tion? I think you’ll find that ev­ery in­stance I brought up ve­g­anism was highly rele­vant to the con­ver­sa­tion.

Jill: Yes, some peo­ple have said that, but I hap­pen to be­lieve you when you say that you’ve only brought it up in rele­vant con­texts for you.

John: Then what’s the prob­lem? I’m stat­ing rele­vant true be­liefs that add to the to­tal­ity of the con­ver­sa­tion and steer it in con­ver­sa­tion­ally rele­vant di­rec­tions.

Jill: The prob­lem is twofold. Firstly, peo­ple find it an­noy­ing to re­tread the same con­ver­sa­tion over and over. More im­por­tantly, this topic usu­ally leads to de­mon con­ver­sa­tions, and I fear that con­tinued dis­cus­sion of the topic at the rate its’ cur­rently dis­cussed could lead to a schism. Both of these out­comes go against our value of be­ing a pre­miere com­mu­nity that at­tracts the smartest peo­ple, as they’re ac­tu­ally driv­ing these peo­ple away!

John: Ex­cuse me for say­ing so, but this a clear ap­peal to con­se­quence!

Jill: Is it? I’m not say­ing that the nega­tive con­se­quences to the com­mu­nity mean that what you’re say­ing is false—that would be a clear log­i­cal fal­lacy. In­stead I’m just ask­ing you to bring up this ar­gu­ment less of­ten be­cause I think it will lead to bad out­comes.

John: Ok, maybe it’s not a log­i­cal fal­lacy, but it is dan­ger­ous. This com­mu­nity is built on a foun­da­tion of truth seek­ing, and once we start aban­don­ing that be­cause of peo­ple’s feel­ings, we de­volve into tribal dy­nam­ics and tone ar­gu­ments!

Jill: Yes, truth­seek­ing is very im­por­tant. How­ever, It’s clear that just choos­ing one value as sa­cred , and not al­low­ing for trade­offs can lead to very dys­func­tional be­lief sys­tems,.I be­lieve you’ve pointed at a clear ten­sion in our val­ues as they’re cur­rently stated. The ten­sion be­tween free­dom of speech and truth, and the value of mak­ing a space that peo­ple ac­tu­ally want to have in­tel­lec­tual dis­cus­sions at.

John: You’re say­ing there’s a ten­sion, but to me there’s a clear and ob­vi­ous win­ner. Un­der your pro­posed rules, any­one will be able to silence any­thing sim­ply by say­ing they don’t like it!

Jill: If I find some­one try­ing to silence good ar­gu­ments through that tac­tic, I’ll sit them down and have a similar con­ver­sa­tion to the one we’re hav­ing now.

John: That’s even worse! That means that in­stead of the putting the al­lowed con­ver­sa­tion top­ics up to vote, we’re putting them in the hands of one per­son, you! You can silence any con­ver­sa­tion you want.

Jill: I can see how it would seem that way, but I be­lieve we’ve cul­ti­vated some great cul­tural norms that make it harder for me to play to poli­ti­cal games like that. Firstly, our norm of rad­i­cal trans­parency means that this and all similar con­ver­sa­tions I have like this will be recorded and shared with ev­ery­one, and any such poli­ti­cal moves by me will be laugh­ably trans­par­ent.

John: That makes sense. Also, Hi Mom!

Jill: Se­cond, our or­ga­ni­za­tion al­lows any­one to ap­ply the val­ues to any­one else, so if you see ME not fol­low­ing the val­ues in any of my talks, you can call me out on it and I’ll com­ply.

John: Sure, you say that now, but be­cause of your role you can just defy that rule when­ever you want! Jill: That’s true, and it’s one of the rea­sons I’ve worked to cul­ti­vate in­tegrity as a leader. Has there been any in­stance of my be­hav­ior where you think I would ac­tu­ally do that?

John: No I sup­pose not. Are there any other cul­tural norms pre­vent­ing you from us­ing the ar­bi­trary na­ture of de­ci­sions for your own gain? Jill: There’s one more. Our or­ga­ni­za­tion has a clear set of val­ues, and as the leader one of my roles is to spear­head the change the val­ues in clear ways when there’s ten­sion be­tween them. So I’m not just go­ing to talk to you, I’m ac­tu­ally go­ing to sug­gest to the or­ga­ni­za­tion that we clar­ify our val­ues such that they tell us to do in these rel­a­tively com­mon situ­a­tions, and I’m go­ing to have you help me.

John: I think that makes sense. We can prob­a­bly make a list of top­ics that peo­ple are al­lowed to taboo, and a list of top­ics peo­ple are not al­lowed to taboo, and then I’ll always know what it’s ok to “ap­peal to con­se­quences” on. Jill: I’m afraid that par­tic­u­lar rule would be un­wise. I think there’s prac­ti­cally un­limited scis­sor state­ments that could cause schisms in our com­mu­nity, and a skil­led ad­ver­sary could eas­ily find one that’s not on our list of ap­proved top­ics. No, I’m afraid we’ll need to make a gen­eral value that can cover these situ­a­tions in the gen­eral case.

John: Oh, so try­ing to avoid ap­peals to con­se­quence ar­gu­ment can ac­tu­ally be used by some­one look­ing to harm our com­mu­nity? That’s in­ter­est­ing! But it’s not clear to me that there is a gen­eral rule that can cover all the cases.

Jill: There is. The gen­eral rule is that peo­ple should give equal weight to their own needs, the needs of the peo­ple they’re in­ter­act­ing with, and the needs of the or­ga­ni­za­tion as a whole.

John: I’m not sure I get it.

Jill: Well, you have a need to ex­press that ev­ery­one should be a ve­gan. It’s clearly very im­por­tant to you, or you wouldn’t bring it up so much. At the same time, many of the peo­ple in our com­mu­nity have a need to have va­ri­ety in their con­ver­sa­tion, and you should be aware of this when talk­ing with them. Fi­nally, our or­ga­ni­za­tion has a need to not ex­pe­rience/​dis­cuss scis­sor state­ments too of­ten or too fre­quently, in or­der to re­main healthy and avoid fre­quent schisms. By bring­ing this topic up so much, you’re putting your needs above the needs of oth­ers you’re in­ter­act­ing with and the group, in­stead of bring­ing it up less fre­quently, which would be plac­ing the needs on equal ground.

John: That makes sense. I sup­pose by the same to­ken, if there’s a re­ally in­ter­est­ing topic that’s helpful for the group to know about, and lots of peo­ple want to talk about, it would be putting your own needs above oth­ers needs if you said it hurt your feel­ings so peo­ple couldn’t talk about it.

Jill: Ex­actly!

John: So this rule seems plau­si­ble to me, and I’m sure it would be great for many peo­ple, but I have to ad­mit its’ not for me. I’d much pre­fer a space where peo­ple are al­lowed to say any­thing they want to me, and I can say any­thing I want to them in re­turn.

Jill: I agree that this may not be the best rule for ev­ery­body. That’s why next week we’re go­ing to start ex­per­i­ment­ing with The Archipelago Model. As I said, I want you to tone it down in the main room, which fol­lows the Ma­tu­rity value men­tioned above. How­ever, we’ve des­ig­nated a side room that in­stead fol­lows Crocker’s Rules. You’re al­lowed to go to ei­ther room, but when in that room, must fol­low the stated val­ues of the room. And most im­por­tantly, all con­ver­sa­tions are recorded and can be listened to by any­one in the com­mu­nity!

John: Cool, that seems worth­while, but very messy and likely to have nu­mer­ous hid­den failure modes…

Jill: I agree, but it at least seems worth a shot!


So you prob­a­bly no­ticed already, but this post wasn’t re­ally about Ap­peal to Con­se­quences at all. In­stead, it’s a med­i­ta­tion on how good or­ga­ni­za­tions deal with ten­sions in their val­ues, and avoid the or­ga­ni­za­tion be­ing over­run by skil­led so­ciopaths. A lot of these sug­ges­tions and ideas come from the work I’ve been do­ing over the past year or so to figure out what makes great or­ga­ni­za­tions and com­mu­ni­ties. I’d be par­tic­u­larly in­ter­ested in peo­ples’ in­ner sim of how the or­ga­ni­za­tion de­scribed by John and Jill above would go hor­ribly wrong, and counter ideas about what could be done to fix THOSE is­sues.