When I say that public discourse is really important, what I mean is that if you tell the truth in public about what you believe and why (possibly investing a lot of effort and using a lot of hyperlinks to bridge the inferential distance)
I want to agree with the general point here, but I find it breaking down in some of the cases I’m considering. I think the underlying generator is something like “communication is a two-way street”, and it makes sense to not just emit sentences that compile and evaluate to ‘true’ in my ontology, but that I expect to compile and evaluate to approximately what I wanted to convey in their ontology.
Does that fall into ‘harming my faction in a marketing war’ according to you?
No, I agree that authors should write in language that their audience will understand. I’m trying to make a distinction between having intent to inform (giving the audience information that they can use to think with) vs. persuasion (trying to exert control over the audience’s conclusion). Consider this generalization of a comment upthread—
Consider the idea that X implies Y. I think this is a perfectly correct point, but I’m also willing to never make it, because a lot of people will respond by concluding that not-X, because they’re emotionally attached to not-Y, and I care a lot more about people having correct beliefs about the truth value of X than Y.
This makes perfect sense as part of a consequentialist algorithm for maximizing the number of people who believe X. The algorithm works just as well, and for the same reasons whether X = “superintelligence is an existential risk” and Y = “returns from stopping global warming are smaller than you might otherwise think” (when many audience members have global warming “cause-area loyalty”), or whether X = “you should drink Coke” and Y = “returns from drinking Pepsi are smaller than you might otherwise think” (when many audience members have Pepsi brand loyalty). That’s why I want to call it a marketing algorithm—the function is to strategically route around the audience’s psychological defenses, rather than just tell them stuff as an epistemic peer.
To be clear, if you don’t think you’re talking to an epistemic peer, strategically routing around the audience’s psychological defenses might be the right thing to do! For an example that I thought was OK because I didn’t think it significantly distorted the discourse, see my recent comment explaining an editorial choice I made in a linkpost description. But I think that when one does this, it’s important to notice the nature of what one is doing (there’s a reason my linked comment uses the phrase “marketing keyword”!), and track how much of a distortion it is relative to how you would talk to an epistemic peer. As you know, quality of discourse is about the conversation executing an algorithm that reaches truth, not just convincing people of the conclusion that (you think) is correct. That’s why I’m alarmed at the prospect of someone feeling guilty (!?) that honestly reporting their actual reasoning might be “harming the discourse” (!?!?).
“Intent to inform” jives with my sense of it much more than “tell the truth.”
On reflection, I think the ‘epistemic peer’ thing is close but not entirely right. Definitely if I think Bob “can’t handle the truth” about climate change, and so I only talk about AI with Bob, then I’m deciding that Bob isn’t an epistemic peer. But if I have only a short conversation with Bob, then there’s a Gricean implication point that saying X implicitly means I thought it was more relevant to say than Y, or is complete, or so on, and so there are whole topics that might be undiscussed because I don’t want to send the implicit message that my short thoughts on the matter are complete enough to reconstruct my position or that this topic is more relevant than other topics.
---
More broadly, I note that I often see “the discourse” used as a term of derision, I think because it is (currently) something more like a marketing war than an open exchange of information. Or, like a market left to its own devices, it has Goodharted on marketing. It is unclear to me whether it’s better to abandon it (like, for example, not caring about what people think on Twitter) or attempt to recapture it (by pushing for the sorts of ‘public goods’ and savvy customers that cause markets to Goodhart less on marketing).
I want to agree with the general point here, but I find it breaking down in some of the cases I’m considering. I think the underlying generator is something like “communication is a two-way street”, and it makes sense to not just emit sentences that compile and evaluate to ‘true’ in my ontology, but that I expect to compile and evaluate to approximately what I wanted to convey in their ontology.
Does that fall into ‘harming my faction in a marketing war’ according to you?
No, I agree that authors should write in language that their audience will understand. I’m trying to make a distinction between having intent to inform (giving the audience information that they can use to think with) vs. persuasion (trying to exert control over the audience’s conclusion). Consider this generalization of a comment upthread—
This makes perfect sense as part of a consequentialist algorithm for maximizing the number of people who believe X. The algorithm works just as well, and for the same reasons whether X = “superintelligence is an existential risk” and Y = “returns from stopping global warming are smaller than you might otherwise think” (when many audience members have global warming “cause-area loyalty”), or whether X = “you should drink Coke” and Y = “returns from drinking Pepsi are smaller than you might otherwise think” (when many audience members have Pepsi brand loyalty). That’s why I want to call it a marketing algorithm—the function is to strategically route around the audience’s psychological defenses, rather than just tell them stuff as an epistemic peer.
To be clear, if you don’t think you’re talking to an epistemic peer, strategically routing around the audience’s psychological defenses might be the right thing to do! For an example that I thought was OK because I didn’t think it significantly distorted the discourse, see my recent comment explaining an editorial choice I made in a linkpost description. But I think that when one does this, it’s important to notice the nature of what one is doing (there’s a reason my linked comment uses the phrase “marketing keyword”!), and track how much of a distortion it is relative to how you would talk to an epistemic peer. As you know, quality of discourse is about the conversation executing an algorithm that reaches truth, not just convincing people of the conclusion that (you think) is correct. That’s why I’m alarmed at the prospect of someone feeling guilty (!?) that honestly reporting their actual reasoning might be “harming the discourse” (!?!?).
“Intent to inform” jives with my sense of it much more than “tell the truth.”
On reflection, I think the ‘epistemic peer’ thing is close but not entirely right. Definitely if I think Bob “can’t handle the truth” about climate change, and so I only talk about AI with Bob, then I’m deciding that Bob isn’t an epistemic peer. But if I have only a short conversation with Bob, then there’s a Gricean implication point that saying X implicitly means I thought it was more relevant to say than Y, or is complete, or so on, and so there are whole topics that might be undiscussed because I don’t want to send the implicit message that my short thoughts on the matter are complete enough to reconstruct my position or that this topic is more relevant than other topics.
---
More broadly, I note that I often see “the discourse” used as a term of derision, I think because it is (currently) something more like a marketing war than an open exchange of information. Or, like a market left to its own devices, it has Goodharted on marketing. It is unclear to me whether it’s better to abandon it (like, for example, not caring about what people think on Twitter) or attempt to recapture it (by pushing for the sorts of ‘public goods’ and savvy customers that cause markets to Goodhart less on marketing).