In light of your recent post on trying to establish a set of norms and guidelines for LessWrong (I think you accidentally posted it before it was finished, since some chunks of it were still missing, but it seemed to elaborate on things you put forth in stag hunt), it seems worthwhile to revisit this comment you made about a month ago that I commented on. In my comment I focused on the heat of your comment, and how that heat could lead to misunderstandings. In that context, I was worried that a more incisive critique would be counterproductive. Among other things, it would be increasing the heat in a conversation that I believed to be too heated. The other worries were that I expected that you would interpret the critique as an attack that needed defending, I intuited that you were feeling bad and that taking a very critical lens to your words would worsen your mood, and that this comment is going to take me a bunch of work (Author’s note: I’ve finished writing it. It took about 6 hours to compose, although that includes some breaks). In this comment, I’m going to provide that more incisive critique.
My goal is to engender a greater degree of empathy in you when you engage with commenters that disagree with you. This higher empathy would probably result in lower heat, which would allow you to more come closer to the truth since you would receive higher quality criticism. This is related to what habryka says here, where they say that ”...I think the outcome would have been better if you had waited to write your long comment. This comment felt like it kicked up the heat a bunch...”, and Elizabeth says here that “I expect this feeling to be common, and for that lack of feedback to be detrimental to your model building even if you start out far above average.” In order to do this, I’m going to reread your Stag Hunt post, reread the comment chain leading up to your comment, and then do a line-by-line analysis of that comment looking for violations of the guidelines to rationalist discourse that you set in Stag Hunt.
My goal is twofold: to provide evidence that you would be helped by greater empathy (and lower heat) directed towards your critics, and to echo what I see as the meat of Jennifer’s comment; that if I were to adopt the framing I see in Stag Hunt, it would be on net detrimental to the LessWrong community.
Before all that, I want to reiterate: I like the beginning of your comment. Pointing out the rock-and-a-hard-place dilemma that you feel after reading her comment is a valuable insight, but I think that for the most part your comment would be stronger without the heated line-by-line critique of her comment. She gave you that invitation to do this and so the line-by-line focus on flaws in her comment is appropriate, but the heat you brought and your apparent confidence in assessing her mental state seems unwarranted. While you did not give such permission in that comment of yours, in the post itself you said:
I’d really like it if I were embedded in a supportive ecosystem. If there were clear, immediate, and reliable incentives for doing it right, and clear, immediate, and reliable disincentives for doing it wrong. If there were actual norms (as opposed to nominal ones, norms-in-name-only) that gave me hints and guidance and encouragement. If there were dozens or even hundreds of people around, such that I could be confident that, when I lose focus for a minute, someone else will catch me.
Catch me, and set me straight.
Because I want to be set straight.
Because I actually care about what’s real, and what’s true, and what’s justified, and what’s rational, even though my brain is only kinda-sorta halfway on board, and keeps thinking that the right thing to do is Win.
Sometimes, when people catch me, I wince, and sometimes, I get grumpy, because I’m working with a pretty crappy OS, here. But I try to get past the wince as quickly as possible, and I try to say “thank you,” and I try to make it clear that I mean it, because honestly, the people that catch me are on my side. They are helping me live up to a value that I hold in my own heart, even though I don’t always succeed in embodying it.
I like it when people save me from the mistakes I listed above. I genuinely like it, even if sometimes it takes my brain a moment to catch up.
I think that Jennifer’s comment was, in part, doing this. I agree that her comment was highly flawed, and many of the critiques in your line-by-line are valid, but I expect that the net effect of your comment is to discourage both comments like hers (which it seems to me you think are a net negative contribution to the discussion), and also comments like this one. I should note here a great irony in the fact that this particular comment of yours has garnered the most analysis of this sort by me compared to any of your others. I think this is simply because I take great joy in pointing out what I see as hypocrisies, and so I would be surprised if it generalized to a similar comment to this one that was made in a different context. The rubric I’ll be using to evaluate your comments is going to be the degree to which the comment falls into the mistakes you outline in Stag Hunt:
1 Make no attempt to distinguish between what it feels is true and what is reasonable to believe. 2 Make no attempt to distinguish between what it feels is good and what is actually good. 3 Make wildly overconfident assertions that it doesn’t even believe (that it will e.g. abandon immediately if forced to make a bet). 4 Weaponize equivocation and maximize plausible deniability à la motte-and-bailey, squeezing the maximum amount of wiggle room out of words and phrases. Say things that it knows will be interpreted a certain way, while knowing that they can be defended as if they meant something more innocent. 5 Neglect the difference between what things look like and what they actually are; fail to retain any skepticism on behalf of the possibility that I might be deceived by surface resemblance. 6 Treat a 70% probability of innocence and a 30% probability of guilt as a 100% chance that the person is 30% guilty (i.e. kinda guilty). 7 Wantonly project or otherwise read into people’s actions and statements; evaluate those actions and statements by asking “what would have to be true inside my head, for me to output this behavior?” and then just assume that that’s what’s going on for them. 8 Pretend that it is speaking directly to a specific person while secretly spending the majority of its attention and optimization power on playing to some imagined larger audience. 9 Generate interventions that will make me feel better, regardless of whether or not they’ll solve the problem (and regardless of whether or not there even is a real problem to be solved, versus an ungrounded anxiety/imaginary injury).
I added the numbers because that makes them easier to reference. I am sufficiently confused by 1, 2, and 9 that I don’t think I’d be able to identify them if I saw them, so I’ll ignore those. The rest I’ll summarize in one-or-two word phrases, which will make them easier to reference throughout in a way that is more legible to readers.
3: Overconfidence 4: Motte-and-bailey 5: [blank] (In the process of making this list, I couldn’t figure out a short handle for this that wasn’t just “Overconfidence” or “Strawmanning”, although there does seem to be a difference between this and those. I’m a bit stuck and confused here, presumably I’m lacking some understanding of what this is that would let me compress it.) 6: Failure to track uncertainty. (I’m not sure if this point is intended to be an instance of the broader class of not tracking uncertainty or specific to tracking guilt). 7: Failure of empathy. 8: Playing to the crowd.
You also accuse Jennifer of strawmanning throughout, which I’ll add to the argumentative tactics that you would like pointed out to you. I take strawmanning to mean “The act of presenting a weaker version of someone’s argument to argue against. This is most noticable when paraphrasing their statement in words they would not endorse, and then putting those words in quotation marks”.
Before any analysis of your comment, I’d like to summarize Jennifer’s comment in my own words (from memory, I read her comment for the second time about 2 hours ago and I’m doing this while about 1⁄4 of the way through analyzing your comment):
You seem to be advocating for a more conflict-oriented framing of lesswrong discourse than I’m comfortable with. You keep coming back to a weed/weeding framing and a stag hunt, but I don’t think that the rate of comments that violate an unstated set of rationalist norms has a substantive impact on our ability to engage in good discussions. When you propose that weeds be pruned from our garden, I take you to mean that users who violate those norms ought to be banned, and I wonder what metric will be used to do the banning. I suspect it will be on net destructive towards the goal of a prosperous garden for rationalist discourse. Indeed, if people who violate those norms ought to be banned, I suspect that I would advocate for your banning because you do those very things. I’m being critical of your post (“pokey”), and it seems to me that you find it unpleasant. Do we really want the levels of criticality to increase?
This is presumably quite different from what she actually said, but that’s the essence of what I understood her to mean.
Anyways, enough exposition. I’ll be quoting everything you say, line by line, and doing my best to describe the degree to which it lapses into any of the fallacies outlined above. I’ll also provide running commentary to stitch everything together into a cohesive mass. Some lines won’t have any commentary, which I’ll denote with ”.”. If I interrupt a paragraph, I’ll end the quote with ”...” and begin the next quote with ”...”. I’m aiming for either dispassionate or empathetic tone throughout, wish me great skill:
If you think I’m irrational, please enumerate the ways. Please be nuanced and detailed and unconfused. List 100 little flaws if you like.
I’m having a hard time doing this because your two comments are both full of things that seem to me to be doing exactly the fog-inducing, confusion-increasing thing. But I’m also reasonably confident that my menu of options looks like:
Don’t respond, and the-audience-as-a-whole, i.e. the-culture-of-LessWrong, will largely metabolize this as tacit admission that you were right, and I was unable to muster a defense because I don’t have one that’s grounded in truth
Respond in brief, and the very culture that I’m saying currently isn’t trying to be careful with its thinking and reasoning will round-off and strawman and project onto whatever I say. This seems even likelier than usual here in this subthread, given that your first comment does this all over the place and is getting pretty highly upvoted at this point.
This makes it easier for me to model you and improves my sense of clarity surrounding the disagreement since I read it as a description of how you see yourself and how you see the disagreement between yourself and Jennifer. This is far and away my favorite part of your post.
In my view the individual points take an overly negative view of the outcomes of your potential options. If you didn’t respond, I think you are overestimating the degree to which I and other commenters will think that Jennifer is right (relative to how “right” I think she is now, having read your response several times). If you responded in brief, it’s harder for me to guess how I would view your comment because you did not respond in brief. Had you only included the part quoted above, for instance, I would have flagged Stag Hunt and Jennifer’s comments as likely rooted in an unstated disagreement about something more fundamental than what the two of you are explicitly talking about, but I wouldn’t know what it was (although it’s hard to say how much of that is my current view intruding).
Respond at length, here but not elsewhere, and try to put more data and models out there to bridge the inferential gaps (this feels doomy/useless, though, because this is a site already full of essays detailing all of the things wrong with your comments)
This comment supposes in a parenthetical that there are many things wrong with Jennifer’s comment, but has not yet fortified that claim. From a rhetorical standpoint, I see this as justifying the subsequent line-by-line analysis of Jennifer’s comment. It’s also not clear to me why the existence of essays that describe the issues with Jennifer’s comment make the citation of those essays in refuting her comment sensation-of-doom inducing. I’m guessing it’s because you believe that if an essay exists that describes the problematic outcomes of a rhetorical/argumentative device you are about to use, you should never use that device?
There might be some Overconfidence in here, since I suspect that (had people not read your comment) Jennifer’s comment would score less-than-the-mean in terms of its violation of site norms, although I don’t know how we would measure this (and therefore turn it into a bet, which would let you examine the degree to which your comment engages in Overconfidence for yourself).
Respond at length to all such comments, even though it’s easier to produce bullshit than to refute bullshit, meaning that I’m basically committing to put forth two hours of effort for every one that other people can throw at me, which is a recipe for exhaustion and demoralization and failure, and which is precisely why the OP was written. “People not doing the thing are outgunning people doing the thing, and this causes people doing the thing to give up and LessWrong becomes just a slightly less poisonous corner of a poisonous internet.”
I notice that this implies, but does not quite state, that Jennifer’s comment is bullshit.
Like, you and another user who pushed back in ways that I think are strongly contra the established virtues of rationality both put forth this unfalsifiable claim that “things just get better and better! Relax and just let the weeds and the plants duke it out, and surely the plants will win!”
Strawmanning. Jennifer’s comment seems closer to “while weeds may indeed exist, they are hard to differentiate from the plants the garden is intended to cultivate and may have no negative effects on those plants”.
Completely ignoring the assertion I made, with substantial effort and detail, that it’s bad right now, and not getting better. Refusing to engage with it at all. Refusing to grant it even the dignity of a hypothesis.
I took Jennifer’s comment as disagreeing with that state of affairs, proposing that weeds might not be easily differentiable from non-weeds, and challenging the weeding/garden framing entirely. I think that Jennifer’s comment would be stronger if she spoke to the specific instances you highlighted in the parenthetical of commenting/upvotes-gone-awry, although I should note that I found the comments that did that elsewhere somewhat confusing.
That seems bad.
And it doesn’t matter how many times I do a deep, in-depth analysis of all the ways that a bad comment was bad, because the next person posting a bad comment didn’t read it and doesn’t care, and there aren’t enough other people chiming in. I’ve answered the call that you’re making here half a dozen times, elsewhere. More than once on this very post. But that doesn’t count for anything in your book, and the audience doesn’t see it or care about it. From the audience’s perspective, you made a pretty good comment and I didn’t substantively respond, and that’s not a good look, eh?
This reads to me as a mixture of several things:
A statement about your own mind (i.e. that you feel you are losing a social war), which you are the true authority on.
A statement about the state of LessWrong norms (i.e. that you feel that LessWrong norms are bad, and that your current attempts to improve them have no impact)
A statement about me and others who are reading this exchange between you and Jennifer (that we have not noticed that Jennifer violates some discourse norms in her comment because she is upvoted: a Failure of empathy)
I also have a couple points I’d like to respond to:
When you say “I’ve answered the call that you’re making here...”, I don’t know what call you’re referencing.
You say that “there aren’t enough other people chiming in” in reference to “in-depth analysis of all the ways that a bad comment was bad”. I think I’m doing here (although I don’t endorse it phrased in those terms). I also feel discouraged w.r.t. making comments like these when I read that, although I’m not sure why. Perhaps I don’t like being told I’m on the losing side of a war. Perhaps I don’t like anticipating that this comment is futile.
I don’t want to keep falling prey to this dynamic. But here, since you asked. I don’t have what it takes to do a thorough analysis of why each of these is bad, or a link to the full-length essay outlining the rule each thing broke (because LessWrong has one in its canon in almost every case), but I’ll at least provide a short pointer.
Like… this is literally black and white thinking?
Fallacy of the grey, ironic in this case. “Black and white thinking” is not always bad or inappropriate; some things are in fact more or less binary and using the label “black and white thinking” to delegitimize something without checking to what degree it’s actually right to be thinking in binaries is disingenuous and sloppy.
And why would a good and sane person ever want
I addressed this a little in my largely-downvoted comment above, but: bad rhetoric, trying to make the idea that your opponent is good and sane seem incredulous. Trying to win the argument without actually having it. And, as I noted, implicitly conflating your inability to imagine a reason with there not being one—...
This seems like a good critique.
...having the general effect of nudging readers toward a belief that anything they don’t already see must not be real.
That isn’t the effect that her rhetoric had on me, so I disagree with you on the object level.
I also think that normatively people ought to be cautious about reasoning about the consequences that other people’s comments might have on an imagined audience, since it seems like the sort of thing that can be leveraged to disparage many comments that are on net beneficial to the platform.
Maybe your initial desires are improper?
“Maybe your initial desires are improper, but instead of saying in what way they might be improper, or trying to highlight a more proper set of desires and bridge the gap, I’m going to do the Carlson/Shapiro thing of ‘just asking a question’ and then not settling it, because I can score points with the implication and then fade into the mists. I don’t have to stick my neck out or put any skin in the game.”
Strawmanning, playing to the crowd.
Just because voting is wrong, here and there… like… so what? Some of my best comments have gotten negative votes and some of the ones I’m most ashamed of go to the top. This means that the voters are sometimes dumb. That’s OK. That’s life. Maybe educate them?
Completely ignoring an explicit, central assumption of the essay, made at length and defended in detail, about the cumulative effect of the little things. Instead of engaging with my claim that the little stuff matters, and trying to zero in on whether or not it does, and how and why, just dismissing it out of hand with a fraction of the effort put forth in the OP. Also, infuriatingly smug and dismissive with “maybe educate them?” as if I do not spend tremendous time and effort doing exactly that. While actively undermining my literal attempt to do some educating, no less. Like, what do you think this pair of posts is?
Failure of empathy. It seems to me that Jennifer’s dismissal of the importance of the relative scoring of a couple of comments stemmed from not seeing it tied to the point that the little things matter. There are 2173 words between the paragraph that begins “Yet I nevertheless feel that I encounter resistance of various forms when attempting to point at small things as if they are important...” and the paragraph in which you identify comments that had bad outcomes as measured by upvotes in your view (which begins “(I set aside a few minutes to go grab some examples...)”). That’s a fair bit of time to track that particular point. Do you expect everyone to track your arguments with that level of fidelity? Do you track others’ arguments that well? I’ll remark that I typically don’t, although I might manage to when it comes to pointing out hypocrisy because it’s something that I have a proclivity for.
I’ll also remark that I read this response as smug and dismissive, although my hypocrisy detector is rather highly tuned right now, and so I’m more likely to read hypocrisy when it isn’t present.
Lesswrong never understood this stuff, and I once thought I could/should teach it but then I just drifted away instead. I feel bad about that. Please don’t make this place worse again by caring about points for reasons other than making comments occur in the right order on the page.
“I failed at this, so I’m going to undermine other people trying to do a similar thing, and call it savviness. Also, here, have some strawmanning of your point.”
Strawmanning of the hypocritical variety.
I take Jennifer to be talking about the fact that the community does not agree with her with respect to voting norms (as measured by the behavior that she observes on LessWrong).
We don’t need to organize a stag hunt to exterminate the weeds. We need to plant good seeds and get them into the sunlight at the top of the trellis, so long as it isn’t too much work to do so. The rest might be mulch, but mulch is good too <3
Assertion with no justification and no detail and no model. Ignoring the entire claim of the OP, which is that the current thing is observably not working...
Her statement here seems to follow from her elsewhere stating that the goal of gardening is to grow the desired plants, and that weeding is largely immaterial to that goal. I agree that she has not provided a causal mechanism by which weeding, when brought back to the state of LessWrong comment culture, is immaterial to thriving plant life. However, I don’t recall you making the other argument in your OP. You gestured towards that fact and it rested as a background assumption in much of your post, but it’s not one that I remember you arguing or providing evidence for (beyond the claim that you are better than average at detecting the degree to which such things are problematic). I’m not going to re-re-read your OP to check this, but if you did make this claim I would like to hear it.
… And again, a fraction of the effort required to refute, so offering me the choice of “let the audience absorb how Jennifer just won with all these zingers, or burn two or more hours for every one she spent.”
I did not read her comment as a zinger. Also playing to the audience.
A way you could have engaged with is by explaining why adversarial attacks on the non-desired weeds would be a good use of resources rather than just… like… living and letting live, and trying to learn from things you initially can’t appreciate?
Isolated demand for rigor. Putting the burden of proof on my position instead of yours, rather than cooperatively asking hey, can we talk about where the burden of proof lies? Also ignoring the fact that I literally just wrote two essays explaining why adversarial attacks on the weeds would be a good use of resources. Instead of noting confusion about that (“I think you think you’ve made a case here, but I didn’t follow it; can you expand on X?”) just pretending like I hadn’t done the work...
Hmm, it looks like I also missed your argument in favor of the cost effectiveness of adversarial attacks on the weeds. I recall that your previous essay discussed the value of a concentration of force, which is a reason to support such attacks, but is not an argument about its cost effectiveness (you say a valuable use of resources, and I use cost effective. If there’s a material difference there, let me know).
Same thing happening with “I’m saying that your proposed rules are bad because they request expensive actions for unclear benefits that seem likely to lead to unproductive conflict if implemented… probably… but not certainly.”
Strawmanning.
...and I’d like to know what those are, how they can be detected in people or conversations or whatever??
Literally listed in the essay. Literally listed in the essay.
From memory, you listed fallacies that you yourself tended to fall into but when it came to evidence taken from other commenters it was a list of links without much context. There’s also a difference between having a list of fallacies and having a mechanism by which those fallacies can be detected and corrected. Perhaps you’re referring to the list of ideas that you list as “bad ideas” at the end, but then I’m confused about the degree to which you actually believe they’re bad ideas. If she is saying that the strategy of selecting for weeds against desirable plants is necessary before the call to action (she is saying something probably importantly different, but tracking point of views is getting exhausting), and you have preemptively agreed that you do not have a good mechanism to do this, then I don’t understand why you disagree with her disagreement here.
Perhaps you could explain “epistemic hygiene” to me in mechanistic detail, and show how I’m messing it up?
Again the trap
I feel I’ve talked about this particular phrase enough.
...”just spend lots and lots of time explaining it to me in particular, even as I gloss over and ignore the concrete bits of explanation you’ve already done?”...
Strawmanning
...Framing things such that non-response will seem like I’m being uncooperative and unreasonable, when in fact you’re just refusing to meet me halfway. And again ignoring that a bunch of this work has already been done in the essay, and a bunch of other work has already been done on LessWrong as a whole, and the central claim is “we’ve already done this work, we should stop leaving ourselves in a position to have to shore this up over and over and over again and just actually cohere some standards.”
Failure of empathy, and possibly playing to the audience (to the extent that you are accusing her of playing to the audience without outright saying it).
But anyway, I’m doing it (a little) here...
Good!
...For the hundredth time, even though it won’t actually help much and you’ll still be upvoted and I’ll still be downvoted and I’ll have to do this all over again next time and come on, I just want a place that actually cares about promoting clear thinking.
Overconfidence.
You don’t wander into a martial arts dojo, interrupt the class, and then sort-of-superciliously sneer that the martial arts dojo shouldn’t have a preference between [martial arts actions] and [everything else] and certainly shouldn’t enforce that people limit themselves to [martial arts actions] while participating in the class, that’s black-and-white thinking, just let everyone put their ideas into a free marketplace!
To the extent that you’re accusing Jennifer of sneering about you caring about rationalist discourse norms on LessWrong, this is a failure of empathy.
Well-kept gardens die by pacifism. If you don’t think that a garden being well-kept is a good thing, that’s fine. Go live in a messy garden. Don’t actively undermine someone trying to clean up a garden that’s trying to be neat.
My understanding of Jennifer’s comment is that she believes you will make the garden messier with the arguments you are putting forth in Stag Hunt.
Alternately, “we used to feel comfortable telling users that they needed to just go read the Sequences. Why did that become less fashionable, again?”
I don’t know the extent to which this is a rhetorical question, but to answer it earnestly I would expect that telling a user to read the sequences is an act that takes several orders of magnitude less effort than actually reading the sequences. I’m not confident about what the relative orders of magnitude should be between the critique-er and the critique-ee, but 1:2 (for a total of 1:10 effort) is where my intuition places the ratio. Reading a comment, deciding that it is unworthy of LessWrong discourse norms, and typing “read the sequences” is probably closer to a 1:5 ratio between the orders of magnitude of effort (i.e. it takes 100,000 times much effort to read the entirety of the sequences than it does to make such a comment).
I try to mostly make peace, because I believe conflict and “intent to harm” is very very costly.
Except that you’re actively undermining a thing which is either crucial to this site’s goals, or at least plausibly so (hence my flagging it for debate). The veneer of cooperation is not the same thing as actually not doing damage.
This read to me as Jennifer stating her desire for cooperation, which is a signal that doesn’t come free! It cost her something, at a minimum the effort to type it.
Your response reads to me as throwing that request for cooperation back in her face and using her intent to cooperate as evidence that she is somehow even less cooperative than you expected prior to this statement. It’s possible that you just intended to disagree with her on the material fact that she intends cooperation, or observing that her actions do not align with her words.
If we really need to start banning the weeds, for sure and for true… because no one can grow, and no one can be taught, and errors in rationality are terrible signs that a person is an intrinsically terrible defector… then I might propose that you be banned?
Strawmanning. Strawmanning.
I agree that the beginning of that statement is strawmanning.
The core of that statement in my eyes is the last statement; that if she agreed with the argument you put forth in stag hunt as she understands it, she would advocate for your banning.
To avoid further illusions of transparency, I’ll analyze how I would act if I based my actions on what I understand you to argue in Stag Hunt: If I were to suspend my own judgment and base my actions solely on my best attempt to interpret what you advocate for in stag hunt, I would strong downvote your comment because I see it as much much more “weed-like” than the average comment on LessWrong. It is a violation of the point of view you put forth in stag hunt because it normalizes bad forms (I suspect it succeeds despite this because it is prefaced with a valuable insight). I believe it normalizes bad forms because I see it as strawmanning, projecting statements and actions into others minds, pretending to speak to Jennifer while actually speaking mostly to the LessWrong community at large, and failing to retain skepticism that you might have deceived yourself w.r.t. the extent of Jennifer’s violations of rationalist discourse.
Instead, I weakly upvoted it because the first part of it is very useful, and responded to what I saw as the primary fault with the rest of it; that you engaged with Jennifer’s comment from a very conflict-centric point of view which led to high heat. As a result of this framing, you misunderstood most of her comment.
But I don’t think we have to fight, because I think that the world is big, everyone can learn, and the best kinds of conflicts are small, with pre-established buffering boundaries, and they end quickly, and hopefully lead to peace, mutual understanding, and greater respect afterwards.
Except that you’re actively undermining my attempt to pre-establish boundaries here. To enshrine, in a place called “LessWrong,” that the principles of reasoning and discourse promoted by LessWrong ought maybe be considered better than their opposites.
The boundaries that Jennifer is referring to here are boundaries on the extent of the conflict. What you advocate for in Stag Hunt is an expanding of those boundaries, and it was not clear to me upon reading it where those boundaries would end.
The thing I want you to learn is that proactively harming people for failing to live up to an ideal (absent bright lines and jurisprudence and a system for regulating the processes of declaring people to have done something worth punishing, and so on) is very costly, in ways that cascade and iterate, and get worse over time.
“The thing I want to do is strawman what you’re arguing for as ‘proactively harming people for failing to live up to an ideal,’ such that I can gently condescend to you about how it’s costly and cascades and leads to vaguely undefined bad outcomes. This is much easier for me to do than to lay out a model, or detail, or engage with the models and details that you went to great lengths to write up in your essays.”
While I agree that Jennifer is strawmanning here, this is the second instance of accusing Jennifer of strawmanning while strawmanning.
“I have a nuanced understanding of evil, and know it when I see it, and when I see it I weed it” is a bad plan for making the world good.
STRAWMANNING. “You said [A]. Rather than engage with [A], I’m going to pretend that you said [B] and offer up a bunch of objections to [B], skipping over the part where those objections are only relevant if, and to the degree that, [A→B], which I will not bother arguing for or even detailing in brief.”
Same as above.
The specific problem: whats the inter-rater reliability like for “decisions to weed”? I bet it is low. It is very very hard to get human inter-rater-reliability numbers above maybe 95%. How do people deal with the inevitable 1 in 20 errors? If you have fewer than 20 people, this could work, but if you have 2000 people… its a recipe for disaster.
“I bet it is low, but rather than proposing a test, I’m going to just declare it impossible on the scale of this site.”
Strawmanning. I take Jennifer as reiterating one of her central points here: if we take it as true that there are good comments and bad comments, and that we want to do something about the bad comments, then through what policy are we going to identify those bad comments (leaving aside what we then do about those bad comments)?
You had what you remarked were very bad ideas. Jennifer’s argument rests on the claim that such methods are rare, costly, or do not exist (but does not make that claim explicit).
I tried to respond to the last two paragraphs above but it was so thoroughly not even bothering to try to reach across the inferential gap or cooperate—was so thoroughly in violation of the spirit you claim to be defending, but in no way exhibit, yourself—that I couldn’t get a grip on “where to begin.”
This seems mean to me. You already don’t quote everything she says, you don’t have to remark on those last two paragraphs.
I’m not sure that going line by line was the most effective way to achieve my goals. It was costly, but I didn’t see another way to get you to internalize the fact that people are regularly taking costly measures to try to improve your model of the world, and I see you as largely ignoring them or accusing them of wrongdoing. Not all critiques of your work can be as comprehensive as mine is here, since as you pointed out, “it’s easier to produce bullshit than to refute bullshit” (I granted myself this one zinger as motivation for finishing this comment, if others remain in the text they are not intended).
Meta-question: Is this the sort of thing that’s appropriate to post as a top-level post? It seems fairly specific, but I worked hard on it and I imagine it as encapsulating the virtues that you put forth in Stag Hunt and your hopefully-soon-to-be-posted guidelines for rationalist discourse.
Edited for clarity on the 1:5 point and a few typos.
In light of your recent post on trying to establish a set of norms and guidelines for LessWrong (I think you accidentally posted it before it was finished, since some chunks of it were still missing, but it seemed to elaborate on things you put forth in stag hunt), it seems worthwhile to revisit this comment you made about a month ago that I commented on. In my comment I focused on the heat of your comment, and how that heat could lead to misunderstandings. In that context, I was worried that a more incisive critique would be counterproductive. Among other things, it would be increasing the heat in a conversation that I believed to be too heated. The other worries were that I expected that you would interpret the critique as an attack that needed defending, I intuited that you were feeling bad and that taking a very critical lens to your words would worsen your mood, and that this comment is going to take me a bunch of work (Author’s note: I’ve finished writing it. It took about 6 hours to compose, although that includes some breaks). In this comment, I’m going to provide that more incisive critique.
My goal is to engender a greater degree of empathy in you when you engage with commenters that disagree with you. This higher empathy would probably result in lower heat, which would allow you to more come closer to the truth since you would receive higher quality criticism. This is related to what habryka says here, where they say that ”...I think the outcome would have been better if you had waited to write your long comment. This comment felt like it kicked up the heat a bunch...”, and Elizabeth says here that “I expect this feeling to be common, and for that lack of feedback to be detrimental to your model building even if you start out far above average.” In order to do this, I’m going to reread your Stag Hunt post, reread the comment chain leading up to your comment, and then do a line-by-line analysis of that comment looking for violations of the guidelines to rationalist discourse that you set in Stag Hunt.
My goal is twofold: to provide evidence that you would be helped by greater empathy (and lower heat) directed towards your critics, and to echo what I see as the meat of Jennifer’s comment; that if I were to adopt the framing I see in Stag Hunt, it would be on net detrimental to the LessWrong community.
Before all that, I want to reiterate: I like the beginning of your comment. Pointing out the rock-and-a-hard-place dilemma that you feel after reading her comment is a valuable insight, but I think that for the most part your comment would be stronger without the heated line-by-line critique of her comment. She gave you that invitation to do this and so the line-by-line focus on flaws in her comment is appropriate, but the heat you brought and your apparent confidence in assessing her mental state seems unwarranted. While you did not give such permission in that comment of yours, in the post itself you said:
I think that Jennifer’s comment was, in part, doing this. I agree that her comment was highly flawed, and many of the critiques in your line-by-line are valid, but I expect that the net effect of your comment is to discourage both comments like hers (which it seems to me you think are a net negative contribution to the discussion), and also comments like this one. I should note here a great irony in the fact that this particular comment of yours has garnered the most analysis of this sort by me compared to any of your others. I think this is simply because I take great joy in pointing out what I see as hypocrisies, and so I would be surprised if it generalized to a similar comment to this one that was made in a different context. The rubric I’ll be using to evaluate your comments is going to be the degree to which the comment falls into the mistakes you outline in Stag Hunt:
I added the numbers because that makes them easier to reference. I am sufficiently confused by 1, 2, and 9 that I don’t think I’d be able to identify them if I saw them, so I’ll ignore those. The rest I’ll summarize in one-or-two word phrases, which will make them easier to reference throughout in a way that is more legible to readers.
3: Overconfidence
4: Motte-and-bailey
5: [blank] (In the process of making this list, I couldn’t figure out a short handle for this that wasn’t just “Overconfidence” or “Strawmanning”, although there does seem to be a difference between this and those. I’m a bit stuck and confused here, presumably I’m lacking some understanding of what this is that would let me compress it.)
6: Failure to track uncertainty. (I’m not sure if this point is intended to be an instance of the broader class of not tracking uncertainty or specific to tracking guilt).
7: Failure of empathy.
8: Playing to the crowd.
You also accuse Jennifer of strawmanning throughout, which I’ll add to the argumentative tactics that you would like pointed out to you. I take strawmanning to mean “The act of presenting a weaker version of someone’s argument to argue against. This is most noticable when paraphrasing their statement in words they would not endorse, and then putting those words in quotation marks”.
Before any analysis of your comment, I’d like to summarize Jennifer’s comment in my own words (from memory, I read her comment for the second time about 2 hours ago and I’m doing this while about 1⁄4 of the way through analyzing your comment):
This is presumably quite different from what she actually said, but that’s the essence of what I understood her to mean.
Anyways, enough exposition. I’ll be quoting everything you say, line by line, and doing my best to describe the degree to which it lapses into any of the fallacies outlined above. I’ll also provide running commentary to stitch everything together into a cohesive mass. Some lines won’t have any commentary, which I’ll denote with ”.”. If I interrupt a paragraph, I’ll end the quote with ”...” and begin the next quote with ”...”. I’m aiming for either dispassionate or empathetic tone throughout, wish me great skill:
This makes it easier for me to model you and improves my sense of clarity surrounding the disagreement since I read it as a description of how you see yourself and how you see the disagreement between yourself and Jennifer. This is far and away my favorite part of your post.
In my view the individual points take an overly negative view of the outcomes of your potential options. If you didn’t respond, I think you are overestimating the degree to which I and other commenters will think that Jennifer is right (relative to how “right” I think she is now, having read your response several times). If you responded in brief, it’s harder for me to guess how I would view your comment because you did not respond in brief. Had you only included the part quoted above, for instance, I would have flagged Stag Hunt and Jennifer’s comments as likely rooted in an unstated disagreement about something more fundamental than what the two of you are explicitly talking about, but I wouldn’t know what it was (although it’s hard to say how much of that is my current view intruding).
This comment supposes in a parenthetical that there are many things wrong with Jennifer’s comment, but has not yet fortified that claim. From a rhetorical standpoint, I see this as justifying the subsequent line-by-line analysis of Jennifer’s comment. It’s also not clear to me why the existence of essays that describe the issues with Jennifer’s comment make the citation of those essays in refuting her comment sensation-of-doom inducing. I’m guessing it’s because you believe that if an essay exists that describes the problematic outcomes of a rhetorical/argumentative device you are about to use, you should never use that device?
There might be some Overconfidence in here, since I suspect that (had people not read your comment) Jennifer’s comment would score less-than-the-mean in terms of its violation of site norms, although I don’t know how we would measure this (and therefore turn it into a bet, which would let you examine the degree to which your comment engages in Overconfidence for yourself).
I notice that this implies, but does not quite state, that Jennifer’s comment is bullshit.
Strawmanning. Jennifer’s comment seems closer to “while weeds may indeed exist, they are hard to differentiate from the plants the garden is intended to cultivate and may have no negative effects on those plants”.
I took Jennifer’s comment as disagreeing with that state of affairs, proposing that weeds might not be easily differentiable from non-weeds, and challenging the weeding/garden framing entirely. I think that Jennifer’s comment would be stronger if she spoke to the specific instances you highlighted in the parenthetical of commenting/upvotes-gone-awry, although I should note that I found the comments that did that elsewhere somewhat confusing.
This reads to me as a mixture of several things:
A statement about your own mind (i.e. that you feel you are losing a social war), which you are the true authority on.
A statement about the state of LessWrong norms (i.e. that you feel that LessWrong norms are bad, and that your current attempts to improve them have no impact)
A statement about me and others who are reading this exchange between you and Jennifer (that we have not noticed that Jennifer violates some discourse norms in her comment because she is upvoted: a Failure of empathy)
I also have a couple points I’d like to respond to:
When you say “I’ve answered the call that you’re making here...”, I don’t know what call you’re referencing.
You say that “there aren’t enough other people chiming in” in reference to “in-depth analysis of all the ways that a bad comment was bad”. I think I’m doing here (although I don’t endorse it phrased in those terms). I also feel discouraged w.r.t. making comments like these when I read that, although I’m not sure why. Perhaps I don’t like being told I’m on the losing side of a war. Perhaps I don’t like anticipating that this comment is futile.
This seems like a good critique.
That isn’t the effect that her rhetoric had on me, so I disagree with you on the object level.
I also think that normatively people ought to be cautious about reasoning about the consequences that other people’s comments might have on an imagined audience, since it seems like the sort of thing that can be leveraged to disparage many comments that are on net beneficial to the platform.
Strawmanning, playing to the crowd.
Failure of empathy. It seems to me that Jennifer’s dismissal of the importance of the relative scoring of a couple of comments stemmed from not seeing it tied to the point that the little things matter. There are 2173 words between the paragraph that begins “Yet I nevertheless feel that I encounter resistance of various forms when attempting to point at small things as if they are important...” and the paragraph in which you identify comments that had bad outcomes as measured by upvotes in your view (which begins “(I set aside a few minutes to go grab some examples...)”). That’s a fair bit of time to track that particular point. Do you expect everyone to track your arguments with that level of fidelity? Do you track others’ arguments that well? I’ll remark that I typically don’t, although I might manage to when it comes to pointing out hypocrisy because it’s something that I have a proclivity for.
I’ll also remark that I read this response as smug and dismissive, although my hypocrisy detector is rather highly tuned right now, and so I’m more likely to read hypocrisy when it isn’t present.
Strawmanning of the hypocritical variety.
I take Jennifer to be talking about the fact that the community does not agree with her with respect to voting norms (as measured by the behavior that she observes on LessWrong).
Her statement here seems to follow from her elsewhere stating that the goal of gardening is to grow the desired plants, and that weeding is largely immaterial to that goal. I agree that she has not provided a causal mechanism by which weeding, when brought back to the state of LessWrong comment culture, is immaterial to thriving plant life. However, I don’t recall you making the other argument in your OP. You gestured towards that fact and it rested as a background assumption in much of your post, but it’s not one that I remember you arguing or providing evidence for (beyond the claim that you are better than average at detecting the degree to which such things are problematic). I’m not going to re-re-read your OP to check this, but if you did make this claim I would like to hear it.
I did not read her comment as a zinger. Also playing to the audience.
Hmm, it looks like I also missed your argument in favor of the cost effectiveness of adversarial attacks on the weeds. I recall that your previous essay discussed the value of a concentration of force, which is a reason to support such attacks, but is not an argument about its cost effectiveness (you say a valuable use of resources, and I use cost effective. If there’s a material difference there, let me know).
Strawmanning.
From memory, you listed fallacies that you yourself tended to fall into but when it came to evidence taken from other commenters it was a list of links without much context. There’s also a difference between having a list of fallacies and having a mechanism by which those fallacies can be detected and corrected. Perhaps you’re referring to the list of ideas that you list as “bad ideas” at the end, but then I’m confused about the degree to which you actually believe they’re bad ideas. If she is saying that the strategy of selecting for weeds against desirable plants is necessary before the call to action (she is saying something probably importantly different, but tracking point of views is getting exhausting), and you have preemptively agreed that you do not have a good mechanism to do this, then I don’t understand why you disagree with her disagreement here.
I feel I’ve talked about this particular phrase enough.
Strawmanning
Failure of empathy, and possibly playing to the audience (to the extent that you are accusing her of playing to the audience without outright saying it).
Good!
Overconfidence.
To the extent that you’re accusing Jennifer of sneering about you caring about rationalist discourse norms on LessWrong, this is a failure of empathy.
My understanding of Jennifer’s comment is that she believes you will make the garden messier with the arguments you are putting forth in Stag Hunt.
I don’t know the extent to which this is a rhetorical question, but to answer it earnestly I would expect that telling a user to read the sequences is an act that takes several orders of magnitude less effort than actually reading the sequences. I’m not confident about what the relative orders of magnitude should be between the critique-er and the critique-ee, but 1:2 (for a total of 1:10 effort) is where my intuition places the ratio. Reading a comment, deciding that it is unworthy of LessWrong discourse norms, and typing “read the sequences” is probably closer to a 1:5 ratio between the orders of magnitude of effort (i.e. it takes 100,000 times much effort to read the entirety of the sequences than it does to make such a comment).
This read to me as Jennifer stating her desire for cooperation, which is a signal that doesn’t come free! It cost her something, at a minimum the effort to type it.
Your response reads to me as throwing that request for cooperation back in her face and using her intent to cooperate as evidence that she is somehow even less cooperative than you expected prior to this statement. It’s possible that you just intended to disagree with her on the material fact that she intends cooperation, or observing that her actions do not align with her words.
I agree that the beginning of that statement is strawmanning.
The core of that statement in my eyes is the last statement; that if she agreed with the argument you put forth in stag hunt as she understands it, she would advocate for your banning.
To avoid further illusions of transparency, I’ll analyze how I would act if I based my actions on what I understand you to argue in Stag Hunt: If I were to suspend my own judgment and base my actions solely on my best attempt to interpret what you advocate for in stag hunt, I would strong downvote your comment because I see it as much much more “weed-like” than the average comment on LessWrong. It is a violation of the point of view you put forth in stag hunt because it normalizes bad forms (I suspect it succeeds despite this because it is prefaced with a valuable insight). I believe it normalizes bad forms because I see it as strawmanning, projecting statements and actions into others minds, pretending to speak to Jennifer while actually speaking mostly to the LessWrong community at large, and failing to retain skepticism that you might have deceived yourself w.r.t. the extent of Jennifer’s violations of rationalist discourse.
Instead, I weakly upvoted it because the first part of it is very useful, and responded to what I saw as the primary fault with the rest of it; that you engaged with Jennifer’s comment from a very conflict-centric point of view which led to high heat. As a result of this framing, you misunderstood most of her comment.
The boundaries that Jennifer is referring to here are boundaries on the extent of the conflict. What you advocate for in Stag Hunt is an expanding of those boundaries, and it was not clear to me upon reading it where those boundaries would end.
While I agree that Jennifer is strawmanning here, this is the second instance of accusing Jennifer of strawmanning while strawmanning.
Same as above.
Strawmanning. I take Jennifer as reiterating one of her central points here: if we take it as true that there are good comments and bad comments, and that we want to do something about the bad comments, then through what policy are we going to identify those bad comments (leaving aside what we then do about those bad comments)?
You had what you remarked were very bad ideas. Jennifer’s argument rests on the claim that such methods are rare, costly, or do not exist (but does not make that claim explicit).
This seems mean to me. You already don’t quote everything she says, you don’t have to remark on those last two paragraphs.
I’m not sure that going line by line was the most effective way to achieve my goals. It was costly, but I didn’t see another way to get you to internalize the fact that people are regularly taking costly measures to try to improve your model of the world, and I see you as largely ignoring them or accusing them of wrongdoing. Not all critiques of your work can be as comprehensive as mine is here, since as you pointed out, “it’s easier to produce bullshit than to refute bullshit” (I granted myself this one zinger as motivation for finishing this comment, if others remain in the text they are not intended).
Meta-question: Is this the sort of thing that’s appropriate to post as a top-level post? It seems fairly specific, but I worked hard on it and I imagine it as encapsulating the virtues that you put forth in Stag Hunt and your hopefully-soon-to-be-posted guidelines for rationalist discourse.
Edited for clarity on the 1:5 point and a few typos.