Responding to the section about my argument (and a few related parts):
You title it “The Founding Values Have Not Been Refuted.” But I never set out to refute the founding values! We prioritize the founding values a bit differently; perhaps from your perspective it looks like I’m dethroning the central one. But it really seems relevant to me that I am the one arguing Eliezer’s position from 13 years ago. To summarize my thoughts on Said and your arguments here, I might as well quote him from earlier, “If you fail to achieve a correct answer, it is futile to protest that you acted with propriety.”
Vaniver then claims that the new culture is the more sophisticated one (analogous to FDT), but the justification he offers is fundamentally lacking if not absent.
I don’t justify it in that comment; the justifications are spread across many comments and many posts and several authors. I would attempt to justify it here, but on reflection I think that’s putting the cart in front of the horse. Cultures are package deals which have many impacts, both positive and negative; no real culture is born from picking a justification and then optimizing purely for that. They are grown, negotiated, and cultivated across many people and many actions.
To continue hammering on my example of the Royal Society, the thing that they wanted to do was understand the world better. They succeeded much more than others before them. Why? Maybe they had more geniuses; maybe they had more wealth; maybe they had better social technology. It seems to me like one of the contributors here is that they believed that social graces were important, and that managing the feelings of the participants was important. Not the most important thing—they weren’t a social club first and foremost—but not banned from consideration, a sort of anti-trump that loses every competition it enters. They wanted people to stay engaged in the project, to keep supplying data and attending meetings, and they thought seriously about how.
I am using it to answer the question: “which of these views incorporates the other?”. I think it’s a valid tool for that.
Now, is that question relevant? I was replying to parts of Said’s comment where he objected to habryka’s framing of the ban post:
This phrasing assumes that there’s something to “understand” (and which I do not understand), and something which I should wish to “learn” (and which I have failed, or have not tried, to learn). This, of course, begs the question. The unambiguous reality is that I have disagreements with the LW moderation team about various things (including, as is critical here, various questions about what are proper rules, norms, and practices for a discussion forum like this one).
...
But by describing the situation as one in which he has some (presumptively correct) understanding, which remains only for him to impart to me, and some (presumptively useful) skill, which remains only for me to learn, @habryka attempts to sidestep the need to make his case.
I agreed that it was a disagreement about principles, and that it wouldn’t be correct to just say “we’re more sophisticated, therefore we’re right.” The way I’d put it today is that including all the same principles in your calculation and more doesn’t give you a strategy-stealing argument in the way that including all the same information and more does, because it doesn’t mean you are setting the prices correctly. Said and Zack appear to think some things should have 0 cost, and habryka and I think those things have a cost that cannot be ignored; the question is about what price the culture should put on them.
I think this does point to what I find frustrating about this conversation, however; I think we never managed to find cruxes because you thought our charges were inadmissible, and rounded them into something different and more base. (We’ll return to this point.)
I have to question whether Vaniver read past the title of “Critic Contributions Are Logically Irrelevant”
So far, I’ve read everything you’ve written on LW moderation issues, because I view it as my responsibility to.
I think I adequately handled the case of objections not intended as logical criticisms in the final section
I don’t think so. Or, that is, I think your post spends nearly two thousand words to investigate a simple puzzle, badly. habryka and Duncan complain about the hedonic and reputational effects of a style of behavior. “But what does this have to do with correctness?” you ask, and spend 1300 words demonstrating that, as the alert reader might have anticipated immediately, it is not about putting correctness before all other concerns. Therefore it must be about something else—perhaps, for example, ad revenue.
In a fit of Said-like politeness, I comment to ask why you don’t view comments as speech acts, a category which naturally resolves the puzzle. (Said replies that it’s not unreasonable to, and that most comments are primarily about the information content—which, while true, doesn’t obviate that we’re talking about the cases where it’s not true.)
This time, thankfully, you cut to the chase:
The relevance of people having objections about commenters that don’t make sense as logical criticisms is that if the objectors have their way, that establishes a precedent of the forum being a place where logical criticism is dismissed in favor of other concerns, which might be desirable for a social or hobby group whose only mission is for its members to have a fun time, but is contrary to the mission of an intellectual forum like Less Wrong purports to be.
We were open about changing the culture and the discourse norms at the very beginning! One of the goals of LW 2.0 has always been to make posting fun again.
I’m not sure how to say this more clearly, but we disagree about how to accomplish the mission of an intellectual forum like Less Wrong purports to be. It’s the same disagreement as the interpretation of the Royal Society or Socrates. We think it advances instead of betrays our Sacred Mission to enforce a minimum standard of social graces. It is easier for you to pretend that this is about us seeking ad revenue, apparently.
I don’t think your values could build Less Wrong. (Obviously Said can build websites, and has built other forums.) You’re welcome to try, and to put your ideas into practice, and see what comes of them.[1]
Vaniver’s suggestion that I reread “Feeling Rational” is a non sequitur.
The sequitur is that in the post EY argues that a popular belief about rationality is that rationality opposes all emotion, but this isn’t how probability theory works, and instead the question is whether or not emotions are concordant with reality. Rational emotions are downstream of models that correspond to reality; irrational emotions are downstream of models that don’t correspond to reality. When someone finds one of Achmiz’s comments hurtful, is it rational for them to do so? Well, that depends on whether or not the models underlying their hurt correspond to reality.
[Incidentally, one of the reasons why I think this isn’t “the weaponization of emotional harm” is that we’re not just measuring emotional outbursts or tears or whatever, but instead trying to figure out how it connects to our goals for LessWrong, and sometimes the hurt seems real and relevant and sometimes it fails on either count.]
A site dedicated to advancing the art of rationality shouldn’t go out of its way to silence people with high values of the trait in order to accommodate people with low values of the trait.
I think you’re mistaking defense and offense. I’m not the one being silenced here, and it’s not Said’s fortitude we’re complaining about.
That is, yes, fortitude is a virtue, as is readiness of comprehension. But it is likewise a virtue to be smooth instead of rough, and clear instead of confusing. Someone’s whose posts are confusing and incoherent may find themselves managed out—either by the karma system, or getting rate-limited, or not accepted as a new user—even if they readily understand all the posts and comments other people write.
Is it a good thing that Said requires more fortitude in those around him than the typical poster? You could imagine a school having an evil potions master in order to cause its students to develop a certain kind of resilience, but that’s not what’s going on here. Instead we had years of asking and telling Said to be easier to deal with, and eventually told him that if he was going to keep it up, he would have to do it elsewhere.
But the mere observation that present-day Vaniver possesses “the concept of emotional tallness” doesn’t explain what the younger Vaniver was allegedly wrong about!
Younger Vaniver was using his personal experience—and the expectations downstream primarily of that, and advice he should have reversed—rather than empiricism, such as by polling others or doing interviews. He wasn’t attuned to the questions of what is good for the site overall, rather focused on the momentary turn-by-turn of the conversation. “Wait, someone in a position of power doing something because it’s fun or unfun?” he said with a look of disapproval, and was not tracking that how much and what Eliezer was posting was downstream of what sort of an environment LessWrong was, and that it was smoother and more rational to work inside of those dynamics rather than attempt to force them into a shape they were not, and that commenters were being shaped by the culture that they were shaping.
Now, is this being corrupted by social pressure? I mean, you could say that, in the same way that a lot of my hygiene habits are the result of social pressure exerted on me, but that doesn’t mean it’s corruption that I shower regularly. There is an actual tension between being able to work with anyone and cultivating a circle of contacts where people can trust that someone being your friend means they’ll likely be their friend too.
As a grown-up on an intellectual discussion forum, it’s not other people’s job to manage your feelings.
Note this is a one-way arrow. I don’t think I ever viewed it as other people’s job to manage my feelings, but I did think I had some duty to other people to manage their feelings. (Otherwise, why be polite?) It was a limited duty—their feelings could be disproportional—but the fundamental question was not whether I was justified in losing but whether I got what I wanted.
That is, in the Sequences era, it was understood that even if you don’t have any persistent and demanding critics in real life, you should try to simulate them.
In the Sequences era, it was also understood why our kind can’t cooperate. We have been trying to fix that, with our new culture.
To be clear, I don’t think we could build Less Wrong either; I think EY deserves that credit. I don’t think we could even have built LW 2.0 without the inheritance of the URL and The Sequences.
But it really seems relevant to me that I am the one arguing Eliezer’s position from 13 years ago.
To be clear, I did say “Founding Values” and not “Founders”. Yudkowsky from twenty-three years ago put up a page about Crocker’s rules. (To preëmpt the obvious knee-jerk objection: yes, I did read the part about “Crocker’s Rules does not mean you can insult people.”)
The reason I’m considering the 2002 page about Crocker’s rules but not the 2013 comment about hedonics to be part of the founding ideal, is because to me, the founding ideal isn’t about this Eliza Yudowski person.
It’s about the philosophical vision articulated in “The Bottom Line”, “A Rational Argument”, “What Is Evidence?”, the “Technical Explanation”, and the “Twelve Virtues”—about seeking the mathematical laws that govern how a small part of the world (a “map”) can function as a predictive model of the rest (the “territory”), and how conditional predictions can be used to steer the world.
Crocker’s rules is obviously consilient with the philosophical vision. “Anyone is allowed to call you a moron and claim to be doing you a favor,” “[w]hich, in point of fact, they would be,” says the page. (You see, because having an accurate map is instrumentally convergent for rational agents: if I were an idiot, I would want to know about it, because that might have decision-relevant consequences.)
Of course, humans can’t embody the ideal, but it’s important that there is an ideal, and claims that humans shouldn’t even try to better approximate the ideal because it’s supposedly impossible don’t comport with the philosophical vision I learned from the texts. (As it is written, “The ninth virtue is perfectionism. [...] In every art, if you do not seek perfection you will halt before taking your first steps. If perfection is impossible that is no excuse for not trying.”) If later statements by this Yudowski person contradict the philosophical ideal, I go with the ideal, not the person, because that’s what the texts taught me to do.
It seems to me like one of the contributors here is that they believed that social graces were important, and that managing the feelings of the participants was important.
Unfortunately, the reason I haven’t been able to engage with this intuitively-surprising-to-me claim much is because I’m not a specialist in 17th Century English history, and so I’m not in a good position to evaluate the claim without a lot of catch-up labor.
Said and Zack appear to think some things should have 0 cost, and habryka and I think those things have a cost that cannot be ignored; the question is about what price the culture should put on them.
So it’s not that the cost should be literally zero. (I agree that Crocker’s rules doesn’t mean you get to insult people; Said believes in his version of politeness.) It’s that there is a normative ideal about information processing, and you want a culture that socializes the humans into aspiring towards the normative ideal even when it’s hard (Bayesian reasoners wouldn’t hide from information, I want to be more like that), and the “emotional tallness” excuse doesn’t even pay lip service to the ideal.
as the alert reader might have anticipated immediately, it is not about putting correctness before all other concerns
You say that so casually! Okay, sure, humans can’t put correctness before all other concerns. But that’s, like, non-normative, right? I’m more optimistic about a culture that encourages striving for the ideal in a non-formalized way, than a pseudo-economic culture that talks a good game about “prices” (which, according to the microeconomic theory, should theoretically exist), but which, in practice, I think psychologically functions as an excuse to not even try. Is that a better crux?
perhaps, for example, ad revenue.
I should confess that that was not my finest work, rhetoric-wise. (I was in a rush that week to ship as many posts as I could before Habryka got the draw on me, posting on the 14th, 16th, 17th, and 20th, and quality may have suffered somewhat.) Hopefully my comments here about the importance of normative ideals in culture is clearer.
we disagree about how to accomplish the mission of an intellectual forum like Less Wrong purports to be
was not tracking that how much and what Eliezer was posting was downstream of what sort of an environment LessWrong was, and that it was smoother and more rational to work inside of those dynamics
I’d expect you’d agree that focusing on Yudkowsky feeling comfortable with posting is Goodhartable. We likely disagree on how Goodharted it is in practice. (As I’ve written about at length elsewhere, I think a lot of Yudkowsky’s work since at least 2016 is just not very good for reasons that have to do with him giving up on the normative ideal.)
I’ve attempted to read primary sources from around the origins of Science (quite a bit, though still less than I’d like and also it’s been 20 years), and my own impression is that Science was enabled partly by a designed shift in social graces (by humans who thought social graces matter), but, not a content-neutral sort of advance in social graces (nothing like “this way we can create more harmony in arbitrary situations”), but an engineered change in social etiquette aimed specifically at making it easy for gentlemen who cared about their reputations, and cared also about not being challenged to duels etc., to be able to verify one anothers’ experiments. Like, “we have a norm around here in which everyone aspires to verify everything for themselves, and so if they ask you how you did your experiment, and share their own attempted replications and results, they won’t be doubting your honor, they’ll instead be practicing our local virtues. Also, everyone will keep their speech strictly about what happened when they tried different experiments, and will not e.g. accuse one another of lying—they’ll only say that their own attempt yielded a different result.”
I do think it was pretty darn different from Crocker’s Rules.
The reason it seems like an intuitively surprising claim to me is because—as a non-specialist, I thought the standard explanation for the Royal Society’s success was, um, Science?
This explanation is on the wrong level of abstraction. From my point of view, the whole question is “How did this group of humans manage to do science together?” It wasn’t by embroidering “Science!” onto their banners; there were lots of details to their organization.
(And the practice of science has changed substantially, over the years, such that even a practicing scientist today might be far out of touch with the underpinnings of the true science. Comparing organizations, comparing eras, asking which is better—this takes scholarship and empiricism to figure out.)
Like, on the simplest level, Nullius in Verbia cannot be the basis of science because of limited human observational capacity. (Do you think the Earth has magma inside it? Is that because you checked, or because someone told you?) The actual basis of science has to be something more like “take people’s word only on clearly laid-out chains of checkable inferences and observations, and careful observation of their character”.[1]
Crocker’s rules is obviously consilient with the philosophical vision. “Anyone is allowed to call you a moron and claim to be doing you a favor,” “[w]hich, in point of fact, they would be,” says the page. (You see, because having an accurate map is instrumentally convergent for rational agents: if I were an idiot, I would want to know about it, because that might have decision-relevant consequences.)
Crocker’s rules erode the foundation on which they rest, in a way that I suspect Eliezer learned by experience. If it is costly to call you a moron, then people deciding whether or not to do it will have to determine whether or not it’s worth making the bet, and to the extent they have skill at telling whether or not you’re a moron, they’ll only call you out when it’s correct. But if it’s too costly, you miss out on corrections you would’ve want to have; the only way to get all of those is by making it free, and thus all bets worthwhile.
But all bets? The only reason to tell an unknown they’re a moron is because they have made a mistake; there are many reasons to call a celebrity a moron which do not depend on the celebrity having made a mistake. Crocker’s Rules liberates the autistic, but it also attracts the cruel and meanspirited to you, the sneerers who delight in you having abandoned a sort of social self-defense. At some point the signal from criticism drops into the negative, and you find yourself missing out again, because it is no longer worthwhile to pay attention to the pile of messages.
[This is why, if you do follow Crocker’s Rules, it primarily makes sense to do so for private communications; there’s less reward for sending someone hatemail than there is for writing hateful comments.]
That is--
Of course, humans can’t embody the ideal, but it’s important that there is an ideal, and claims that humans shouldn’t even try to better approximate the ideal because it’s supposedly impossible don’t comport with the philosophical vision I learned from the texts.
I think it would be different if we were saying “look, Said hurts our feelings and that’s not where we’d like to spend our limited pain tolerance.”[2] Instead we’re saying things like the preceding sections, that “yeah I can see why people might think that Crocker’s Rules make sense, but actually we think the society where people expect that other people are (or should be) following Crocker’s Rules is worse than one where they don’t.”
That is, the ideal is actually different based on your goals and situation. The Tao of a lone scholar is not the same as the Tao of a participant in a university or a scene; arguments that you receive a benefit in some situation from following a policy do not constitute a complete argument for that policy.
You say that so casually! Okay, sure, humans can’t put correctness before all other concerns. But that’s, like, non-normative, right?
I think the thing that often is going on is that someone will complain that Said is annoying, or that they didn’t like his speech act along some layer other than correctness. This will then be interpreted by you or Said as being about correctness—the deflection of legitimate concerns rather than addressing them—which will then be viewed by the someone as deflection of their legitimate concerns (about something other than correctness). Unsurprisingly, little progress is made from that starting point.
To return to the point of instrumental convergence—many things are instrumentally convergent for rational agents, like continued survival, or other agents thinking highly of them, and so on. A thing that we need for a proper intellectual scene is more like sportsmanship, in that people want the truth to win socially, or are identifying with the sport instead of with their team. A professor who suppresses papers critical of his contribution to an academic field is not being irrational—he is pursuing a plan that achieves his values based on an accurate understanding of the world—he is instead being evil or anti-social, in that he’s putting his wants above humanity’s shared understanding; we wish he was operating on different values.
I think this is the layer where you need to make your case—that Said was pushing people towards the right sort of sportsmanship—and I think this was not true on net.[3]
[Edit] On reflection I think this section might be confusing and I should try to lay out my reasoning more clearly. I think one of the arguments for Said being fine is that it is the user’s job to have an emotional orientation where they like Said’s style of criticism, and that’s because that emotional orientation is both good for them personally and also good for society. I buy the argument that the emotional orientation has benefits for society—for example, I think it’s better if the professor has it—but I think the question is “which environment causes there to be more of that orientation?” and I think your plan is “require it and let them sort out how it’s produced” in a way that I think is trying to simplify the world so that you neither need to figure out whether your plan is efficacious or put in any of the work that would cause the system to be better.
Like, I’m sticking my neck out here on the Royal Society analogy, and doing it primarily on the strength of one historian. If it turns out that my understanding of that historian’s work is confused, or the work is discredited, so too my point, and this is bad for my overall credibility.
I might buy that Said was operating in a way that would work well in a culture already high in this sort of sportsmanship, but that’s different from causing the culture to have more of it.
Thanks for your patience through the disruption of festival season.
I think one of the arguments for Said being fine is that it is the user’s job to have an emotional orientation where they like Said’s style of criticism, and that’s because that emotional orientation is both good for them personally and also good for society. I buy the argument that the emotional orientation has benefits for society—for example, I think it’s better if the professor has it—
Great to hear!!
I think this is the layer where you need to make your case—that Said was pushing people towards the right sort of sportsmanship—
Sure. As an example of Achmiz pushing people towards the right sort of sportsmanship, I nominate an April 2023 comment in which Achmiz explains why it’s good sportmanship[1] to not complain about being asked for examples.
—and I think this was not true on net.
But why? I just pointed you to what I claim is a very clear example of Achmiz pushing people towards the right sort of sportsmanship, by means of arguing for it at length. How else is one supposed to do it? Reply!
If you concede that the April 2023 comment is an instance of Achmiz pushing people towards the right sort of sportsmanship, what textual evidence do you have of Achmiz doing more to push people away from the right sort of sportsmanship to justify your “not true on net” judgement? What else could Achmiz possibly have done that would have had a different outcome, but explain at length why he thinks his commenting style is correct when the style became a topic of meta conversation? Reply!
but I think the question is “which environment causes there to be more of that orientation?” and I think your plan is “require it and let them sort out how it’s produced” in a way that I think is trying to simplify the world so that you neither need to figure out whether your plan is efficacious
I think “require it and let them sort out how it’s produced” is a perfectly effacious plan. Do you think it wouldn’t work? Why not? Reply!
The basic reason I think it would work is that people respond to incentives and learn the behaviors that their culture considers high-status, and that the behavior in question is learnable, like showering regularly.[2]
In theory, I concede that my plan could fail if the subculture didn’t have “enough status rewards to hand out” to “pay for” the cost of learning (relative to other subcultures that people could spend their lives in): a perfect rationality textbook that no one wants to read wouldn’t raise the sanity waterline of our Earth.[3]
In practice, I think the subculture did and does have enough rewards to hand out. Not only did middle Yudkowsky’s[4] philosophical insight point to a precious timeless ideal, but his generational writing talent planted a flag or beacon for people receptive to the ideal to congregate. People want to be near the beacon! If those entrusted with stewarding the beacon say they had no choice but to alter the message or be destroyed, I mostly just don’t believe them. (But maybe you think the decline of Less Wrong 1.0 is definitive evidence that I can’t successfully explain away.) I think the stewards had (and have) a choice to either maintain standards in order to be faithful to the timeless ideal or betray them in order to be a slightly cooler Bay Area party scene, and they chose (and are choosing) to betray them.
My belief that “require it and let them sort out how it’s produced” works is grounded in personal experience. An illustrative anecdote: I had a painful formative experience in a June 2008 thread on Overcoming Bias. I was offended by another commenter’s anecdote which I construed as misogynistic, to which I replied, “are you aware that this is exactly the sort of psychology that leads to rape?”
That’s not the sort of thing I would ever write today—it was an ad hominem appeal to consequences that didn’t address the commenter’s point—but at the time, I felt righteous: I had spent enough time being socialized by the feminist blogosphere (Feministe, Pandagon, &c.) and only seven months being socialized by Overcoming Bias, such that “punish misogyny” was salient to me as a moral priority and “don’t try to suppress information with ad hominem appeals to consequences” was not.
It would be years before I fully understood why that was a bad comment on my part, but the first step along that road—and the reason it was a painful formative experience—is that Michael Vassar slapped me down hard for it:
Z.M. Davis:
You know, what you just did was, judged by someone who sees this as a forum for truth seeking, REALLY CREEPY. My first impulse is to say that you should be permanently banned for trying to tamper with evidence at the scene of scientific inquiry through moral intimidation aimed at making people reluctant to volunteer or even think about and learn from surprising information. My second thought on the matter is that you just don’t know better and have not done anything similar before, but just to be clear, that is the only post in the history of this blog for which I would even suggest a ban based on a single post on the grounds that the penalty for censorship must be censorship if we are to maintain an open exchange.
That hurt to read! It hurt a lot. But I needed to hear it. To borrow a deep learning metaphor, a high-status group member[5] telling me off was applying a gradient update to me in the direction of, “don’t try to suppress information with ad hominem appeals to consequences”, and, as in deep learning, “require it and let the network/brain sort out how it’s produced” was in fact sufficient.
If a moderator of the new school had been there at the time, perhaps they could have made a case that Vassar’s slapdown and ban threat might have caused me to leave instead, which would be purportedly bad because it would cause the community to lose a valuable future contributor.
In reality, that wasn’t actually a risk. Yudkowsky’s writing was so good that Amanda Marcotte couldn’t compete for my loyalty. I wanted to be near the beacon. I wanted to be near the beacon so badly that I was not only willing to learn things, but willing to learn things even if the process of learning was emotionally uncomfortable.
If your new culture isn’t even trying to teach emotionally uncomfortable things, then you can’t have an art of rationality in the tradition that advised people to try to think the thought that hurt the most.
Where is that tradition today? It doesn’t seem like a coincidence that Michael Vassar has been purged, too.[6] Upthread, you write that the new culture is trying to fix the problem that our kind can’t cooperate. Well, sure. That’s been a source of tension between us since at least 2019. The problem is that rather than inventing new and untried coordination mechanisms as if out of dath ilan, you seem to be using the same playbook that all groups on Earth use to consolidate their power: purge the most principled group members (who are willing to, e.g., enforce norms against appeals to consequences against ingroup members) precisely because they’re principled, and principled people aren’t team players.
I’m sticking my neck out here on the Royal Society analogy, and doing it primarily on the strength of one historian. If it turns out that my understanding of that historian’s work is confused, or the work is discredited, so too my point
Another thing that confuses me about the Royal Society example is that it seems to argue more for banning me rather than Achmiz. The Royal Society (Shapin is telling me via you) wanted people to politely compare their results and not slap people with a glove saying, “You lie, sir.” Okay, but I’m the one who writes exhaustive 80,000-word memoirs slapping my enemies with a glove and calling them liars—and survives, somehow, while Achmiz is the one who asks “Examples?”—and gets purged for “weaponized obtuseness.” I’m actually a little confused about what’s going on here and you might be in a better position to explain it than me. Is it just that my high-effort style makes it look bad to purge me after I put so much work in, whereas Achmiz’s questions put the interpretive labor burden on the author?
He doesn’t use the literal word sportsmanship that you just introduced into the conversation, but he’s appealing to the same ethos when he writes that being asked for examples is “not destructive, but unambiguously constructive and beneficial”—analogously to how you shouldn’t resent your opponent in a sport trying their best rather than letting you win without having to try yourself.
I think “behavior” is a better term than “emotional orientation” for the desired quality here. I’m not necessarily expecting people to enjoy Achmiz-class criticism; I’m expecting people to either address it on the merits or ignore it and let the karma voters decide, and to never, ever complain to the moderators about it. (Although where behavior goes, emotional orientation may follow.)
But what I’m not conceding is important: if humans aren’t interested in the philosophical ideal, then from the standpoint of the philosophy, that’s a matter of human nature being bad rather than the ideal being wrong.
Let’s say that the period of middle Yudkowsky (as contrasted to early or late Yudkowsky) begins with 2005′s “Technical Explanation” and ends with 2012′s “Highly Advanced Epistemology 101 for Beginners”. At the time of the latter, you complained that it was “troublesome” and “embarrassing” that Yudkowsky sullied an otherwise good technical introduction to Bayesian networks with a pummeling of one of his pet strawmen. In retrospect, this was an early warning sign of the tragic decline that would continue through Yudkowsky’s late period to today.
But why? I just pointed you to what I claim is a very clear example of Achmiz pushing people towards the right sort of sportsmanship, by means of arguing for it at length. How else is one supposed to do it? Reply!
I think there are obviously means for encouraging other people to do things besides arguing for it at length. For example, one could reward people for doing it, or one can model it in one’s own behavior.
I think “require it and let them sort out how it’s produced” is a perfectly effacious plan. Do you think it wouldn’t work? Why not? Reply!
Perfectly? Is such exaggeration proper, here?
I think people differ, and so there’s some value in systems that simply judge the external interface and let the individual sort out how they comply with the interface. But I think people are also similar, and there’s some value in us peeling back the layers and comparing how we’re put together. Part of the study of rationality is figuring out the mechanisms of internal techniques; like looking closely enough at meditation to figure out which biological systems it interacts with, or looking closely enough at therapy to distill out Focusing, and so on.
But maybe you think the decline of Less Wrong 1.0 is definitive evidence that I can’t successfully explain away.
Do you have an explanation for the decline of Less Wrong 1.0? That event feels pretty important to my thinking, here, and my read of you is something like “well we didn’t try to Correct Plan hard enough”. Like, if Yudkowsky had more personal virtue, then he wouldn’t have stopped posting on LW 1.0 and there wouldn’t have been the decline.
To express some annoyance, this is 20 pages of detailed back-and-forth, where a historian criticizes the piece, the editor who commissioned the historian’s essay criticizes the essay, then Shapin responds, then the historian responds to Shapin. It is worth reading if you think the historian makes good points; it is not worth reading if you think the counterpoints hold. (I, reading thru it, found myself agreeing with Dear and Shapin more than Feingold.) I am trying to understand what is going on; I am looking for evidence that bears on the question. For recreational debating, it doesn’t matter whether or not arguments are good—in fact, often bad arguments are more fun to tear apart!--but that is not the game I’m trying to play, at the moment.
I’m actually a little confused about what’s going on here and you might be in a better position to explain it than me. Is it just that my high-effort style makes it look bad to purge me after I put so much work in, whereas Achmiz’s questions put the interpretive labor burden on the author?
I also don’t have a complete picture here. I don’t think the effort story resonates that well because I think the outputs are more meaningful than the inputs. I do think we had more basic disagreements with Said about what made for good conversational approaches.
On reflection, no. Thanks for pointing that out. Swap out “perfectly” for “adequately”, and I stand by the rest of the comment.
re: relevance of the Royal Society
...why do you doubt the details are a crux?
Well, it’s not a double crux. Maybe it’s a crux for you.
The reason it’s not a crux for me is because it just seems so remote from the matter at hand. I already have years of direct personal experience productively collaborating with Achmiz on the kind of intellectual work that I want to do. His value to me is not in doubt. It’s just really hard to see what I could possibly learn about 17th-century England that would make me change my mind about that!
It seems like the argument would have to go through Achmiz’s mere presence on the website discouraging so much contribution—despite the existence of the user-level ban feature!—that somehow I’m better off privately benefitting from Achmiz’s counsel via email while he’s officially disgraced and banned from the website which is the central gathering place for the kind of work I do.
I suppose it’s not logically impossible, but it would be super weird for the empirics to shake out that way, and empirical evidence from 17th century England just seems too distant in time and culture to move the needle. The Royal Society isn’t totally irrelevant, of course: human nature hasn’t changed; they were our honorable ancestors doing the natural philosophy thing, and we’re doing a natural philosophy thing.
But our founding texts (which we agree, per the footnote on your 30 May comment, that the current project is running off the fumes of) are very explicit about aiming to do better thanour ancestors. The Sequences are exhorting the reader to follow the timeless ideal of Bayesian reasoning wherever it leads, not imitate how Robert Boyle won friends and influenced people in 17th-century England. (And my discourse-normsposts are likewise trying to appeal to the timeless ideal.)
re: Less Wrong 1.0
Do you have an explanation for the decline of Less Wrong 1.0?
I do think this is at least more relevant than the Royal Society, just because there’s less of a generalization gap to cross between “running a website in 2016” to “running the same website in 2026.”
What I think I can say (and I think you’d agree with the chatbot on these) is that the decline of the original website was not monocausal, and that the differences with LessWrong 2.0 are multidimensional. Things like LessWrong 2.0 having dedicated staff (instead of charity hours from Tricycle) and a less notoriously cursed codebase (in which to, e.g., implement moderation tools that can deal with the Eugene_Nier problem) are factors in LessWrong 2.0′s success.
As a result, when choosing interventions to make sure LessWrong 2.0 doesn’t die, it’s not a binary choice of “intervene or don’t”; there’s a high dimensional space of potential things to push on. I don’t find Habryka’s claim that “the specific way Said has been commenting on the site had a non-trivial chance of basically just killing the site” to be credible—particularly in light of my investigation of alleged author complaints in §III.2. To the extent that authors being discouraged by criticism was a problem, there were other available levers to pull that would save the website with less damage to the mission, such as encouraging use of the user-ban functionality, as I argue for in §IV.1 and suggested in July 2025. When I see Habryka reaching for the lever of purging someone he clearly personally dislikes when less intrusive measures were available and untried, I think I’m justified in judging this as a betrayal of the mission rather than a sincere disagreement about how to pursue the mission (forced by the need to avoid repeating the fate of the original site). If the claim isn’t that the Achmiz ban was necessary to literally save the site, but merely to optimize levels of user engagement, that’s not really substantively disagreeing with my model: I’m saying that you’ve betrayed the mission for the sake of popularity.
my read of you is something like “well we didn’t try to Correct Plan hard enough”. Like, if Yudkowsky had more personal virtue, then he wouldn’t have stopped posting on LW 1.0 and there wouldn’t have been the decline.
I wouldn’t go to “there wouldn’t have been the decline” (because of the high-dimensional multi-causuality), but yes, I do think that factor would have helped, and that Yudkowsky has more generally relinquished his Art and lost his powers. Granted, it’s not obvious how to intervene on a leader’s personal virtue (as contrasted to followers, who can often be bullied as Vassar did to me in 2008), but I think giving up on the idea of personal virtue is much worse!
re: people also being similar
But I think people are also similar, and there’s some value in us peeling back the layers and comparing how we’re put together
Right. That’s why I agree with 2013-Vaniver that “responding appropriately to criticism” is a trainable skill. I don’t think I’m an “emotionally tall” mutant preaching an impossible standard. I think other people could learn it, too, if the culture weren’t saturated with anti-epistemology claiming that they shouldn’t need to.
conflict vs. mistake theories of the present discussion
I am trying to understand what is going on
Are you? In your 30 May comment, you mocked me for spending too many words explaining why prioritizing hedonic and reputational effects trades off against error-correction and therefore correctness, “as the alert reader might have anticipated immediately”. But if it’s supposed to be so obvious that the mod team isn’t maximizing correctness, why would I believe you when you claim to be trying to understand what is going on, if I think that understanding what is going on would almost certainly have negative hedonic and reputational effects (because it would be a weird coincidence if the story that was true also made all the local power players look good)? Maybe you’re trying to understand in the privacy of your own mind, while carefully avoiding any speech acts that would deal unacceptable reputational damage? But the difference isn’t decision-relevant to me; I don’t get to interact with your private intentions in their purity; all I get to see is your speech acts, which strike me as evasive.
(Sorry, I know that was a super rude thing to say, but given what you told me about not putting correctness above all other concerns, you should have expected that I would entertain the hypothesis that you were telling the truth about that and consider the implications.)
I think Ben Hoffman’s theory about “a policy of unprincipled and unbounded submission to threats by the people (or personas) whose feelings are supposed to matter” seems like a better fit to the behavior I’m seeing from you and Anna Salamon. You report that you didn’t have “a personal problem with Said or his comments”, that you “generally found them easy to read and easy to respond to.” Salamon reports that her “personal experiences of Said on LW were clearly and substantially net-positive.” And yet both of you are arguing in favor of the ban to me—not because either of you are willing to testify in your own voice that you think Said’s comments are so bad that you’re not willing to share the website with him, but because other people—largely other unspecified people, given my investigation in §III.2!—aren’t willing to share the website with him. I think that’s weird! Naïvely, if I personally think something is fine, I should go on the record opposing banning it (for the record, even if I’m not the boss and it’s not my decision).
I need something like Hoffman’s theory in order to explain the behavior I’m seeing in you. Importantly, it’s a conflict theory, not a mistake theory: a story about different agents with conflicting goals, rather than everyone and the God-Empress sharing the same utility function. What you consider reputational and hedonic “costs” (to be minimized by the God-Empress for the good of all), Hoffman and I are construing as “threats” (in which an agent claims that they’ll deal disutility to everyone else if their demands aren’t met).
I think if you were actually trying to understand what was going on rather than trying to rationalize a policy handed to you by upstream social forces, you would show more evidence of having thought about the impacts on the site’s mission if the moderators took the other side of the conflict and stood up to the threats by telling complainants to downvote and move on with their lives.
In previous discussion, you’ve told me that the mod team isn’t going to tell users to “get gud” and that trying to shame you into stopping isn’t going to work. I understand that my complaints aren’t going to work. The ban isn’t going to be reversed; the function of this discussion is to get a shared accounting of what happened, where I’m not currently accepting the disvalue attributed to Achmiz as a legitimate cost on the shared ledger. You’ve told me that you felt more optimistic about communication of the form, “Okay, what is the thing that matters here, and how do we get it?” and not “the mods care about Something that is not Logic.” But the problem with that communication prescription is that it presupposes a mistake theory which you haven’t given me reason to believe in!
Do you seriously think that the mods telling complaintants “downvote on move on with your lives” would kill the site? (To be absolutely clear, I’m asking for a conditional prediction, not a policy change; I understand that the mods aren’t going to do that; I’m asking what you think would happen if you did that.)
I would expect some marginal users to bounce, but I don’t think the effect would be quantitatively large, and I think the resulting culture would be much more aligned to the mission. I think most people would grow, the way Michael Vassar helped me grow in 2008.
I could imagine you disagreeing with me on this as an empirical matter of human psychology, but there’s a missing mood in your behavior given that you’ve already agreed that a pro-criticism, pro-being-questioned emotional orientation is good for individuals and good for Society. If it turns out that I’m empirically wrong to think that it’s possible for people to learn the thing we both agree is prosocial, then my reaction is not, “I guess my approach was impractical, therefore wrong, and the new culture is a better way”, but rather, “oh God, we’re screwed, apparently almost no humans are capable of rationality; we were already dead; we were never going to survive.”
Previously, you’ve told me that you think Less Wrong has changed and that Habryka is the person most trying to surf those changes. I think the difference between you and me is that when civilization has fallen to a zombie apocalypse, I think it’s more dignified to go down fighting rather than joining the zombies because they’re the winning side (because surfing the changes means siding with the winners).
Again, I realize that that’s not a nice thing for me to say about you, but I think you’ve been commendably clear about your position, and I think it makes sense for me to be clear, too.
re: encouraging sportsmanship
I think there are obviously means for encouraging other people to do things besides arguing for it at length. For example, one could reward people for doing it, or one can model it in one’s own behavior.
Thanks. That answers my question of how else is one supposed to do it; now that you point it out, I agree that rewarding and modeling could also work.
Conspicuously, that doesn’t answer my question about what textual evidence supports Achmiz doing more to push people away from the right sort of sportsmanship, such as would justify your “not true on net” judgement. I think that after having conceded, as you have, that the pro-criticism orientation is prosocial, the “obvious” conclusion is that the Achmiz ban was unjustified; in that context, the “not true on net” judgement, stated without evidence, looks suspiciously like an ad hoc goalpost-moving excuse. (It’s an odd notion of sportmanship that demands that players not only exhibit good sportsmanship themselves, but also do so in a way that causes others to as well. Traditionally, it’s “how you play the game”, not “how you influence others to play the game.”) I don’t think “he wasn’t modeling it” is credible. Is your claim then that “he wasn’t rewarding it enough”? What would rewarding it look like?
Responding to the section about my argument (and a few related parts):
You title it “The Founding Values Have Not Been Refuted.” But I never set out to refute the founding values! We prioritize the founding values a bit differently; perhaps from your perspective it looks like I’m dethroning the central one. But it really seems relevant to me that I am the one arguing Eliezer’s position from 13 years ago. To summarize my thoughts on Said and your arguments here, I might as well quote him from earlier, “If you fail to achieve a correct answer, it is futile to protest that you acted with propriety.”
I don’t justify it in that comment; the justifications are spread across many comments and many posts and several authors. I would attempt to justify it here, but on reflection I think that’s putting the cart in front of the horse. Cultures are package deals which have many impacts, both positive and negative; no real culture is born from picking a justification and then optimizing purely for that. They are grown, negotiated, and cultivated across many people and many actions.
To continue hammering on my example of the Royal Society, the thing that they wanted to do was understand the world better. They succeeded much more than others before them. Why? Maybe they had more geniuses; maybe they had more wealth; maybe they had better social technology. It seems to me like one of the contributors here is that they believed that social graces were important, and that managing the feelings of the participants was important. Not the most important thing—they weren’t a social club first and foremost—but not banned from consideration, a sort of anti-trump that loses every competition it enters. They wanted people to stay engaged in the project, to keep supplying data and attending meetings, and they thought seriously about how.
I am using it to answer the question: “which of these views incorporates the other?”. I think it’s a valid tool for that.
Now, is that question relevant? I was replying to parts of Said’s comment where he objected to habryka’s framing of the ban post:
I agreed that it was a disagreement about principles, and that it wouldn’t be correct to just say “we’re more sophisticated, therefore we’re right.” The way I’d put it today is that including all the same principles in your calculation and more doesn’t give you a strategy-stealing argument in the way that including all the same information and more does, because it doesn’t mean you are setting the prices correctly. Said and Zack appear to think some things should have 0 cost, and habryka and I think those things have a cost that cannot be ignored; the question is about what price the culture should put on them.
I think this does point to what I find frustrating about this conversation, however; I think we never managed to find cruxes because you thought our charges were inadmissible, and rounded them into something different and more base. (We’ll return to this point.)
So far, I’ve read everything you’ve written on LW moderation issues, because I view it as my responsibility to.
I don’t think so. Or, that is, I think your post spends nearly two thousand words to investigate a simple puzzle, badly. habryka and Duncan complain about the hedonic and reputational effects of a style of behavior. “But what does this have to do with correctness?” you ask, and spend 1300 words demonstrating that, as the alert reader might have anticipated immediately, it is not about putting correctness before all other concerns. Therefore it must be about something else—perhaps, for example, ad revenue.
In a fit of Said-like politeness, I comment to ask why you don’t view comments as speech acts, a category which naturally resolves the puzzle. (Said replies that it’s not unreasonable to, and that most comments are primarily about the information content—which, while true, doesn’t obviate that we’re talking about the cases where it’s not true.)
This time, thankfully, you cut to the chase:
We were open about changing the culture and the discourse norms at the very beginning! One of the goals of LW 2.0 has always been to make posting fun again.
I’m not sure how to say this more clearly, but we disagree about how to accomplish the mission of an intellectual forum like Less Wrong purports to be. It’s the same disagreement as the interpretation of the Royal Society or Socrates. We think it advances instead of betrays our Sacred Mission to enforce a minimum standard of social graces. It is easier for you to pretend that this is about us seeking ad revenue, apparently.
I don’t think your values could build Less Wrong. (Obviously Said can build websites, and has built other forums.) You’re welcome to try, and to put your ideas into practice, and see what comes of them.[1]
The sequitur is that in the post EY argues that a popular belief about rationality is that rationality opposes all emotion, but this isn’t how probability theory works, and instead the question is whether or not emotions are concordant with reality. Rational emotions are downstream of models that correspond to reality; irrational emotions are downstream of models that don’t correspond to reality. When someone finds one of Achmiz’s comments hurtful, is it rational for them to do so? Well, that depends on whether or not the models underlying their hurt correspond to reality.
[Incidentally, one of the reasons why I think this isn’t “the weaponization of emotional harm” is that we’re not just measuring emotional outbursts or tears or whatever, but instead trying to figure out how it connects to our goals for LessWrong, and sometimes the hurt seems real and relevant and sometimes it fails on either count.]
I think you’re mistaking defense and offense. I’m not the one being silenced here, and it’s not Said’s fortitude we’re complaining about.
That is, yes, fortitude is a virtue, as is readiness of comprehension. But it is likewise a virtue to be smooth instead of rough, and clear instead of confusing. Someone’s whose posts are confusing and incoherent may find themselves managed out—either by the karma system, or getting rate-limited, or not accepted as a new user—even if they readily understand all the posts and comments other people write.
Is it a good thing that Said requires more fortitude in those around him than the typical poster? You could imagine a school having an evil potions master in order to cause its students to develop a certain kind of resilience, but that’s not what’s going on here. Instead we had years of asking and telling Said to be easier to deal with, and eventually told him that if he was going to keep it up, he would have to do it elsewhere.
Younger Vaniver was using his personal experience—and the expectations downstream primarily of that, and advice he should have reversed—rather than empiricism, such as by polling others or doing interviews. He wasn’t attuned to the questions of what is good for the site overall, rather focused on the momentary turn-by-turn of the conversation. “Wait, someone in a position of power doing something because it’s fun or unfun?” he said with a look of disapproval, and was not tracking that how much and what Eliezer was posting was downstream of what sort of an environment LessWrong was, and that it was smoother and more rational to work inside of those dynamics rather than attempt to force them into a shape they were not, and that commenters were being shaped by the culture that they were shaping.
Now, is this being corrupted by social pressure? I mean, you could say that, in the same way that a lot of my hygiene habits are the result of social pressure exerted on me, but that doesn’t mean it’s corruption that I shower regularly. There is an actual tension between being able to work with anyone and cultivating a circle of contacts where people can trust that someone being your friend means they’ll likely be their friend too.
Note this is a one-way arrow. I don’t think I ever viewed it as other people’s job to manage my feelings, but I did think I had some duty to other people to manage their feelings. (Otherwise, why be polite?) It was a limited duty—their feelings could be disproportional—but the fundamental question was not whether I was justified in losing but whether I got what I wanted.
In the Sequences era, it was also understood why our kind can’t cooperate. We have been trying to fix that, with our new culture.
To be clear, I don’t think we could build Less Wrong either; I think EY deserves that credit. I don’t think we could even have built LW 2.0 without the inheritance of the URL and The Sequences.
To be clear, I did say “Founding Values” and not “Founders”. Yudkowsky from twenty-three years ago put up a page about Crocker’s rules. (To preëmpt the obvious knee-jerk objection: yes, I did read the part about “Crocker’s Rules does not mean you can insult people.”)
The reason I’m considering the 2002 page about Crocker’s rules but not the 2013 comment about hedonics to be part of the founding ideal, is because to me, the founding ideal isn’t about this Eliza Yudowski person.
It’s about the philosophical vision articulated in “The Bottom Line”, “A Rational Argument”, “What Is Evidence?”, the “Technical Explanation”, and the “Twelve Virtues”—about seeking the mathematical laws that govern how a small part of the world (a “map”) can function as a predictive model of the rest (the “territory”), and how conditional predictions can be used to steer the world.
Crocker’s rules is obviously consilient with the philosophical vision. “Anyone is allowed to call you a moron and claim to be doing you a favor,” “[w]hich, in point of fact, they would be,” says the page. (You see, because having an accurate map is instrumentally convergent for rational agents: if I were an idiot, I would want to know about it, because that might have decision-relevant consequences.)
Of course, humans can’t embody the ideal, but it’s important that there is an ideal, and claims that humans shouldn’t even try to better approximate the ideal because it’s supposedly impossible don’t comport with the philosophical vision I learned from the texts. (As it is written, “The ninth virtue is perfectionism. [...] In every art, if you do not seek perfection you will halt before taking your first steps. If perfection is impossible that is no excuse for not trying.”) If later statements by this Yudowski person contradict the philosophical ideal, I go with the ideal, not the person, because that’s what the texts taught me to do.
Unfortunately, the reason I haven’t been able to engage with this intuitively-surprising-to-me claim much is because I’m not a specialist in 17th Century English history, and so I’m not in a good position to evaluate the claim without a lot of catch-up labor.
The reason it seems like an intuitively surprising claim to me is because—as a non-specialist, I thought the standard explanation for the Royal Society’s success was, um, Science? The experimental method? Right? Like, these are the guys whose motto was Nullius in verba, “Take no one’s word for it”, said to be “an expression of the determination of Fellows to withstand the domination of authority and to verify all statements by an appeal to facts determined by experiment.” I take this to mean that if someone who could contribute to your project (with money, skills, &c.) is sad that you won’t take his word for it, you have to disappoint him and find some other way to keep the project alive.
So it’s not that the cost should be literally zero. (I agree that Crocker’s rules doesn’t mean you get to insult people; Said believes in his version of politeness.) It’s that there is a normative ideal about information processing, and you want a culture that socializes the humans into aspiring towards the normative ideal even when it’s hard (Bayesian reasoners wouldn’t hide from information, I want to be more like that), and the “emotional tallness” excuse doesn’t even pay lip service to the ideal.
You say that so casually! Okay, sure, humans can’t put correctness before all other concerns. But that’s, like, non-normative, right? I’m more optimistic about a culture that encourages striving for the ideal in a non-formalized way, than a pseudo-economic culture that talks a good game about “prices” (which, according to the microeconomic theory, should theoretically exist), but which, in practice, I think psychologically functions as an excuse to not even try. Is that a better crux?
I should confess that that was not my finest work, rhetoric-wise. (I was in a rush that week to ship as many posts as I could before Habryka got the draw on me, posting on the 14th, 16th, 17th, and 20th, and quality may have suffered somewhat.) Hopefully my comments here about the importance of normative ideals in culture is clearer.
Right. (But, um, note that I think most apparent “disagreements” among humans are actually disguised conflicts; I don’t think I’m obligated to take your self-report literally, nor would I expect you to trust mine.)
I’d expect you’d agree that focusing on Yudkowsky feeling comfortable with posting is Goodhartable. We likely disagree on how Goodharted it is in practice. (As I’ve written about at length elsewhere, I think a lot of Yudkowsky’s work since at least 2016 is just not very good for reasons that have to do with him giving up on the normative ideal.)
I’ve attempted to read primary sources from around the origins of Science (quite a bit, though still less than I’d like and also it’s been 20 years), and my own impression is that Science was enabled partly by a designed shift in social graces (by humans who thought social graces matter), but, not a content-neutral sort of advance in social graces (nothing like “this way we can create more harmony in arbitrary situations”), but an engineered change in social etiquette aimed specifically at making it easy for gentlemen who cared about their reputations, and cared also about not being challenged to duels etc., to be able to verify one anothers’ experiments. Like, “we have a norm around here in which everyone aspires to verify everything for themselves, and so if they ask you how you did your experiment, and share their own attempted replications and results, they won’t be doubting your honor, they’ll instead be practicing our local virtues. Also, everyone will keep their speech strictly about what happened when they tried different experiments, and will not e.g. accuse one another of lying—they’ll only say that their own attempt yielded a different result.”
I do think it was pretty darn different from Crocker’s Rules.
This explanation is on the wrong level of abstraction. From my point of view, the whole question is “How did this group of humans manage to do science together?” It wasn’t by embroidering “Science!” onto their banners; there were lots of details to their organization.
(And the practice of science has changed substantially, over the years, such that even a practicing scientist today might be far out of touch with the underpinnings of the true science. Comparing organizations, comparing eras, asking which is better—this takes scholarship and empiricism to figure out.)
Like, on the simplest level, Nullius in Verbia cannot be the basis of science because of limited human observational capacity. (Do you think the Earth has magma inside it? Is that because you checked, or because someone told you?) The actual basis of science has to be something more like “take people’s word only on clearly laid-out chains of checkable inferences and observations, and careful observation of their character”.[1]
Crocker’s rules erode the foundation on which they rest, in a way that I suspect Eliezer learned by experience. If it is costly to call you a moron, then people deciding whether or not to do it will have to determine whether or not it’s worth making the bet, and to the extent they have skill at telling whether or not you’re a moron, they’ll only call you out when it’s correct. But if it’s too costly, you miss out on corrections you would’ve want to have; the only way to get all of those is by making it free, and thus all bets worthwhile.
But all bets? The only reason to tell an unknown they’re a moron is because they have made a mistake; there are many reasons to call a celebrity a moron which do not depend on the celebrity having made a mistake. Crocker’s Rules liberates the autistic, but it also attracts the cruel and meanspirited to you, the sneerers who delight in you having abandoned a sort of social self-defense. At some point the signal from criticism drops into the negative, and you find yourself missing out again, because it is no longer worthwhile to pay attention to the pile of messages.
[This is why, if you do follow Crocker’s Rules, it primarily makes sense to do so for private communications; there’s less reward for sending someone hatemail than there is for writing hateful comments.]
That is--
I think it would be different if we were saying “look, Said hurts our feelings and that’s not where we’d like to spend our limited pain tolerance.”[2] Instead we’re saying things like the preceding sections, that “yeah I can see why people might think that Crocker’s Rules make sense, but actually we think the society where people expect that other people are (or should be) following Crocker’s Rules is worse than one where they don’t.”
That is, the ideal is actually different based on your goals and situation. The Tao of a lone scholar is not the same as the Tao of a participant in a university or a scene; arguments that you receive a benefit in some situation from following a policy do not constitute a complete argument for that policy.
I think the thing that often is going on is that someone will complain that Said is annoying, or that they didn’t like his speech act along some layer other than correctness. This will then be interpreted by you or Said as being about correctness—the deflection of legitimate concerns rather than addressing them—which will then be viewed by the someone as deflection of their legitimate concerns (about something other than correctness). Unsurprisingly, little progress is made from that starting point.
To return to the point of instrumental convergence—many things are instrumentally convergent for rational agents, like continued survival, or other agents thinking highly of them, and so on. A thing that we need for a proper intellectual scene is more like sportsmanship, in that people want the truth to win socially, or are identifying with the sport instead of with their team. A professor who suppresses papers critical of his contribution to an academic field is not being irrational—he is pursuing a plan that achieves his values based on an accurate understanding of the world—he is instead being evil or anti-social, in that he’s putting his wants above humanity’s shared understanding; we wish he was operating on different values.
I think this is the layer where you need to make your case—that Said was pushing people towards the right sort of sportsmanship—and I think this was not true on net.[3]
[Edit] On reflection I think this section might be confusing and I should try to lay out my reasoning more clearly. I think one of the arguments for Said being fine is that it is the user’s job to have an emotional orientation where they like Said’s style of criticism, and that’s because that emotional orientation is both good for them personally and also good for society. I buy the argument that the emotional orientation has benefits for society—for example, I think it’s better if the professor has it—but I think the question is “which environment causes there to be more of that orientation?” and I think your plan is “require it and let them sort out how it’s produced” in a way that I think is trying to simplify the world so that you neither need to figure out whether your plan is efficacious or put in any of the work that would cause the system to be better.
Like, I’m sticking my neck out here on the Royal Society analogy, and doing it primarily on the strength of one historian. If it turns out that my understanding of that historian’s work is confused, or the work is discredited, so too my point, and this is bad for my overall credibility.
The closest to this is the “limited moderation time” argument, but I’m not sure time is actually the most important resource there.
I might buy that Said was operating in a way that would work well in a culture already high in this sort of sportsmanship, but that’s different from causing the culture to have more of it.
Thanks for your patience through the disruption of festival season.
Great to hear!!
Sure. As an example of Achmiz pushing people towards the right sort of sportsmanship, I nominate an April 2023 comment in which Achmiz explains why it’s good sportmanship [1] to not complain about being asked for examples.
But why? I just pointed you to what I claim is a very clear example of Achmiz pushing people towards the right sort of sportsmanship, by means of arguing for it at length. How else is one supposed to do it? Reply!
If you concede that the April 2023 comment is an instance of Achmiz pushing people towards the right sort of sportsmanship, what textual evidence do you have of Achmiz doing more to push people away from the right sort of sportsmanship to justify your “not true on net” judgement? What else could Achmiz possibly have done that would have had a different outcome, but explain at length why he thinks his commenting style is correct when the style became a topic of meta conversation? Reply!
I think “require it and let them sort out how it’s produced” is a perfectly effacious plan. Do you think it wouldn’t work? Why not? Reply!
The basic reason I think it would work is that people respond to incentives and learn the behaviors that their culture considers high-status, and that the behavior in question is learnable, like showering regularly. [2]
In theory, I concede that my plan could fail if the subculture didn’t have “enough status rewards to hand out” to “pay for” the cost of learning (relative to other subcultures that people could spend their lives in): a perfect rationality textbook that no one wants to read wouldn’t raise the sanity waterline of our Earth. [3]
In practice, I think the subculture did and does have enough rewards to hand out. Not only did middle Yudkowsky’s [4] philosophical insight point to a precious timeless ideal, but his generational writing talent planted a flag or beacon for people receptive to the ideal to congregate. People want to be near the beacon! If those entrusted with stewarding the beacon say they had no choice but to alter the message or be destroyed, I mostly just don’t believe them. (But maybe you think the decline of Less Wrong 1.0 is definitive evidence that I can’t successfully explain away.) I think the stewards had (and have) a choice to either maintain standards in order to be faithful to the timeless ideal or betray them in order to be a slightly cooler Bay Area party scene, and they chose (and are choosing) to betray them.
My belief that “require it and let them sort out how it’s produced” works is grounded in personal experience. An illustrative anecdote: I had a painful formative experience in a June 2008 thread on Overcoming Bias. I was offended by another commenter’s anecdote which I construed as misogynistic, to which I replied, “are you aware that this is exactly the sort of psychology that leads to rape?”
That’s not the sort of thing I would ever write today—it was an ad hominem appeal to consequences that didn’t address the commenter’s point—but at the time, I felt righteous: I had spent enough time being socialized by the feminist blogosphere (Feministe, Pandagon, &c.) and only seven months being socialized by Overcoming Bias, such that “punish misogyny” was salient to me as a moral priority and “don’t try to suppress information with ad hominem appeals to consequences” was not.
It would be years before I fully understood why that was a bad comment on my part, but the first step along that road—and the reason it was a painful formative experience—is that Michael Vassar slapped me down hard for it:
That hurt to read! It hurt a lot. But I needed to hear it. To borrow a deep learning metaphor, a high-status group member [5] telling me off was applying a gradient update to me in the direction of, “don’t try to suppress information with ad hominem appeals to consequences”, and, as in deep learning, “require it and let the network/brain sort out how it’s produced” was in fact sufficient.
If a moderator of the new school had been there at the time, perhaps they could have made a case that Vassar’s slapdown and ban threat might have caused me to leave instead, which would be purportedly bad because it would cause the community to lose a valuable future contributor.
In reality, that wasn’t actually a risk. Yudkowsky’s writing was so good that Amanda Marcotte couldn’t compete for my loyalty. I wanted to be near the beacon. I wanted to be near the beacon so badly that I was not only willing to learn things, but willing to learn things even if the process of learning was emotionally uncomfortable.
If your new culture isn’t even trying to teach emotionally uncomfortable things, then you can’t have an art of rationality in the tradition that advised people to try to think the thought that hurt the most.
Where is that tradition today? It doesn’t seem like a coincidence that Michael Vassar has been purged, too. [6] Upthread, you write that the new culture is trying to fix the problem that our kind can’t cooperate. Well, sure. That’s been a source of tension between us since at least 2019. The problem is that rather than inventing new and untried coordination mechanisms as if out of dath ilan, you seem to be using the same playbook that all groups on Earth use to consolidate their power: purge the most principled group members (who are willing to, e.g., enforce norms against appeals to consequences against ingroup members) precisely because they’re principled, and principled people aren’t team players.
I mean, it’s not as if Shapin doesn’t have critics, but I doubt the details are a crux.
Another thing that confuses me about the Royal Society example is that it seems to argue more for banning me rather than Achmiz. The Royal Society (Shapin is telling me via you) wanted people to politely compare their results and not slap people with a glove saying, “You lie, sir.” Okay, but I’m the one who writes exhaustive 80,000-word memoirs slapping my enemies with a glove and calling them liars—and survives, somehow, while Achmiz is the one who asks “Examples?”—and gets purged for “weaponized obtuseness.” I’m actually a little confused about what’s going on here and you might be in a better position to explain it than me. Is it just that my high-effort style makes it look bad to purge me after I put so much work in, whereas Achmiz’s questions put the interpretive labor burden on the author?
He doesn’t use the literal word sportsmanship that you just introduced into the conversation, but he’s appealing to the same ethos when he writes that being asked for examples is “not destructive, but unambiguously constructive and beneficial”—analogously to how you shouldn’t resent your opponent in a sport trying their best rather than letting you win without having to try yourself.
I think “behavior” is a better term than “emotional orientation” for the desired quality here. I’m not necessarily expecting people to enjoy Achmiz-class criticism; I’m expecting people to either address it on the merits or ignore it and let the karma voters decide, and to never, ever complain to the moderators about it. (Although where behavior goes, emotional orientation may follow.)
But what I’m not conceding is important: if humans aren’t interested in the philosophical ideal, then from the standpoint of the philosophy, that’s a matter of human nature being bad rather than the ideal being wrong.
Let’s say that the period of middle Yudkowsky (as contrasted to early or late Yudkowsky) begins with 2005′s “Technical Explanation” and ends with 2012′s “Highly Advanced Epistemology 101 for Beginners”. At the time of the latter, you complained that it was “troublesome” and “embarrassing” that Yudkowsky sullied an otherwise good technical introduction to Bayesian networks with a pummeling of one of his pet strawmen. In retrospect, this was an early warning sign of the tragic decline that would continue through Yudkowsky’s late period to today.
Vassar would soon become President of what was then the Singularity Institute.
Notice “Evil is bad, actually (Vassar and Olivia Schaefer)” being voted up to 120 karma while the top comment at 105 karma and 85 agreement says “this post basically doesn’t make any sense.” This seems diagnostic of a culture that doesn’t care about whether accusations make any sense as long as they’re pointed at acceptable targets whom “everyone knows” are Bad.
I think there are obviously means for encouraging other people to do things besides arguing for it at length. For example, one could reward people for doing it, or one can model it in one’s own behavior.
Perfectly? Is such exaggeration proper, here?
I think people differ, and so there’s some value in systems that simply judge the external interface and let the individual sort out how they comply with the interface. But I think people are also similar, and there’s some value in us peeling back the layers and comparing how we’re put together. Part of the study of rationality is figuring out the mechanisms of internal techniques; like looking closely enough at meditation to figure out which biological systems it interacts with, or looking closely enough at therapy to distill out Focusing, and so on.
Do you have an explanation for the decline of Less Wrong 1.0? That event feels pretty important to my thinking, here, and my read of you is something like “well we didn’t try to Correct Plan hard enough”. Like, if Yudkowsky had more personal virtue, then he wouldn’t have stopped posting on LW 1.0 and there wouldn’t have been the decline.
...why do you doubt the details are a crux?
To express some annoyance, this is 20 pages of detailed back-and-forth, where a historian criticizes the piece, the editor who commissioned the historian’s essay criticizes the essay, then Shapin responds, then the historian responds to Shapin. It is worth reading if you think the historian makes good points; it is not worth reading if you think the counterpoints hold. (I, reading thru it, found myself agreeing with Dear and Shapin more than Feingold.) I am trying to understand what is going on; I am looking for evidence that bears on the question. For recreational debating, it doesn’t matter whether or not arguments are good—in fact, often bad arguments are more fun to tear apart!--but that is not the game I’m trying to play, at the moment.
I also don’t have a complete picture here. I don’t think the effort story resonates that well because I think the outputs are more meaningful than the inputs. I do think we had more basic disagreements with Said about what made for good conversational approaches.
Thanks for your continued patience.
On reflection, no. Thanks for pointing that out. Swap out “perfectly” for “adequately”, and I stand by the rest of the comment.
re: relevance of the Royal Society
Well, it’s not a double crux. Maybe it’s a crux for you.
The reason it’s not a crux for me is because it just seems so remote from the matter at hand. I already have years of direct personal experience productively collaborating with Achmiz on the kind of intellectual work that I want to do. His value to me is not in doubt. It’s just really hard to see what I could possibly learn about 17th-century England that would make me change my mind about that!
It seems like the argument would have to go through Achmiz’s mere presence on the website discouraging so much contribution—despite the existence of the user-level ban feature!—that somehow I’m better off privately benefitting from Achmiz’s counsel via email while he’s officially disgraced and banned from the website which is the central gathering place for the kind of work I do.
I suppose it’s not logically impossible, but it would be super weird for the empirics to shake out that way, and empirical evidence from 17th century England just seems too distant in time and culture to move the needle. The Royal Society isn’t totally irrelevant, of course: human nature hasn’t changed; they were our honorable ancestors doing the natural philosophy thing, and we’re doing a natural philosophy thing.
But our founding texts (which we agree, per the footnote on your 30 May comment, that the current project is running off the fumes of) are very explicit about aiming to do better than our ancestors. The Sequences are exhorting the reader to follow the timeless ideal of Bayesian reasoning wherever it leads, not imitate how Robert Boyle won friends and influenced people in 17th-century England. (And my discourse-norms posts are likewise trying to appeal to the timeless ideal.)
re: Less Wrong 1.0
I do think this is at least more relevant than the Royal Society, just because there’s less of a generalization gap to cross between “running a website in 2016” to “running the same website in 2026.”
I think my honest answer here has to be No, I don’t have an explanation—in the sense that I don’t think I can outperform the replacement-level chatbot answer to this question.
What I think I can say (and I think you’d agree with the chatbot on these) is that the decline of the original website was not monocausal, and that the differences with LessWrong 2.0 are multidimensional. Things like LessWrong 2.0 having dedicated staff (instead of charity hours from Tricycle) and a less notoriously cursed codebase (in which to, e.g., implement moderation tools that can deal with the Eugene_Nier problem) are factors in LessWrong 2.0′s success.
As a result, when choosing interventions to make sure LessWrong 2.0 doesn’t die, it’s not a binary choice of “intervene or don’t”; there’s a high dimensional space of potential things to push on. I don’t find Habryka’s claim that “the specific way Said has been commenting on the site had a non-trivial chance of basically just killing the site” to be credible—particularly in light of my investigation of alleged author complaints in §III.2. To the extent that authors being discouraged by criticism was a problem, there were other available levers to pull that would save the website with less damage to the mission, such as encouraging use of the user-ban functionality, as I argue for in §IV.1 and suggested in July 2025. When I see Habryka reaching for the lever of purging someone he clearly personally dislikes when less intrusive measures were available and untried, I think I’m justified in judging this as a betrayal of the mission rather than a sincere disagreement about how to pursue the mission (forced by the need to avoid repeating the fate of the original site). If the claim isn’t that the Achmiz ban was necessary to literally save the site, but merely to optimize levels of user engagement, that’s not really substantively disagreeing with my model: I’m saying that you’ve betrayed the mission for the sake of popularity.
I wouldn’t go to “there wouldn’t have been the decline” (because of the high-dimensional multi-causuality), but yes, I do think that factor would have helped, and that Yudkowsky has more generally relinquished his Art and lost his powers. Granted, it’s not obvious how to intervene on a leader’s personal virtue (as contrasted to followers, who can often be bullied as Vassar did to me in 2008), but I think giving up on the idea of personal virtue is much worse!
re: people also being similar
Right. That’s why I agree with 2013-Vaniver that “responding appropriately to criticism” is a trainable skill. I don’t think I’m an “emotionally tall” mutant preaching an impossible standard. I think other people could learn it, too, if the culture weren’t saturated with anti-epistemology claiming that they shouldn’t need to.
conflict vs. mistake theories of the present discussion
Are you? In your 30 May comment, you mocked me for spending too many words explaining why prioritizing hedonic and reputational effects trades off against error-correction and therefore correctness, “as the alert reader might have anticipated immediately”. But if it’s supposed to be so obvious that the mod team isn’t maximizing correctness, why would I believe you when you claim to be trying to understand what is going on, if I think that understanding what is going on would almost certainly have negative hedonic and reputational effects (because it would be a weird coincidence if the story that was true also made all the local power players look good)? Maybe you’re trying to understand in the privacy of your own mind, while carefully avoiding any speech acts that would deal unacceptable reputational damage? But the difference isn’t decision-relevant to me; I don’t get to interact with your private intentions in their purity; all I get to see is your speech acts, which strike me as evasive.
(Sorry, I know that was a super rude thing to say, but given what you told me about not putting correctness above all other concerns, you should have expected that I would entertain the hypothesis that you were telling the truth about that and consider the implications.)
I think Ben Hoffman’s theory about “a policy of unprincipled and unbounded submission to threats by the people (or personas) whose feelings are supposed to matter” seems like a better fit to the behavior I’m seeing from you and Anna Salamon. You report that you didn’t have “a personal problem with Said or his comments”, that you “generally found them easy to read and easy to respond to.” Salamon reports that her “personal experiences of Said on LW were clearly and substantially net-positive.” And yet both of you are arguing in favor of the ban to me—not because either of you are willing to testify in your own voice that you think Said’s comments are so bad that you’re not willing to share the website with him, but because other people—largely other unspecified people, given my investigation in §III.2!—aren’t willing to share the website with him. I think that’s weird! Naïvely, if I personally think something is fine, I should go on the record opposing banning it (for the record, even if I’m not the boss and it’s not my decision).
I need something like Hoffman’s theory in order to explain the behavior I’m seeing in you. Importantly, it’s a conflict theory, not a mistake theory: a story about different agents with conflicting goals, rather than everyone and the God-Empress sharing the same utility function. What you consider reputational and hedonic “costs” (to be minimized by the God-Empress for the good of all), Hoffman and I are construing as “threats” (in which an agent claims that they’ll deal disutility to everyone else if their demands aren’t met).
I think if you were actually trying to understand what was going on rather than trying to rationalize a policy handed to you by upstream social forces, you would show more evidence of having thought about the impacts on the site’s mission if the moderators took the other side of the conflict and stood up to the threats by telling complainants to downvote and move on with their lives.
In previous discussion, you’ve told me that the mod team isn’t going to tell users to “get gud” and that trying to shame you into stopping isn’t going to work. I understand that my complaints aren’t going to work. The ban isn’t going to be reversed; the function of this discussion is to get a shared accounting of what happened, where I’m not currently accepting the disvalue attributed to Achmiz as a legitimate cost on the shared ledger. You’ve told me that you felt more optimistic about communication of the form, “Okay, what is the thing that matters here, and how do we get it?” and not “the mods care about Something that is not Logic.” But the problem with that communication prescription is that it presupposes a mistake theory which you haven’t given me reason to believe in!
Do you seriously think that the mods telling complaintants “downvote on move on with your lives” would kill the site? (To be absolutely clear, I’m asking for a conditional prediction, not a policy change; I understand that the mods aren’t going to do that; I’m asking what you think would happen if you did that.)
I would expect some marginal users to bounce, but I don’t think the effect would be quantitatively large, and I think the resulting culture would be much more aligned to the mission. I think most people would grow, the way Michael Vassar helped me grow in 2008.
I could imagine you disagreeing with me on this as an empirical matter of human psychology, but there’s a missing mood in your behavior given that you’ve already agreed that a pro-criticism, pro-being-questioned emotional orientation is good for individuals and good for Society. If it turns out that I’m empirically wrong to think that it’s possible for people to learn the thing we both agree is prosocial, then my reaction is not, “I guess my approach was impractical, therefore wrong, and the new culture is a better way”, but rather, “oh God, we’re screwed, apparently almost no humans are capable of rationality; we were already dead; we were never going to survive.”
Previously, you’ve told me that you think Less Wrong has changed and that Habryka is the person most trying to surf those changes. I think the difference between you and me is that when civilization has fallen to a zombie apocalypse, I think it’s more dignified to go down fighting rather than joining the zombies because they’re the winning side (because surfing the changes means siding with the winners).
Again, I realize that that’s not a nice thing for me to say about you, but I think you’ve been commendably clear about your position, and I think it makes sense for me to be clear, too.
re: encouraging sportsmanship
Thanks. That answers my question of how else is one supposed to do it; now that you point it out, I agree that rewarding and modeling could also work.
Conspicuously, that doesn’t answer my question about what textual evidence supports Achmiz doing more to push people away from the right sort of sportsmanship, such as would justify your “not true on net” judgement. I think that after having conceded, as you have, that the pro-criticism orientation is prosocial, the “obvious” conclusion is that the Achmiz ban was unjustified; in that context, the “not true on net” judgement, stated without evidence, looks suspiciously like an ad hoc goalpost-moving excuse. (It’s an odd notion of sportmanship that demands that players not only exhibit good sportsmanship themselves, but also do so in a way that causes others to as well. Traditionally, it’s “how you play the game”, not “how you influence others to play the game.”) I don’t think “he wasn’t modeling it” is credible. Is your claim then that “he wasn’t rewarding it enough”? What would rewarding it look like?