I do know that you and I have clashed before on what constitutes adhering to the standards, though I believe (as I believe you believe) that in such situations it should be possible to simply take the conversation up a meta level.
On the current margin, I am interested in users taking more risks with individual comments and posts, not less. People take more risk when the successes are rewarded more, and when the failures are punished less. I generally encourage very low standards for individual comments, similar to how I have very low standards for individual word-choice or sentence structure. I want to reward or punish users for their body of contributions rather than pick each one apart and make sure it’s “up to standard”. (As an example, see the how this moderation notice is framed as a holistic evaluation of a user’s contributions, not about a single comment.)
So yes, Said, I am broadly opposed to substantially increasing the standards applied to each individual comment or paragraph. I am much more in favor of raising the amount of reward you can get for putting in remarkable amounts of effort and contributing great insights and knowledge. I think your support of Eliezer by making readthesequences.com and your support of Gwern with the site re-design are examples of the kind of things I think will make people like Eliezer and Gwern feel like their best writing is rewarded, rather than increased punishment for their least good comments and posts.
I really don’t care if someone ‘misses the mark’ most of the time, if they succeed the few times required on the path to greatness. Users like John Wentworth and Alex Flint produce lots of posts that receive relatively low karma, and also a (smaller) number of ‘hits’ that are some of my favorite site-content in recent years. I think if they attempted to write in a way that meant the bottom 80% of posts (by karma) didn’t get published, then they would not feel comfortable risking the top 20% of posts, as it wouldn’t be clear to them that they would meet the new standard, and it wouldn’t be worth the risk.
I think one of the successes of LW 2.0 has been in reducing the pain associated with having content not be well received, while also increasing the visibility and reward for contributions that are excellent (via frontpage karma-date-weighted sorting and by curation and published books and more).
″...back in my day physics classes gave lots of hard problems that most students couldn’t do. So there was a lot of noise in particular grades, and students cared as much or more about possibly doing unusually well as doing unusually badly. One stellar performance might make your reputation, and make up for lots of other mediocre work. But today, schools give lots of assignments where most get high percent scores, and even many where most get 100% scores. In this sort of world, students know it is mostly about not making mistakes, and avoiding black marks. There is otherwise little they can do to stand out...the new focus is on the low, not the high, end of the distribution of outcomes for each event or activity.”
On the topic of LessWrong as a place to practice rationality
Duncan, it seems to me that you want LessWrong to be (in substantial part) a dojo, where we are together putting in effort to help each other be stronger in our epistemic and instrumental processes.
I love this idea. One time where something like this happened was Jacob’s babble challenges last year, where he challenges users to babble 100 ways to solve a given problem: 1, 2, 3, 4, 5, 6, 7. Loads of people got involved, and I loved it. There’s been a few other instances where lots of people tried something difficult, like under Alkjash’s Hammertime Final Exam and Scott Garrabrant’s Fixed Point Exercises. Also lsusr’s The Darwin Game had this energy, as did the winners of my Rationality Exercises Prize.
The above posts are strong evidence to me that people want something like a rationality dojo on LessWrong, a place to practice and become stronger. I think there is a space for this to grow on LessWrong.
To state the obvious, using the risk/reward frame above, I think just punishing people more for not doing their practice would result in far fewer great contributions to the site. But I think it’s very promising to reward people more for putting in very high levels of effortinto practice, by celebrating them and making their achievements legible and giving them prizes. I suspect that this could change the site culture substantially.
As Duncan know better than most any other person I’ve met, you don’t teach just by explaining, and I think there’s real potential for people on LW to practice as well. I’d be quite excited about a world where Duncan builds on the sorts of threads that Jacob and others have made, making rationality exercises and tests for people to practice together on LW, and building up a school of people over the years who gain great skill and produce ambitiously successful projects.
Duncan, it seems on the table to me that you think there’s some promise to doing this too, here on LessWrong. Do you think you’d be up for trying something like the threads above (but with your own flavor)? I’m happy to babble ideas together with you this Friday :)
What you say makes sense if, and only if, the presence of “bad” content is costless.
And that condition has (at least) these prerequisites:
Everyone (or near enough) clearly sees which content is bad; everyone agrees that the content is bad, and also on what makes it bad; and thus…
… the bad content is clearly and publicly judged as such, and firmly discarded, so that…
… nobody adopts or integrates the bad ideas from the bad content, and nobody’s reasoning, models, practices, behavior, etc. is affected (negatively) by the bad content; and relatedly…
… the bad content does not “crowd out” the good content, bad ideas from it do not outcompete opposing good ideas on corresponding topics, the bad ideas in the bad content never become the consensus views on any relevant subjects, and the bad reasoning in the bad content never affects the norms for discussion (of good content, or of anything) on the site (e.g., is never viewed by newcomers, taken to be representative, and understood to be acceptable).
If, indeed, these conditions obtain, then your perspective is eminently reasonable, and your chosen policy almost certainly the right one.
But it seems very clear to me that these conditions absolutely do not obtain. Every single thing I listed above is, in fact, entirely false, on Less Wrong.
And that means that “bad” content is far from costless. It means that such content imposes terrible costs, in fact; it means that tolerating such content means that we tolerate the corrosion of our ability to produce good content—which is to say, our ability to find what is true, and to do useful things. (And when I say “our”, I mean both “Less Wrong’s, collectively” and “the participants’, individually”.)
(Unlike your comment, which is, commendably, rife with examples, you’ll note that my reply provides no examples at all. This is intentional; I have little desire to start a fight, as it were, by “calling out” any posters or commenters. I will provide examples on request… but I suspect that anyone participating in this conversation will have little trouble coming up with more than a few examples, even without my help.)
What you say makes sense if, and only if, the presence of “bad” content is costless.
“Iff” is far too strong. I agree that the “if” claim holds. However, I think that what Ben says also makes sense ifthe bad/high-variance content has costs which are less than its benefits. Demanding costlessness imposes an unnecessarily high standard on positions disagreeing with your own, I think.
Contrasting your position with Ben’s, I sense a potential false dichotomy. Must it be true that either we open the floodgates and allow who-knows-what on the site in order to encourage higher-variance moves, or we sternly allow only the most well-supported reasoning? I think not. What other solutions might be available?
The first—but surely not best—to come to mind is the curation < LW review < ??? pipeline, where posts are subjected to increasing levels of scrutiny and rewarded with increasing levels of visibility. Perhaps there might be some way for people to modulate “how much they update on a post” by “the amount of scrutiny the post has received.” I don’t think this quite fights the corrosion you point at. But it seems like something is possible here, and in any case it seems to me too early to conclude there is only one axis of variation in responses to the situation (free-wheeling vs strict).
I have repeatedly suggested/advocated the (to me, fairly obvious) solution where (to summarize / crystallize my previous commentary on this):
People post things on their personal LW blogs. Post authors have moderation powers on their personal-blog posts.
Things are posted to the front page only if (but not necessarily “if”!) they are intended to be subject to the sort of scrutiny wherein we insist that posts live up to non-trivial epistemic/etc. standards (with attendant criticism, picking-apart, analysis, etc.; and also with attendant downvotes for posts judged to be bad). Importantly, post authors do not have moderation powers in this case, nor the ability to decide on moderation standards for comments on their posts. (In this case a post might be front-paged by the author, or, with the author’s consent, by the mods.)
Posts that go to the front page, are evaluated by the above-described process, and judged to be unusually good, may be “curated” or what have you.
In this case, it would be proper for the community to judge personal-blog posts, that have not been subjected to “frontpage-level” scrutiny, as essentially ignorable. This would go a long way toward ensuring that posts of the “jam-packed with bullshit” type (which would either be posted to personal blogs only, or would go to the front page and be mercilessly torn apart, and clearly and publicly judged to be poor) would be largely costless.
I agree with you that this sort of setup would not quite solve the problem, and also that it would nonetheless improve the situation markedly.
But the LW team has consistently been opposed to this sort of proposal.
It sounds to me like posting on your High-Standards-Frontpage is a very high effort endeavor, an amount of effort that currently only around 3-30 posts each year have put into them. I’ve thought of this idea before with the name “LW Journal” or “LW Peer Review”, which also had a part where it wasn’t only commenters critiquing your post, but we paid a few people full-time for reviewing of the posts in this pipeline, and there was also a clear pass/failure with each submission. (Scott Garrabrant has also suggested this idea to me in the past, as a publishing place for his papers.)
I think the main requirement I see is a correspondingly larger incentive to write something that passes this bar. Else I mostly expect the same fate to befall us as with LW 1.0, where Main became increasingly effortful and unpleasant for authors to post to, such that writers like Scott Alexander moved away to writing on their personal blogs.
(I’m generally interested to hear ideas for what would be a big reward for writers to do this sort of thing. The first ones that come to my mind are “money” and “being published in physical books”.)
I do think that something like this would really help the site in certain ways; I think a lot of people have a hard time figuring out what standard to hold their posts to, and having a clearly “high standard” and “lower standard” place would help authors feel more comfortable knowing what they’re aiming for in their writing. (“Shortform” was an experiment with a kind of lower-standards place.) But I don’t currently see a simple way to cause a lot of people to produce high-effort high-standards content for that part of the site, beyond the amount of effort we currently receive on the highest effort posts each year.
The first ones that come to my mind are “money” and “being published in physical books”.
So I think the Review is pretty good at getting good old content, but I think the thing Said is talking about should happen more quickly, and should be more like Royal Society Letters or w/e.
Actually, I wonder about Rohin’s newsletters as a model/seed. They attract more scrutiny to things, but they come with the reward of Rohin’s summary (and, presumably, more eyeballs than it would have gotten on its own). But also people were going to be writing those things for their own reasons anyway.
I think if we had the Eliezer-curated weekly newsletter of “here are the LW posts that caught my interest plus commentary on them”, we would probably think the reward and scrutiny were balanced. Of course, as with any suggestion that proposes spending Eliezer-time on something, I think this is pretty dang expensive—but the Royal Society Letters were also colossally expensive to produce.
I would likely do this from my own motivation (i.e. not necessarily need money) if I were given at least one of:
a) guaranteed protection from the badgunk comments by e.g. three moderators willing to be dependably high-effort down in the comments
b) given the power to hide badgunk comments pending their author rewriting them to eliminate the badgunk
c) given the power to leave inline commentary on people’s badgunk comments
The only thing holding me back from doing something much more like what Said proposes is “LW comment sections regularly abuse and exhaust me.” Literally that’s the only barrier, and it’s a substantial one. If LW comment sections did not regularly abuse and exhaust me, such that every post feels like I need to set aside fifty hours of life and spoons just in case, then I could and would be much more prolific.
(To be clear: some people whose pushback on this post was emphatically not abuse or exhausting include supposedlyfun, Said, Elizabeth, johnswentsworth, and agrippa.)
a) guaranteed protection from the badgunk comments by e.g. three moderators willing to be dependably high-effort down in the comments
Would you accept this substitute:
“A site/community culture where other commenters will reliably ‘call out’ (and downvote) undesirable comments, and will not be punished for doing so (and attempts to punish them for such ‘vigilante-style’ a.k.a. ‘grassroots’ ‘comment policing’ will themselves be punished—by other commenters, recursively, with support from moderators if required).”
Yes, absolutely. Thanks for noting it. That substitute is much more what the OP is pushing for.
EDIT: With the further claim that, once such activity is reliable and credible, its rate will also decrease. That standards, clearly held and reliably enforced, tend to beget fewer violations in the first place, and that, in other words, I don’t think this would be a permanent uptick in policing.
Strong-upvote. I want to register that I have had disagreements with Said in the past about this, and while I am still not completely sure whether I agree with his frame, recent developments have in fact caused me to update significantly towards his view.
I suspect this is true of others as well, such that I think Said’s view (as well as associated views that may differ in specifics but agree in thrust) can no longer be treated as the minority viewpoint. (They may still be the minority view, but if so I don’t expect it to be a small minority anymore, where “small” might be operationalized as “less than 1 in 5 people on this site”.)
There are, at the very least, three prominent examples that spring to mind of people advocating something like “higher epistemic standards on LW”: Duncan, Said, and (if I might be so bold) myself. There are, moreover, a smattering of comments from less prolific commenters, most of whom seem to express agreement with Duncan’s OP. I do not think this is something that should be ignored, and I think the site may benefit from some kind of poll of its userbase, just to see exactly how much consensus there is on this.
(I recognize that the LW/Lightcone team may nonetheless choose to ignore the result of any such poll, and for the sake of clarity I wish to add that I do not view this as problematic. I do not, on the whole, think that the LW team should be reduced to a proxy that implements whatever the userbase thinks they want; my expectation is that this would produce worse long-term outcomes than if the team regularly exercised their own judgement, even if that judgement sometimes results in policy decisions that conflict with a substantial fraction of the userbase’s desires. Even so, however, I claim that information of the form “X% of LW users believe Y” is useful information to have, and will at the very least play a role in any kind of healthy decision-making process.)
I am generally in favor of people running polls and surveys about information they’re interested in.
(Here’s a very random one I did, and looking through search I see people have done them on general demographics, nootropics, existential risk, akrasia, and more.)
I’m pretty confused by your numbered list, because they seem directly in contradiction with how scientific journals have worked historically. Here’s a quote from an earlier post of mine:
I looked through a volume of the London Mathematical Society, in particular, the volume where Turing published his groundbreaking paper proving that not all mathematical propositions are decidable (thanks to sci-hub for making it possible for me to read the papers!). My eyes looked at about 60% of the pages in the journal (about 12 papers), and not one of them disagreed with any prior work. There was :
A footnote that thanked an advisor for finding a flaw in a proof
An addendum page (to the whole volume) that consisted of a single sentence thanking someone for showing one of their theorems was a special case of someone else’s theorem
One person who was skeptical of another person’s theorem. But that theorem by Ramanujan (who was famous for stating theorems without proofs), and the whole paper primarily found proofs of his other theorems.
There were lots of discussions of people’s work but always building, or extending, or finding a neater way of achieving the same results. Never disagreement, correction, or the finding of errors.
I think that many-to-most papers published in scientific journals are basically on unhelpful questions and add little to the field, and I’d bet some of the proofs are false. And yet it seems to be very rarely published that they’re unhelpful or wrong or even criticized in the journals. People build on the good content, and forget about the rest. And journals provide a publishing house for the best ideas at a given time. (Not too dissimilar to the annual LW Review.)
It seems to me that low-quality content is indeed pretty low cost if you have a good filtering mechanism for the best content, and an incentive for people to produce great content. I think on the margin I am interested in creating more of both — better filtering and stronger incentives. That is where my mind currently goes when I think of ways to improve LessWrong.
This seems like a very odd response given the fact of the replication crisis, the many cases of scientific knowledge being forgotten for decades after being discovered, the rise of false or inaccurate (and sometimes quite harmful!) models, etc.
I think that (in many, or most, scientific fields) people often don’t build on the good content, and don’t forget about the bad content; often, the reverse happens. It’s true that it’s “very rarely published that [bad papers/proofs are] unhelpful or wrong or even criticized in the journals”! But this is, actually, very bad, and is a huge reason why more and more scientific fields are being revealed to be full of un-replicable nonsense, egregious mistakes, and even outright fraud! The filtering mechanisms we have are actually quite poor.
The “papers in scientific journals” example / case study seems to me to yield clear, strong support for my view.
It’s very worthwhile to understand the ways in which academia has died over the last 60 years or so, and part of it definitely involves failures in the journal system. But the axis of public criticism in journals doesn’t seem at all to have been what changed in the last 60 years? Insofar as you think that’s a primary reason, you seem to be explaining a change by pointing to a variable that has not changed.
In replying to your proposed norms, it’s not odd to point out that the very mechanism of labeling everything that’s bad as bad and ensuring we have common knowledge of it, was not remotely present when science was at its most productive — when Turing was inventing his machines, or when Crick & Watson were discovering the structure of DNA. In fact it seems to have been actively opposed in the journal system, because you do not get zero criticism without active optimization for it. That is why it seems to me to be strong evidence against the system you propose.
There may be a system that works via criticizing everything bad in public, but when science was most successful it did not, and instead seems to me to be based around the system I describe (a lot of submissions and high reward for success, little punishment for failure).
As a note on balance, I think you addressed the (predicted) costs of raising standards but not the costs of the existing standards.
I know of at least three people that I believe Ben Pace would consider high-quality potential contributors who are not writing and commenting much on LessWrong because the comments are exhausting and costly in approximately the ways I’m gesturing at.
(We have some disagreement but more overlap than disagreement.)
And I myself am quite likely to keep adding essays but am extremely unlikely to be anything other than a thought-dropper, at the current level of you-can-get-away-with-strawmanning-and-projecting-and-bullying-and-various-other-violations-of-existing-nominal-norms-in-the-comments-and-it’ll-be-highly-upvoted.
I think that should be part of the weighing, too. Like, the cost of people feeling cringier is real, but so too is the cost of people who don’t even bother to show up, because they don’t feel safe doing so.
I also note that I strongly claim that a generally cleaner epistemic pool straightforwardly allows for more experimentation and exploration. If there’s more good faith in the atmosphere (because there’s less gunk causing people to correctly have their shields up) then it’s actually easier rather than harder to e.g. gesture vaguely or take shortcuts or make jokes or oblique connections.
This is in fact a major driving assumption—that LessWrong could potentially be more like e.g. conversations where Ben and Duncan are just talking, and don’t have to fear e.g. a mob of people adversarially interpreting our clumsy first-pass at expressing a thought and never letting it go.
because the comments are exhausting and costly in approximately the ways I’m gesturing at.
(We have some disagreement but more overlap than disagreement.)
As I understand Ben Pace, he’s saying something like “I want people to take more risks so that we find more gold”, and you’re replying with something like “I think people will take more risks if we make the space more safe, by policing things like strawmanning.”
It seems central to me to somehow get precise and connected to reality, like what specific rules you’re suggesting policing (strawmanning? projecting? Everything in the Sequences?), and maybe look at some historic posts and comments and figure out which bits you would police and which you wouldn’t. (I’m really not sure if this is in the ‘overlap’ space or the ‘disagreement’ space.)
Strong-upvote as well for the specificity request; the place where I most strongly expect attempts at “increasing standards” to fail is the point where people realize that broad agreement about direction does not necessarily translate to finer agreement about implementation, and I expect this is best avoided by sharing gears-level models as quickly and as early during the initial discussion as possible. As I wrote in another comment:
Finally, note that at no point have I made an attempt to define what, exactly, constitute “epistemic violations”, “epistemic standards”, or “epistemic hygiene”. This is because this is the point where I am least confident in my model of Duncan, and separately where I also think his argument is at its weakest. It seems plausible to me that, even if [something like] Duncan’s vision for LW were to be realized, there would be still be substantial remaining disagreement about how to evaluate certain edge cases, and that that lack of consensus could undermine the whole enterprise.
(Though my model of Duncan does interject in response to this, “It’s okay if the edge cases remain slightly blurry; those edge cases are not what matter in the vast majority of cases where I would identify a comment as being epistemically unvirtuous. What matters is that the central territory is firmed up, and right now LW is doing extremely poorly at picking even that low-hanging fruit.”)
((At which point I would step aside and ask the real Duncan what he thinks of that, and whether he thinks the examples he picked out from the Leverage and CFAR/MIRI threads constitute representative samples of what he would consider “central territory”.))
I note that I’ve already put something like ten full hours into creating exactly these types of examples, and that fact sort of keeps getting ignored/people largely never engage with them.
Perhaps you are suggesting a post that does that-and-nothing-but-that?
Perhaps you are suggesting a post that does that-and-nothing-but-that?
I think I am suggesting “link to things when you mention them.” Like, if I want to argue with DanielFilan about whether or not a particular garment “is proper” or not, it’s really not obvious what I mean, whereas if I say “hey I don’t think that complies with the US Flag Code”, most of the work is done (and then we figure out whether or not section j actually applies to the garment in question, ultimately concluding that it does not).
Like, elsewhere you write:
The standard is: don’t violate the straightforward list of rationality 101 principles and practices that we have a giant canon of knowledge and agreement upon.
I currently don’t think there exists a ‘straightforward list of rationality 101 principles and practices’ that I could link someone to (in the same way that I can link them to the Flag Code, or to literal Canon Law). Like, where’s the boundary between rationality 101 and rationality 102? (What fraction of rationality 101 do the current ‘default comment guidelines’ contain?)
Given the absence of that, I think you’re imagining much more agreement than exists. Some like the “Double crux” style, but Said disliked it back in 2018 [1][2] and presumably feels the same way now. Does that mean it’s in the canon, like you suggest in this comment, or not?
[Edit: I recall that at some point, you had something that I think was called Sabien’s Rules? I can’t find it with a quick search now, but I think having something like that which you can easily link to and people can either agree with or disagree with will clarify things compared to your current gesturing at a large body of things.]
So yes, Said, I am broadly opposed to substantially increasing the standards applied to each individual comment or paragraph. I am much more in favor of raising the amount of reward you can get for putting in remarkable amounts of effort and contributing great insights and knowledge.
After finishing writing, I did have a further note to add on where I actually think I am more open to raising standards.
As well as rewarding people more for their entire body of contributions to LessWrong, I am also more open to negatively judging people more for their entire body of contributions to LessWrong. Compare two users: one who writes a couple of 100+ karma posts per year but who also has occasional very snarky and rude comments, versus one who never writes snarky comments but always kind of doesn’t understand the dialogue and muddies the waters, and produces 100 comments each year. I think the latter has far more potential to be costly for the site, and the former has the potential to be far more valuable for the site, even though the worst comments of the former are much worse than the worst comments of the latter.
To state the obvious, using the risk/reward frame above, I think just punishing people more for not doing their practice would result in far fewer great contributions to the site. But I think it’s very promising to reward people more for putting in very high levels of effortinto practice, by celebrating them and making their achievements legible and giving them prizes. I suspect that this could change the site culture substantially.
There was the issue with the babble challenges where I felt like effort was not being seen. “Not knowing which norms are materially important feels capricious.”. There is a difference between giving a prize to a valued act and giving valued acts prizes. While it was not a total unmitigated catastrophe I became wary and became suspicious of claims like “hey if you do X I will do Y”.
Yeah that seems fair. I gave feedback to Jacob at the time that his interpretation of the rules didn’t seem like the obvious one to me, and I think the ‘streak’ framing also meant that missing one week took you down to zero, which is super costly if it’s the primary success metric.
7⁄7 attendance and 6⁄7 success resulted in 5 stars. I think the idea was that high cost of missing out would utilise sunk cost to keep the activity going. I am not sur whether bending on rules made it closer to idela or would sticking by the lines and making a fail a full reset done better. Or even if the call between pass and fail was compromised by allowing “fail with reduced concequences”.
Ironically, I considered posting a comment to the effect that I disagree with some parts of your description of “what constitutes adhering to the standards”, but reasoned that it’s rather a moot point until and unless the meta-level issues are resolved…
(I will note that while I do agree that it tends to be possible to go up a meta level, I also think there’s a limit to how much real progress can be made that way. But this is itself already too meta, so let’s table it for now.)
Back to the point: if the Less Wrong team is still opposed to the general approach of “let’s substantially raise our standards, and if that makes many current members leave or go quiet (as surely will happen), well… so be it [or maybe even: ‘good!’]”, then isn’t the answer to “what is to be done?” necessarily “nothing, because nothing is permitted to be done”? In other words, don’t we actually already know that “LW isn’t going to be that place”?
Or is it just that this post is mostly aimed at the LW team, and intended to shift their views (so that they change their as-practiced policy w.r.t. intellectual standards)?
I assumed this post was mostly aimed at the LW team (maybe with some opportunity for other people to weigh in). I think periodically posting posts dedicated to arguing the moderation policy should change is fine and good.
Worth noting that I’ve had different disagreements you with and Duncan. In both cases I think the discussion is much subtler than “increase standards: yay/nay?”. It matters a lot which standards, and what they’re supposed to be doing, and how different things trade off against each other.
Worth noting that I’ve had different disagreements you with and Duncan. In both cases I think the discussion is much subtler than “increase standards: yay/nay?”. It matters a lot which standards, and what they’re supposed to be doing, and how different things trade off against each other.
Yes, yes, that’s all fine, but the critical point of contention here is (and previously has been) the fact that increasing standards (in one way or another) would result in many current participants leaving. To me, this is fine and even desirable. Whereas the LW mod team has consistently expressed their opposition to this outcome…
I think this is a fair summary of what we said years ago. I’m not sure how much people’s minds have changed on the issue. I think Ben Pace’s warning of Said (you have to scroll to the bottom of the page and then expand that thread) and the related comments are probably the place to look, including habryka’s comment here.
Before checking the moderation list, the posts that come to mind (as a place to start looking for this sort of conversation) were Kensho (where I think a lot of the mods viewed Said as asking the right questions) and Meta-Discussion from Circling as a Cousin to Rationality (where I viewed Said as asking the right questions, and I think others didn’t).
I feel a little shy about the word “aimed” … I think that I have aimed posts at the LW team before (e.g. my moderating LessWrong post) but while I was happy and excited about the idea of you guys seeing and engaging with this one, it wasn’t a stealth message to the team. It really was meant for the broader LW audience to see and have opinions on.
I do not have specific insider knowledge, no.
I do know that you and I have clashed before on what constitutes adhering to the standards, though I believe (as I believe you believe) that in such situations it should be possible to simply take the conversation up a meta level.
On the topic of standards
On the current margin, I am interested in users taking more risks with individual comments and posts, not less. People take more risk when the successes are rewarded more, and when the failures are punished less. I generally encourage very low standards for individual comments, similar to how I have very low standards for individual word-choice or sentence structure. I want to reward or punish users for their body of contributions rather than pick each one apart and make sure it’s “up to standard”. (As an example, see the how this moderation notice is framed as a holistic evaluation of a user’s contributions, not about a single comment.)
So yes, Said, I am broadly opposed to substantially increasing the standards applied to each individual comment or paragraph. I am much more in favor of raising the amount of reward you can get for putting in remarkable amounts of effort and contributing great insights and knowledge. I think your support of Eliezer by making readthesequences.com and your support of Gwern with the site re-design are examples of the kind of things I think will make people like Eliezer and Gwern feel like their best writing is rewarded, rather than increased punishment for their least good comments and posts.
I really don’t care if someone ‘misses the mark’ most of the time, if they succeed the few times required on the path to greatness. Users like John Wentworth and Alex Flint produce lots of posts that receive relatively low karma, and also a (smaller) number of ‘hits’ that are some of my favorite site-content in recent years. I think if they attempted to write in a way that meant the bottom 80% of posts (by karma) didn’t get published, then they would not feel comfortable risking the top 20% of posts, as it wouldn’t be clear to them that they would meet the new standard, and it wouldn’t be worth the risk.
I think one of the successes of LW 2.0 has been in reducing the pain associated with having content not be well received, while also increasing the visibility and reward for contributions that are excellent (via frontpage karma-date-weighted sorting and by curation and published books and more).
″...back in my day physics classes gave lots of hard problems that most students couldn’t do. So there was a lot of noise in particular grades, and students cared as much or more about possibly doing unusually well as doing unusually badly. One stellar performance might make your reputation, and make up for lots of other mediocre work. But today, schools give lots of assignments where most get high percent scores, and even many where most get 100% scores. In this sort of world, students know it is mostly about not making mistakes, and avoiding black marks. There is otherwise little they can do to stand out...the new focus is on the low, not the high, end of the distribution of outcomes for each event or activity.”
—Robin Hanson, What is ‘Elite Overproduction’?, August 2021
On the topic of LessWrong as a place to practice rationality
Duncan, it seems to me that you want LessWrong to be (in substantial part) a dojo, where we are together putting in effort to help each other be stronger in our epistemic and instrumental processes.
I love this idea. One time where something like this happened was Jacob’s babble challenges last year, where he challenges users to babble 100 ways to solve a given problem: 1, 2, 3, 4, 5, 6, 7. Loads of people got involved, and I loved it. There’s been a few other instances where lots of people tried something difficult, like under Alkjash’s Hammertime Final Exam and Scott Garrabrant’s Fixed Point Exercises. Also lsusr’s The Darwin Game had this energy, as did the winners of my Rationality Exercises Prize.
The above posts are strong evidence to me that people want something like a rationality dojo on LessWrong, a place to practice and become stronger. I think there is a space for this to grow on LessWrong.
To state the obvious, using the risk/reward frame above, I think just punishing people more for not doing their practice would result in far fewer great contributions to the site. But I think it’s very promising to reward people more for putting in very high levels of effort into practice, by celebrating them and making their achievements legible and giving them prizes. I suspect that this could change the site culture substantially.
As Duncan know better than most any other person I’ve met, you don’t teach just by explaining, and I think there’s real potential for people on LW to practice as well. I’d be quite excited about a world where Duncan builds on the sorts of threads that Jacob and others have made, making rationality exercises and tests for people to practice together on LW, and building up a school of people over the years who gain great skill and produce ambitiously successful projects.
Duncan, it seems on the table to me that you think there’s some promise to doing this too, here on LessWrong. Do you think you’d be up for trying something like the threads above (but with your own flavor)? I’m happy to babble ideas together with you this Friday :)
Re: standards:
What you say makes sense if, and only if, the presence of “bad” content is costless.
And that condition has (at least) these prerequisites:
Everyone (or near enough) clearly sees which content is bad; everyone agrees that the content is bad, and also on what makes it bad; and thus…
… the bad content is clearly and publicly judged as such, and firmly discarded, so that…
… nobody adopts or integrates the bad ideas from the bad content, and nobody’s reasoning, models, practices, behavior, etc. is affected (negatively) by the bad content; and relatedly…
… the bad content does not “crowd out” the good content, bad ideas from it do not outcompete opposing good ideas on corresponding topics, the bad ideas in the bad content never become the consensus views on any relevant subjects, and the bad reasoning in the bad content never affects the norms for discussion (of good content, or of anything) on the site (e.g., is never viewed by newcomers, taken to be representative, and understood to be acceptable).
If, indeed, these conditions obtain, then your perspective is eminently reasonable, and your chosen policy almost certainly the right one.
But it seems very clear to me that these conditions absolutely do not obtain. Every single thing I listed above is, in fact, entirely false, on Less Wrong.
And that means that “bad” content is far from costless. It means that such content imposes terrible costs, in fact; it means that tolerating such content means that we tolerate the corrosion of our ability to produce good content—which is to say, our ability to find what is true, and to do useful things. (And when I say “our”, I mean both “Less Wrong’s, collectively” and “the participants’, individually”.)
(Unlike your comment, which is, commendably, rife with examples, you’ll note that my reply provides no examples at all. This is intentional; I have little desire to start a fight, as it were, by “calling out” any posters or commenters. I will provide examples on request… but I suspect that anyone participating in this conversation will have little trouble coming up with more than a few examples, even without my help.)
“Iff” is far too strong. I agree that the “if” claim holds. However, I think that what Ben says also makes sense if the bad/high-variance content has costs which are less than its benefits. Demanding costlessness imposes an unnecessarily high standard on positions disagreeing with your own, I think.
Contrasting your position with Ben’s, I sense a potential false dichotomy. Must it be true that either we open the floodgates and allow who-knows-what on the site in order to encourage higher-variance moves, or we sternly allow only the most well-supported reasoning? I think not. What other solutions might be available?
The first—but surely not best—to come to mind is the curation < LW review < ??? pipeline, where posts are subjected to increasing levels of scrutiny and rewarded with increasing levels of visibility. Perhaps there might be some way for people to modulate “how much they update on a post” by “the amount of scrutiny the post has received.” I don’t think this quite fights the corrosion you point at. But it seems like something is possible here, and in any case it seems to me too early to conclude there is only one axis of variation in responses to the situation (free-wheeling vs strict).
Re: other solutions:
I have repeatedly suggested/advocated the (to me, fairly obvious) solution where (to summarize / crystallize my previous commentary on this):
People post things on their personal LW blogs. Post authors have moderation powers on their personal-blog posts.
Things are posted to the front page only if (but not necessarily “if”!) they are intended to be subject to the sort of scrutiny wherein we insist that posts live up to non-trivial epistemic/etc. standards (with attendant criticism, picking-apart, analysis, etc.; and also with attendant downvotes for posts judged to be bad). Importantly, post authors do not have moderation powers in this case, nor the ability to decide on moderation standards for comments on their posts. (In this case a post might be front-paged by the author, or, with the author’s consent, by the mods.)
Posts that go to the front page, are evaluated by the above-described process, and judged to be unusually good, may be “curated” or what have you.
In this case, it would be proper for the community to judge personal-blog posts, that have not been subjected to “frontpage-level” scrutiny, as essentially ignorable. This would go a long way toward ensuring that posts of the “jam-packed with bullshit” type (which would either be posted to personal blogs only, or would go to the front page and be mercilessly torn apart, and clearly and publicly judged to be poor) would be largely costless.
I agree with you that this sort of setup would not quite solve the problem, and also that it would nonetheless improve the situation markedly.
But the LW team has consistently been opposed to this sort of proposal.
It sounds to me like posting on your High-Standards-Frontpage is a very high effort endeavor, an amount of effort that currently only around 3-30 posts each year have put into them. I’ve thought of this idea before with the name “LW Journal” or “LW Peer Review”, which also had a part where it wasn’t only commenters critiquing your post, but we paid a few people full-time for reviewing of the posts in this pipeline, and there was also a clear pass/failure with each submission. (Scott Garrabrant has also suggested this idea to me in the past, as a publishing place for his papers.)
I think the main requirement I see is a correspondingly larger incentive to write something that passes this bar. Else I mostly expect the same fate to befall us as with LW 1.0, where Main became increasingly effortful and unpleasant for authors to post to, such that writers like Scott Alexander moved away to writing on their personal blogs.
(I’m generally interested to hear ideas for what would be a big reward for writers to do this sort of thing. The first ones that come to my mind are “money” and “being published in physical books”.)
I do think that something like this would really help the site in certain ways; I think a lot of people have a hard time figuring out what standard to hold their posts to, and having a clearly “high standard” and “lower standard” place would help authors feel more comfortable knowing what they’re aiming for in their writing. (“Shortform” was an experiment with a kind of lower-standards place.) But I don’t currently see a simple way to cause a lot of people to produce high-effort high-standards content for that part of the site, beyond the amount of effort we currently receive on the highest effort posts each year.
So I think the Review is pretty good at getting good old content, but I think the thing Said is talking about should happen more quickly, and should be more like Royal Society Letters or w/e.
Actually, I wonder about Rohin’s newsletters as a model/seed. They attract more scrutiny to things, but they come with the reward of Rohin’s summary (and, presumably, more eyeballs than it would have gotten on its own). But also people were going to be writing those things for their own reasons anyway.
I think if we had the Eliezer-curated weekly newsletter of “here are the LW posts that caught my interest plus commentary on them”, we would probably think the reward and scrutiny were balanced. Of course, as with any suggestion that proposes spending Eliezer-time on something, I think this is pretty dang expensive—but the Royal Society Letters were also colossally expensive to produce.
I would likely do this from my own motivation (i.e. not necessarily need money) if I were given at least one of:
a) guaranteed protection from the badgunk comments by e.g. three moderators willing to be dependably high-effort down in the comments
b) given the power to hide badgunk comments pending their author rewriting them to eliminate the badgunk
c) given the power to leave inline commentary on people’s badgunk comments
The only thing holding me back from doing something much more like what Said proposes is “LW comment sections regularly abuse and exhaust me.” Literally that’s the only barrier, and it’s a substantial one. If LW comment sections did not regularly abuse and exhaust me, such that every post feels like I need to set aside fifty hours of life and spoons just in case, then I could and would be much more prolific.
(To be clear: some people whose pushback on this post was emphatically not abuse or exhausting include supposedlyfun, Said, Elizabeth, johnswentsworth, and agrippa.)
Would you accept this substitute:
“A site/community culture where other commenters will reliably ‘call out’ (and downvote) undesirable comments, and will not be punished for doing so (and attempts to punish them for such ‘vigilante-style’ a.k.a. ‘grassroots’ ‘comment policing’ will themselves be punished—by other commenters, recursively, with support from moderators if required).”
Yes, absolutely. Thanks for noting it. That substitute is much more what the OP is pushing for.
EDIT: With the further claim that, once such activity is reliable and credible, its rate will also decrease. That standards, clearly held and reliably enforced, tend to beget fewer violations in the first place, and that, in other words, I don’t think this would be a permanent uptick in policing.
Strong-upvote. I want to register that I have had disagreements with Said in the past about this, and while I am still not completely sure whether I agree with his frame, recent developments have in fact caused me to update significantly towards his view.
I suspect this is true of others as well, such that I think Said’s view (as well as associated views that may differ in specifics but agree in thrust) can no longer be treated as the minority viewpoint. (They may still be the minority view, but if so I don’t expect it to be a small minority anymore, where “small” might be operationalized as “less than 1 in 5 people on this site”.)
There are, at the very least, three prominent examples that spring to mind of people advocating something like “higher epistemic standards on LW”: Duncan, Said, and (if I might be so bold) myself. There are, moreover, a smattering of comments from less prolific commenters, most of whom seem to express agreement with Duncan’s OP. I do not think this is something that should be ignored, and I think the site may benefit from some kind of poll of its userbase, just to see exactly how much consensus there is on this.
(I recognize that the LW/Lightcone team may nonetheless choose to ignore the result of any such poll, and for the sake of clarity I wish to add that I do not view this as problematic. I do not, on the whole, think that the LW team should be reduced to a proxy that implements whatever the userbase thinks they want; my expectation is that this would produce worse long-term outcomes than if the team regularly exercised their own judgement, even if that judgement sometimes results in policy decisions that conflict with a substantial fraction of the userbase’s desires. Even so, however, I claim that information of the form “X% of LW users believe Y” is useful information to have, and will at the very least play a role in any kind of healthy decision-making process.)
I am generally in favor of people running polls and surveys about information they’re interested in.
(Here’s a very random one I did, and looking through search I see people have done them on general demographics, nootropics, existential risk, akrasia, and more.)
I’m pretty confused by your numbered list, because they seem directly in contradiction with how scientific journals have worked historically. Here’s a quote from an earlier post of mine:
I think that many-to-most papers published in scientific journals are basically on unhelpful questions and add little to the field, and I’d bet some of the proofs are false. And yet it seems to be very rarely published that they’re unhelpful or wrong or even criticized in the journals. People build on the good content, and forget about the rest. And journals provide a publishing house for the best ideas at a given time. (Not too dissimilar to the annual LW Review.)
It seems to me that low-quality content is indeed pretty low cost if you have a good filtering mechanism for the best content, and an incentive for people to produce great content. I think on the margin I am interested in creating more of both — better filtering and stronger incentives. That is where my mind currently goes when I think of ways to improve LessWrong.
This seems like a very odd response given the fact of the replication crisis, the many cases of scientific knowledge being forgotten for decades after being discovered, the rise of false or inaccurate (and sometimes quite harmful!) models, etc.
I think that (in many, or most, scientific fields) people often don’t build on the good content, and don’t forget about the bad content; often, the reverse happens. It’s true that it’s “very rarely published that [bad papers/proofs are] unhelpful or wrong or even criticized in the journals”! But this is, actually, very bad, and is a huge reason why more and more scientific fields are being revealed to be full of un-replicable nonsense, egregious mistakes, and even outright fraud! The filtering mechanisms we have are actually quite poor.
The “papers in scientific journals” example / case study seems to me to yield clear, strong support for my view.
It’s very worthwhile to understand the ways in which academia has died over the last 60 years or so, and part of it definitely involves failures in the journal system. But the axis of public criticism in journals doesn’t seem at all to have been what changed in the last 60 years? Insofar as you think that’s a primary reason, you seem to be explaining a change by pointing to a variable that has not changed.
In replying to your proposed norms, it’s not odd to point out that the very mechanism of labeling everything that’s bad as bad and ensuring we have common knowledge of it, was not remotely present when science was at its most productive — when Turing was inventing his machines, or when Crick & Watson were discovering the structure of DNA. In fact it seems to have been actively opposed in the journal system, because you do not get zero criticism without active optimization for it. That is why it seems to me to be strong evidence against the system you propose.
There may be a system that works via criticizing everything bad in public, but when science was most successful it did not, and instead seems to me to be based around the system I describe (a lot of submissions and high reward for success, little punishment for failure).
As a note on balance, I think you addressed the (predicted) costs of raising standards but not the costs of the existing standards.
I know of at least three people that I believe Ben Pace would consider high-quality potential contributors who are not writing and commenting much on LessWrong because the comments are exhausting and costly in approximately the ways I’m gesturing at.
(We have some disagreement but more overlap than disagreement.)
And I myself am quite likely to keep adding essays but am extremely unlikely to be anything other than a thought-dropper, at the current level of you-can-get-away-with-strawmanning-and-projecting-and-bullying-and-various-other-violations-of-existing-nominal-norms-in-the-comments-and-it’ll-be-highly-upvoted.
I think that should be part of the weighing, too. Like, the cost of people feeling cringier is real, but so too is the cost of people who don’t even bother to show up, because they don’t feel safe doing so.
I also note that I strongly claim that a generally cleaner epistemic pool straightforwardly allows for more experimentation and exploration. If there’s more good faith in the atmosphere (because there’s less gunk causing people to correctly have their shields up) then it’s actually easier rather than harder to e.g. gesture vaguely or take shortcuts or make jokes or oblique connections.
This is in fact a major driving assumption—that LessWrong could potentially be more like e.g. conversations where Ben and Duncan are just talking, and don’t have to fear e.g. a mob of people adversarially interpreting our clumsy first-pass at expressing a thought and never letting it go.
As I understand Ben Pace, he’s saying something like “I want people to take more risks so that we find more gold”, and you’re replying with something like “I think people will take more risks if we make the space more safe, by policing things like strawmanning.”
It seems central to me to somehow get precise and connected to reality, like what specific rules you’re suggesting policing (strawmanning? projecting? Everything in the Sequences?), and maybe look at some historic posts and comments and figure out which bits you would police and which you wouldn’t. (I’m really not sure if this is in the ‘overlap’ space or the ‘disagreement’ space.)
Strong-upvote as well for the specificity request; the place where I most strongly expect attempts at “increasing standards” to fail is the point where people realize that broad agreement about direction does not necessarily translate to finer agreement about implementation, and I expect this is best avoided by sharing gears-level models as quickly and as early during the initial discussion as possible. As I wrote in another comment:
I note that I’ve already put something like ten full hours into creating exactly these types of examples, and that fact sort of keeps getting ignored/people largely never engage with them.
Perhaps you are suggesting a post that does that-and-nothing-but-that?
I think I am suggesting “link to things when you mention them.” Like, if I want to argue with DanielFilan about whether or not a particular garment “is proper” or not, it’s really not obvious what I mean, whereas if I say “hey I don’t think that complies with the US Flag Code”, most of the work is done (and then we figure out whether or not section j actually applies to the garment in question, ultimately concluding that it does not).
Like, elsewhere you write:
I currently don’t think there exists a ‘straightforward list of rationality 101 principles and practices’ that I could link someone to (in the same way that I can link them to the Flag Code, or to literal Canon Law). Like, where’s the boundary between rationality 101 and rationality 102? (What fraction of rationality 101 do the current ‘default comment guidelines’ contain?)
Given the absence of that, I think you’re imagining much more agreement than exists. Some like the “Double crux” style, but Said disliked it back in 2018 [1] [2] and presumably feels the same way now. Does that mean it’s in the canon, like you suggest in this comment, or not?
[Edit: I recall that at some point, you had something that I think was called Sabien’s Rules? I can’t find it with a quick search now, but I think having something like that which you can easily link to and people can either agree with or disagree with will clarify things compared to your current gesturing at a large body of things.]
Sabien’s Sins is linked in the OP (near the end, in the list of terrible ideas).
I will probably make a master linkpost somewhere in my next four LW essays. Thanks.
Where? Is it the quoted lines?
Huh, not sure how I missed that; thanks for pointing it out.
Indeed, my opinion of “double crux” has not improved since the linked comments were written.
After finishing writing, I did have a further note to add on where I actually think I am more open to raising standards.
As well as rewarding people more for their entire body of contributions to LessWrong, I am also more open to negatively judging people more for their entire body of contributions to LessWrong. Compare two users: one who writes a couple of 100+ karma posts per year but who also has occasional very snarky and rude comments, versus one who never writes snarky comments but always kind of doesn’t understand the dialogue and muddies the waters, and produces 100 comments each year. I think the latter has far more potential to be costly for the site, and the former has the potential to be far more valuable for the site, even though the worst comments of the former are much worse than the worst comments of the latter.
There was the issue with the babble challenges where I felt like effort was not being seen. “Not knowing which norms are materially important feels capricious.”. There is a difference between giving a prize to a valued act and giving valued acts prizes. While it was not a total unmitigated catastrophe I became wary and became suspicious of claims like “hey if you do X I will do Y”.
Yeah that seems fair. I gave feedback to Jacob at the time that his interpretation of the rules didn’t seem like the obvious one to me, and I think the ‘streak’ framing also meant that missing one week took you down to zero, which is super costly if it’s the primary success metric.
7⁄7 attendance and 6⁄7 success resulted in 5 stars. I think the idea was that high cost of missing out would utilise sunk cost to keep the activity going. I am not sur whether bending on rules made it closer to idela or would sticking by the lines and making a fail a full reset done better. Or even if the call between pass and fail was compromised by allowing “fail with reduced concequences”.
Ironically, I considered posting a comment to the effect that I disagree with some parts of your description of “what constitutes adhering to the standards”, but reasoned that it’s rather a moot point until and unless the meta-level issues are resolved…
(I will note that while I do agree that it tends to be possible to go up a meta level, I also think there’s a limit to how much real progress can be made that way. But this is itself already too meta, so let’s table it for now.)
Back to the point: if the Less Wrong team is still opposed to the general approach of “let’s substantially raise our standards, and if that makes many current members leave or go quiet (as surely will happen), well… so be it [or maybe even: ‘good!’]”, then isn’t the answer to “what is to be done?” necessarily “nothing, because nothing is permitted to be done”? In other words, don’t we actually already know that “LW isn’t going to be that place”?
Or is it just that this post is mostly aimed at the LW team, and intended to shift their views (so that they change their as-practiced policy w.r.t. intellectual standards)?
I assumed this post was mostly aimed at the LW team (maybe with some opportunity for other people to weigh in). I think periodically posting posts dedicated to arguing the moderation policy should change is fine and good.
Worth noting that I’ve had different disagreements you with and Duncan. In both cases I think the discussion is much subtler than “increase standards: yay/nay?”. It matters a lot which standards, and what they’re supposed to be doing, and how different things trade off against each other.
Yes, yes, that’s all fine, but the critical point of contention here is (and previously has been) the fact that increasing standards (in one way or another) would result in many current participants leaving. To me, this is fine and even desirable. Whereas the LW mod team has consistently expressed their opposition to this outcome…
Would you mind quoting, anonymized if necessary? Mostly because I’m curious whether the summary will seem to me actually match the words.
I think this is a fair summary of what we said years ago. I’m not sure how much people’s minds have changed on the issue. I think Ben Pace’s warning of Said (you have to scroll to the bottom of the page and then expand that thread) and the related comments are probably the place to look, including habryka’s comment here.
Before checking the moderation list, the posts that come to mind (as a place to start looking for this sort of conversation) were Kensho (where I think a lot of the mods viewed Said as asking the right questions) and Meta-Discussion from Circling as a Cousin to Rationality (where I viewed Said as asking the right questions, and I think others didn’t).
Sure, I will see if I can locate some of the threads I have in mind. It may take me a day or several before I’ve got the time to do it, though.
Seems low-urgency to me.
I feel a little shy about the word “aimed” … I think that I have aimed posts at the LW team before (e.g. my moderating LessWrong post) but while I was happy and excited about the idea of you guys seeing and engaging with this one, it wasn’t a stealth message to the team. It really was meant for the broader LW audience to see and have opinions on.