The (Unofficial) Less Wrong Comment Challenge
Feedback is important. It’s true in Machine Learning, and it’s true in writing. So when you write a cool new post on LW, you generally hope for feedback. Positive feedback, fingers crossed, but good negative feedback is also very nice for improving your writing chops and your ideas. Yet getting meaningful feedback can prove difficult on LW. This Challenge aims at starting a community trend to solve this problem.
First, I need to frame the issue. How do you get feedback on LW? Through two mechanisms: karma and comments.
High karma (let’s say above 50) show that people find your post valuable. Maybe they even find your post clear and well-written; they might even agree with you. It would be cool to have comments, but you still have valuable feedback without any.
Low karma (The “Low Karma” filter is for less than −10, but I’ll count any negative karma) show that people actively disagree with either your idea or your way of presenting it. That’s a useful signal, but one that really warrants comment to understand what people disliked enough to downvote you.
Comments (let’s say at least 3 comments/comment threads from 3 different authors, all concerning the actual subject of the post instead of some other thing mentioned in the post), regardless of the karma, show an involvement of readers. Once again, they can be positive or negative; what really matters is that they interact with the subject of the post.
Okay. But what happens when none of these criteria is met? Posts with karma between 0 and 49 and less than 3 comments are not that rare: looking at last week’s frontpage posts (from Monday to Sunday my time, CET), there are 18 such posts out of 47, which is about 38%. Not a majority, but still more than a third. And I’m pretty sure many of the posts that didn’t make the cut could benefit from more feedback.
This is why I’m proposing the (Unofficial) Less Wrong Comment Challenge: committing to read and comment all posts in a given category, or by a given author, or with a given tag, in a specified period. The simplest example would be something like: read and comment every new frontpage post on LW for the rest of November. But the parameters allow for tweaks. For example, maybe you’re free this week, but not after. And maybe you don’t feel like you have the level for reading Alignment Forum posts. In that context, you could challenge yourself to read and comment every frontpage post not from the AF for the rest of the week. Or maybe you’re into a specific author, and you want to read and comment all of their posts as they go along.
An obvious response to this challenge is that you don’t know what to comment. Which completely overlooks the fact that half, maybe more, of the feedback signal comes from how much did you enjoy the post, found it interesting, and everything else in your experience of reading this post. No one is unqualified to share their experience. Just a “I really enjoyed this post, even if I had nothing else to say” is useful feedback. So is “I’m not sure why, but I had trouble connecting with the arguments of the post”.
So you have it. This is the (Unofficial) Less Wrong Comment Challenge. I don’t add any constraints on when you need to finish reading the posts of the day, because you’ll probably need your own, if only for timezone reasons. If you’re interested, just add a comment to this post with your own challenge, and maybe updates in the corresponding thread.
Disclaimer: this idea came to me because I’m unsatisfied with the level of feedback of my own posts. But instead of finding a solution just for me, I decided that pushing a new social norm, as hard as it was, might be more productive for the community. That being said, feel free to comment my posts, including this one.
My comment challenge:
I will read and comment every frontpage post on LW for the rest of November (starting today the 11th). The constraints for time are that I need to comment on everything posted and made to frontpage in the day (before midnight CET) before 2am CET the next day. That leaves me some time to read the late night (for me) posts.
I notice myself being excited about this general idea but wary of this implementation: this might cause people to post more, which might rapidly create a situation where you don’t actually have the bandwidth to comment on literally everything, and the perfect might become the enemy of the good.
It might be safer to commit to commenting on N posts per day (maybe always commenting on the post with the least comments from the past day/week/month, or something).
Thanks for the feedback and the worries! I actually already added caveats to my original challenge, including giving me a bit more time to comment. This is easier for me to implement than a policy about the posts with the least comments or karma, because these things change. And I also like the challenge of reading beyond what I would usually read on LW.
On the effect of my challenge, I somehow doubt that just having me comment more will make everyone post more. That being said, there are indeed more posts these last few days than for many previous days. I leave myself some margin to not comment on everything if there are 30+ frontpage posts per day, but I don’t think we’re there yet.
I’m adding
twothree caveats, to make this both useful and feasible:I don’t have to answer questions, because this is one context where I think that feedback without a point to make is not really valuable.
I don’t have to comment on Covid tagged posts, because I don’t want to.
I don’t have to comment on all posts by 2am CET, notably because some big complex posts can be posted really close to midnight CET, and that’s really too hard. So I should just read and comment all post for a given day before 11pm59 the next day.
Worth noting, you can comment on questions without answering them (the site treats them differently). I’ve found that most question posts contain enough information that can be meaningful responded to without attempting to answer the question
True. I think I’ll still keep questions as optional, because I do think that feedback without an answer is less useful than for regular posts.
Logging my failures, which might help in understand what is feasible and unfeasible in this challenge:
1st day (11th November): Failed to comment on The Darwin Game—Rounds 1 to 2, which is cool but I’m not following the game, so I have nothing to say about it.
3rd day (13th November): Failed to comment on A framework for thinking about single predictions, because I’m really not into forecasting, and don’t know enough to provide any kind of constructive feedback.
4th day (14th November): Failed to comment on Specialized Labor and Counterfactual Compensation because it’s a complex subject in which I know almost nothing about and which doesn’t interests me.
8th day (18th November): Failed to comment on Solomonoff Induction and Sleeping Beauty because I don’t have the background and the post doesn’t reexplain the main points necessary for me to get it, and failed to comment on Propinquity Cities So Far because I could not understand half the things I was reading—to much economics for my current level.
10th day (20th November): Failed to comment on Simpson’s paradox and the tyranny of strata because I didn’t know the paradox well enough to judge this post. One day late for commenting on Inner Alignment in Salt-Starved Rats because I was tired and busy.
12th day (22nd November): Failed to comment on It’s not economically inefficient for a UBI to reduce recipient’s employment because I had nothing relevant to add to the discussion, and it seemed like the kind of post where comments without content are a negative.
13th day to 17th day (23rd November to 27th November): Complete failure. I did comment some posts, but there is not a single day in this interval where I commented every post. Oops.
20th day (30th November): Half a failure. Missed 3⁄4 posts.
Ah, this is also good incentive for me to finish up some of the posts I’m working on before the end of the month! :)
Definitely! That would be an awesome consequence of this challenge!
I often feel like I have very little to contribute in a given discussion, so I typically don’t comment, but I will comment more, as this post both presents a cool community experiment & has caused me to update my estimate of all feedback’s value upwards. Also, I like posts like this which try to push the community of the forum in a new direction to see if it adds or subtracts value.
My comment challenge: I will comment on all front-page posts that I see & read that are not from AF, unless I see a disrupting decrease in my willingness to read posts.
That’s a most weasely statement! If you don’t know if it’s as important as they say, can you actually try commenting more or not? And if you can try, will you? And if you do try, will you succeed?
I interpret the quoted statement as “I am willing to make an effort that I don’t usually do, by commenting more, based on your assessment of the importance of giving feedback”, assuming good faith.
There’s an uncertainty, of course, as whether it will actually turn out important. “I can try” suggests they will try even if they don’t know, and we won’t know if they will succeed until they try.
Yes, you can interpret the statement in an uncharitable way with respect to their goodwill, but this is not what is, in my opinion, conducive to healthy comment sections in general.
That’s certainly the intended meaning, but the statement itself is in fact heavy with qualifiers. This fact doesn’t vary with goodwill.
I think the first of the qualifiers, “If what you say is true, I’ll act on what you say,” is actually a harmful antipattern, since it commits to action on the assumption that the statement is true while pretending to commit to action in case the statement is true, thus reserving doubt about the statement. It would be healthier to either commit to action on the assumption of doubt, or to refuse to commit because of doubt.
This makes sense, and I’ve updated the comment to reflect what I meant more accurately. Though I think the improvement is very minor, and your time could likely be spent on more important things than providing marginal improvements to LessWrong comments, I thank you none-the-less.
The original comment seems to have been edited to a sharper statement (thanks, D0TheMath), I hope it’s enough to clear up things.
I agree this qualifier pattern is harmful, in the context of collective action problems, when mutual trust and commitment has to be more firmly established. I don’t believe we’re in that context, hence my comment.
It’s important to practice habits when they are useless, or else you end up unpracticed when you need them. So I disagree that a (cheap) habit not being useful in some case is a reason to disregard lack of its use in that case.
I like this :) Look forward to seeing how it goes. I cannot participate myself, I am a bit overwhelmed with things at the minute, also I am following a (currently secret) challenge of my own for the next week makes this one in particular hard to do.
We discussed the topic of feedback with Adam. I approve of this challenge and will attempt to comment on at least half of all new posts from today to the end of November. Eventually renewing it if it works well.
I’ve been meaning to get out of mostly-lurking mode for months now, and this is as good of an opportunity as it gets.
I also want to mention the effect of “this comment could be a post”, which can help people “upgrade” from commenting, to shortform or longform, if they feel (like me), that there’s some quality bar to clear to feel comfortable posting more and get your ideas out there (hello, self-confidence issues).
You won’t get feedback if you don’t post somewhere anyways, and that could start with comments!
I wonder what could be done to make commenting more rewarding for people.
My current best answer is by giving clear guidelines on what kind of comments the author wants. I found myself feeling bad about commenting on my enjoyment of the post (and nothing else) sometimes, because I didn’t know whether such comments were valuable for the author.
I will expand on it in my post at the end of November about how my comment challenge went.
Some authors have stated their preferences on this (generally such comments are marginally positive but stating one specific thing you liked is a big improvement).
Thanks for the link! I am actually thinking about something that appears in or after the commenting guidelines. So that when you comment, you have that information.
I like this idea, selfishly, having two posts in the frontpage-[0-50]-uncommented category last week myself.
If anyone is confused seeing that my account has no posts… I just discovered that I accidentally have two accounts. This is me too: https://www.lesswrong.com/users/allswellthatsmaxwell
I was indeed wondering about it as I just read your first comment :D
For extra convenience you could even comment again with your alt account (wait, which is the main? Which is the alt? Does it matter?)
Here I am a third time, this time with extra convenience :). This is my main, since it has the actual posts. Gonna get rid of the other one… or something.
I also felt frustrated by lack of feedback my posts got, my response was to write this: https://www.lesswrong.com/posts/2E3fpnikKu6237AF6/the-case-for-a-bigger-audience Maybe submitting LW posts to targeted subreddits could be high impact?
LessWrong used to have a lot of comments back in the day. I wonder if part of the issue is simply that the number of posts went up, which means a bigger surfaces for readers to be spread across. Why did the writer/reader ratio go up? Perhaps because writing posts falls into the “endorsed” category, whereas reading/writing comments feels like “time-wasting”. And as CFAR et al helped rationalists be more productive, they let activities labeled as “time-wasting” fall by the wayside. (Note that there’s something rather incoherent about this: If the subject matter of the post was important enough to be worth a post, surely it is also worth reading/commenting?)
Anyway, here are the reasons why commenting falls into the “endorsed” column for me:
It seems neglected. See above argument.
I suspect people actually read comments a fair amount. I know I do. Sometimes I will skip to the comments before reading the post itself.
Writing a comment doesn’t trigger the same “officialness” anxiety that writing a post does. I don’t feel obligated to do background research, think about how my ideas should be structured, or try to anticipate potential lines of counterargument.
Taking this further, commenting doesn’t feel like work. So it takes fewer spoons. I’m writing this comment during a pre-designated goof off period, in fact. The ideal activity is one which is high-impact yet feels like play. Commenting and brainstorming are two of the few things that fall in that category for me.
I know there was an effort to move the community from Facebook to LW recently. Maybe if we pitched LW as “just as fun as Facebook, but discussing more valuable things and adding to a searchable/taggable knowledge archive” that could lure people over? IMO the concept of “work that feels like play” is underrated in the rationalist and EA communities.
Unfortunately, even though I find it fun to write comments, I tend to get demoralized a while later when my comments don’t get comment replies themselves :P So that ends up being an “endorsed” reason to avoid commenting.
This would increase the level of noise. A good change along these lines is for people who have something to say to speak up more frequently (as opposed to everyone speaking up more frequently, regardless of whether they have something nontrivial to say). But even that is potentially hazardous, since a norm suggesting an obligation to voice your thoughts makes reading more costly, possibly pushes readers away.
I think you’re pointing to all the probable issues with this sort of idea. But still, I think that this particular proposal deals with them:
About the level of noise: I agree that if every user posted a comment on everything, that might be too much. But we’re very far from that state. So this needs to work incredibly well to end up in this failure mode, at which point we will probably see it coming
Another point about noise: I don’t think that having feedback, even if only something like “I like this post”/”I disliked this post”/”I had trouble to care”, is noise.For me, this is incredibly important, and every person I talked with on this subject agreed. But I guess that’s a community decision to consider, and it’s more important to see what most people around here think than just me.
About feeling forced to voice your thoughts: this is why this challenge is completely customizable. You can say that you’ll try to write one comment a day, or even one every two days, or every week. So I don’t think it will actually push readers away, because they are in total control of their level of commitment. See for example what D0TheMath writes in his comment.
Whether people agree is beside the point. Is this actually true? To figure it out, it’s necessary to be more clear on what the statement means. In what way, specifically, is this kind of feedback important? If someone says “I like your post”, it’s very hard to learn anything in particular from that. If it’s customary to say things like that, some people would just say them out of misplaced general niceness, and there will be even less meaning in the utterance. (Also, there are upvotes/downvotes to indicate precisely that kind of feedback.) Some people like hearing that someone likes their posts, but that’s different from feedback being instructive. Noise can be pleasant without being enlightening. Finally, it can be motivating to hear that your work is appreciated.
It seems that it’s false that this kind of feedback is important for learning things, but true that it’s important for motivation of some authors. These are different claims that shouldn’t be mixed up. When you say “every person I talked with on this subject agreed”, do you (or they) know what exactly they agree with?
Unfortunately, norms don’t like nuance. This post is very far from igniting a norm, but if hypothetically what it suggests bears fruit, it will be in the form of a norm to comment more, and that norm will end up punishing defectors regardless of whether that was an intended feature of the norm or not.
The title contributes to a bad trend of calling things that are not challenges challenges.
How is it not a challenge? When I google ‘challenge’ the first result is “a call to take part in a contest or competition.” In this case it’s not exactly competitive, but it is a call to take part in something that’s not trivially easy to accomplish (i.e. somewhat challenging).
If a youtube channel would say “I challenege you to comment this video” that woudl be comparble challenge level.
While it can be techically correct there is a trend going on which could be a ask or recommendation, or anything other is stated as a challenge instead.
If I do this ask it is not like I win or that I feel I have achieved something. It is not a big thing but for whoever is concerned about semantics inflation this is how it looks like.
I disagree with your first sentence. Commenting on a single video is pretty trivially easy; committing to comment on every frontpage post whether you feel like it or not (or to sit down for an hour and actually try to generate fifty ideas) is definitely not comparably trivial. When you complete a babble challenge, do you seriously not feel like you’ve achieved something?
I did indeed missidentified what my thought was. It compares in my brain with a youtube video trying to trick me to engage, it is a parasitic rider meme.
It also feels exploitative in that feedback getter gets as benefit and I suffer inconvenience. You are right in that a babble challenge gives a sense of accomplishment. Here there is a sense of nebolousness that I predict signifcantly interferes with that process. Eating a spoon full of cinnamon or sugar or water or anything specific is not the cinnamon challenge.
Inspired by the post I gave signal to the direction of it. The significance of the feedback or did it succeed or not is not apparent at all to me. And it resulted in something that feels like arguing, being defensive about all the things the input isn’t, an emotional net negative. This feels like it doesn’t need to be so, but the reflex to shirk from commenting correctly projects that this is the expectation.