While I think LW’s epistemic culture is better than most, one thing that seems pretty bad is that occasionally mediocre/shitty posts get lots of upvotes simply because they’re written by [insert popular rationalist thinker].
Of course, if LW were truly meritocratic (which it should be), this shouldn’t matter — but in my experience, it descriptively does.
Without naming anyone (since that would be unproductive), I wanted to know if others notice this too? And aside from simply trying not to upvote something because it’s written by a popular author, anyone have good ideas for preventing this?
The effect seems natural and hard to prevent. Basically, certain authors get reputations for being high (quality * writing), and then it makes more sense for people to read their posts because both the floor and ceiling are higher in expectation. Then their worse posts get more readers (who vote) than posts of a similar quality by another author, who’s floor and ceiling is probably lower.
I’m not sure the magnitude of the cost, or that one can realistically expect to ever prevent this effect. For instance, ~all Scott Alexander blogposts get more readership than the best post by many other authors who haven’t built a reputation and readership, and this kind of just seems part of how the reading landscape works.
Of course, it can be frustrating as an author to sometimes see similar quality posts on LW get different karma. I think the answer here is to do more to celebrate the best posts by new authors. The main thing that comes to mind here is curation, where we celebrate and get more readership on the best posts. Perhaps I should also have a term here for “and this is a new author, so I want to bias toward curating them for the first time so that they’re more invested in writing more good content”.
Yes, but you’d naively hope this wouldn’t apply to shitty posts, just to mediocre posts. Like, maybe more people would read, but if the post is actually bad, people would downvote etc.
That’s right. One exception: sometimes I upvote posts/comments written to low standards in order to reward the discussion happening at all. As an example I initially upvoted Gary Marcus’s first LW post in order to be welcoming to him participating in the dialogue, even though I think the post is very low quality for LW.
(150+ karma is high enough and I’ve since removed the vote. Or some chance I am misremembering and I never upvoted because it was already doing well, in which case this serves as a hypothetical that I endorse.)
One thing you could do is give users relatively more voting power if they vote without seeing the author of the post. I.e., you can enable a mode which hides post authors until you give a vote on the anonymized content. After that, you can still vote like normal.
Obviously there are ways author identity can leak through this, but it seems better than nothing.
You should probably link some posts, it’s hard to discuss this so abstractly. And popular rationalist thinkers should be able to handle their posts being called mediocre (especially highly-upvoted ones).
This was also on my mind after seeing Jesse’s short form yesterday. Ryan’s “this is good” comment was above Louis’ thorough explanation of an alternative formal motivation for IFs. That would still be the case if I hadn’t heavy upvoted and weak downvoted.
I personally cast my comment up/downvotes as an expression of my preference ordering for visibility. I would encourage others to also do so. For instance, I suggest Ryan’s comment should’ve been agreement voted rather than upvoted by others. This stance has as a corollary to not vote if you haven’t read the other comments whose rankings you are affecting—or rather vote with any other marker of which LW has many.
This ‘upvotes as visibility preferences’ policy isn’t tractable for posts, so I suspect the solution there—if one is needed—would have to be done on the backend by normalization. Not sure whether this is worth attempting.
A salient example to me: This post essentially consists of Paul briefly remarking on some mildly interesting distinctions about different kinds of x-risks, and listing his precise credences without any justification for them. It’s well-written for what it aims to be (a quick take on personal views), but I don’t understand why this post was so strongly celebrated.
Of course, if LW were truly meritocratic (which it should be), this shouldn’t matter — but in my experience, it descriptively does.
That’s not really true if people are popular rationalist thinkers because of skill at good rationalist writing. Meritocracy does not imply that people get judged based on individual pieces of their work, a meritocracy where people are primarily judged on their total output would still be a meritocracy.
I think the problem is more that post that make points that are popular and fit neatly into the world view of the reader are more likely to get upvotes than posts that challenge the world view of the reader and would require the reader to update their world view. Jimmy’s introductionary post to his sequence is currently as I’m writing it at 12 karma.
While writing quality might be improved, it’s a post that sets out to challenges the readers conception about how human reasoning works in practical contexts and that’s why it’s at low karma. I would love to see more posts like Jimmy that are about the core of how rationality works over posts that feel good to read and that get lot of upvotes without really changing minds much.
I think if someone is very well-known their making a particular statement can be informative in itself, which is probably part of the reason it is upvoted.
I had similar thoughts. And indeed I found funny that a forum of rationalists falls for this.
But this is human nature, and we have finite time and energy to focus on things.
You will find the same for the attention papers written by people at famous institutions get, even if low quality.
Making names, recently, I had a similar feeling when looking at “AI as Normal Technology”. I did not know the authors, and I was skimming it because someone talked about it.
It is also a reason for which sometimes celebrities are called to do propaganda.
A strategy would be to make make posts anonymous for a limited amount of time (perhaps with some text randomization that would slightly change the style of the author/s, even though this might affect quality).
While I think LW’s epistemic culture is better than most, one thing that seems pretty bad is that occasionally mediocre/shitty posts get lots of upvotes simply because they’re written by [insert popular rationalist thinker].
Of course, if LW were truly meritocratic (which it should be), this shouldn’t matter — but in my experience, it descriptively does.
Without naming anyone (since that would be unproductive), I wanted to know if others notice this too? And aside from simply trying not to upvote something because it’s written by a popular author, anyone have good ideas for preventing this?
The effect seems natural and hard to prevent. Basically, certain authors get reputations for being high (quality * writing), and then it makes more sense for people to read their posts because both the floor and ceiling are higher in expectation. Then their worse posts get more readers (who vote) than posts of a similar quality by another author, who’s floor and ceiling is probably lower.
I’m not sure the magnitude of the cost, or that one can realistically expect to ever prevent this effect. For instance, ~all Scott Alexander blogposts get more readership than the best post by many other authors who haven’t built a reputation and readership, and this kind of just seems part of how the reading landscape works.
Of course, it can be frustrating as an author to sometimes see similar quality posts on LW get different karma. I think the answer here is to do more to celebrate the best posts by new authors. The main thing that comes to mind here is curation, where we celebrate and get more readership on the best posts. Perhaps I should also have a term here for “and this is a new author, so I want to bias toward curating them for the first time so that they’re more invested in writing more good content”.
Yes, but you’d naively hope this wouldn’t apply to shitty posts, just to mediocre posts. Like, maybe more people would read, but if the post is actually bad, people would downvote etc.
That’s right. One exception: sometimes I upvote posts/comments written to low standards in order to reward the discussion happening at all. As an example I initially upvoted Gary Marcus’s first LW post in order to be welcoming to him participating in the dialogue, even though I think the post is very low quality for LW.
(150+ karma is high enough and I’ve since removed the vote. Or some chance I am misremembering and I never upvoted because it was already doing well, in which case this serves as a hypothetical that I endorse.)
One thing you could do is give users relatively more voting power if they vote without seeing the author of the post. I.e., you can enable a mode which hides post authors until you give a vote on the anonymized content. After that, you can still vote like normal.
Obviously there are ways author identity can leak through this, but it seems better than nothing.
You should probably link some posts, it’s hard to discuss this so abstractly. And popular rationalist thinkers should be able to handle their posts being called mediocre (especially highly-upvoted ones).
This was also on my mind after seeing Jesse’s short form yesterday. Ryan’s “this is good” comment was above Louis’ thorough explanation of an alternative formal motivation for IFs. That would still be the case if I hadn’t heavy upvoted and weak downvoted.
I personally cast my comment up/downvotes as an expression of my preference ordering for visibility. I would encourage others to also do so. For instance, I suggest Ryan’s comment should’ve been agreement voted rather than upvoted by others. This stance has as a corollary to not vote if you haven’t read the other comments whose rankings you are affecting—or rather vote with any other marker of which LW has many.
This ‘upvotes as visibility preferences’ policy isn’t tractable for posts, so I suspect the solution there—if one is needed—would have to be done on the backend by normalization. Not sure whether this is worth attempting.
Link here since I don’t particularly want to call out Ryan, his comment was fine. https://www.lesswrong.com/posts/7X9BatdaevHEaHres/jesse-hoogland-s-shortform
A salient example to me: This post essentially consists of Paul briefly remarking on some mildly interesting distinctions about different kinds of x-risks, and listing his precise credences without any justification for them. It’s well-written for what it aims to be (a quick take on personal views), but I don’t understand why this post was so strongly celebrated.
idea: popular authors should occasionally write preregistered intentionally subtly bad posts as an epistemic health check
That’s not really true if people are popular rationalist thinkers because of skill at good rationalist writing. Meritocracy does not imply that people get judged based on individual pieces of their work, a meritocracy where people are primarily judged on their total output would still be a meritocracy.
I think the problem is more that post that make points that are popular and fit neatly into the world view of the reader are more likely to get upvotes than posts that challenge the world view of the reader and would require the reader to update their world view. Jimmy’s introductionary post to his sequence is currently as I’m writing it at 12 karma.
While writing quality might be improved, it’s a post that sets out to challenges the readers conception about how human reasoning works in practical contexts and that’s why it’s at low karma. I would love to see more posts like Jimmy that are about the core of how rationality works over posts that feel good to read and that get lot of upvotes without really changing minds much.
I think if someone is very well-known their making a particular statement can be informative in itself, which is probably part of the reason it is upvoted.
I had similar thoughts. And indeed I found funny that a forum of rationalists falls for this.
But this is human nature, and we have finite time and energy to focus on things.
You will find the same for the attention papers written by people at famous institutions get, even if low quality.
Making names, recently, I had a similar feeling when looking at “AI as Normal Technology”. I did not know the authors, and I was skimming it because someone talked about it.
It is also a reason for which sometimes celebrities are called to do propaganda.
A strategy would be to make make posts anonymous for a limited amount of time (perhaps with some text randomization that would slightly change the style of the author/s, even though this might affect quality).