I liked this, but I observe that I don’t think it would have gotten nearly as much karma if it had been written by someone without a reputation already.
...which suggests that we’re maybe doing something wrong?
Or is this in fact more informative, because we know that it came from Eliezer?
What the system is missing is the “ratio between reads to upvotes”. When you just look at upvotes, that’s a retrospective quality signal conflated with the prospective calculation people do about whether to try reading. The latter and earlier decision is obviously going to be username-dependent.
Yeah, I’ve experimented with showing that piece of information, but sadly it made people feel super self-conscious about even clicking on any post, because it would kind of be an implicit downvote. It’s also kind of hard to determine what counts as a view (does any click count? Or any time someone reads to the end? Or sometime someone spends more than 10 seconds on the page?).
Plausible we should experiment more with it. Or maybe do kind of a thing where you can get that piece of information but it’s slightly inconvenient (maybe putting it on-hover is good enough).
for unrelated reasons, I would also like a feature where I can mark a post as read (although that’s probably not common enough to be used as direct signal, but could still be a proxy maybe)
It feels like an important metric is “karma per view” or “karma per read” or “karma per user-minute-looking-at-text” something similar. Currently, we can’t gauge that, and so when someone who gives a strong prior that their post will be worth reading posts, that post will get more views which means more upvotes even if they have a similar “karma per read”.
EDIT: EY’s post loaded the second I posted this, but I promise it was an independent invention
Karma also involves strong upvotes (and that’s before getting into ’people with more karma have their votes counted more). I’m reading this right now at
EY: 37 karma, 19 votes (I don’t know what the base line is for his default upvote of himself*.)
Alex Hollow: 18 karma 11 votes.
If we assume both consist of upvotes, then it seems pretty clear this a result of a) lots of strong upvotes (those votes aren’t counting once). b) people whose ‘upvote’ increases karma by two, with a few strong upvotes thrown in, or c) a mixture of a and b.
I note that when I ran the Conor Moreton experiment, it seemed to me to confirm that karma and attention does accrue to good writing without prior reputation. Perhaps it takes a couple of rounds, to get past people’s ordinary why-should-this-be-promoted-to-my-attention-above-everything-else-competing-for-that-attention heuristics, but.
Nobody had a clue who Conor Moreton was, and after a week, he was being listened to.
It definitely suggests karma is an imperfect measure of quality/usefulness but I don’t think that automatically means we’re making the wrong trade-offs.
(these thoughts are more general than this particular post, which I think is a little different just for being fiction).
Some posts require more energy than others to get the value out of them. When you’re first looking at a post, it can be hard to estimate what the total return on energy is. In particular “deeply challenging and thought provoking” can look a lot “total bullshit”, especially if the writer hasn’t put a ton of skill and effort into making their challenging post easier to access. When faced with a new post that is definitely going to take a lot of energy and has an uncertain payoff, considering the reputation of the writer can be a legit useful heuristic.
This has a bunch of negative consequences, but it’s really unclear to me if those outweigh the benefits, or if something could be done to improve the pareto frontier.
I’m delighted I read the post in the curated newsletter without noticing who was the author and only then decided to head here to upvote it. I wonder if moving the authorship information to the end of the newsletter influences the reader’s willingness to upvote the post – perhaps an opportunity for an A/B test?
Yeah, there are a few posters who’ll get lots of upvotes regardless of the quality or utility of the post. And more comments and discussion as well, which is a kind of self-fulfilling utility of the post.
I don’t think it indicates we’re doing something wrong, just the way cults like LW work. Ok, that’s way too harsh, but if you think it’s important, that probably means you’re taking karma too seriously. Meaningless internet points are meaningless.
I liked this, but I observe that I don’t think it would have gotten nearly as much karma if it had been written by someone without a reputation already.
...which suggests that we’re maybe doing something wrong?
Or is this in fact more informative, because we know that it came from Eliezer?
What the system is missing is the “ratio between reads to upvotes”. When you just look at upvotes, that’s a retrospective quality signal conflated with the prospective calculation people do about whether to try reading. The latter and earlier decision is obviously going to be username-dependent.
Yeah, I’ve experimented with showing that piece of information, but sadly it made people feel super self-conscious about even clicking on any post, because it would kind of be an implicit downvote. It’s also kind of hard to determine what counts as a view (does any click count? Or any time someone reads to the end? Or sometime someone spends more than 10 seconds on the page?).
Plausible we should experiment more with it. Or maybe do kind of a thing where you can get that piece of information but it’s slightly inconvenient (maybe putting it on-hover is good enough).
for unrelated reasons, I would also like a feature where I can mark a post as read (although that’s probably not common enough to be used as direct signal, but could still be a proxy maybe)
Maybe if they spent the estimated amount of time on the page to read the article (or more), that counts as a view.
There’s also something to be said about the impact of strong upvotes.
Yeah, maybe we could show ratio of strong upvotes to upvotes
It feels like an important metric is “karma per view” or “karma per read” or “karma per user-minute-looking-at-text” something similar. Currently, we can’t gauge that, and so when someone who gives a strong prior that their post will be worth reading posts, that post will get more views which means more upvotes even if they have a similar “karma per read”.
EDIT: EY’s post loaded the second I posted this, but I promise it was an independent invention
And ironically, your post has substantially less karma than Eliezer’s does, despite saying basically the same thing!
Karma also involves strong upvotes (and that’s before getting into ’people with more karma have their votes counted more). I’m reading this right now at
EY: 37 karma, 19 votes (I don’t know what the base line is for his default upvote of himself*.)
Alex Hollow: 18 karma 11 votes.
If we assume both consist of upvotes, then it seems pretty clear this a result of a) lots of strong upvotes (those votes aren’t counting once). b) people whose ‘upvote’ increases karma by two, with a few strong upvotes thrown in, or c) a mixture of a and b.
*His karma is 116,019 though.
I note that when I ran the Conor Moreton experiment, it seemed to me to confirm that karma and attention does accrue to good writing without prior reputation. Perhaps it takes a couple of rounds, to get past people’s ordinary why-should-this-be-promoted-to-my-attention-above-everything-else-competing-for-that-attention heuristics, but.
Nobody had a clue who Conor Moreton was, and after a week, he was being listened to.
It definitely suggests karma is an imperfect measure of quality/usefulness but I don’t think that automatically means we’re making the wrong trade-offs.
(these thoughts are more general than this particular post, which I think is a little different just for being fiction).
Some posts require more energy than others to get the value out of them. When you’re first looking at a post, it can be hard to estimate what the total return on energy is. In particular “deeply challenging and thought provoking” can look a lot “total bullshit”, especially if the writer hasn’t put a ton of skill and effort into making their challenging post easier to access. When faced with a new post that is definitely going to take a lot of energy and has an uncertain payoff, considering the reputation of the writer can be a legit useful heuristic.
This has a bunch of negative consequences, but it’s really unclear to me if those outweigh the benefits, or if something could be done to improve the pareto frontier.
I do find more value in reading it given that it came from Eliezer. I do care about what Eliezer writes.
I’m delighted I read the post in the curated newsletter without noticing who was the author and only then decided to head here to upvote it. I wonder if moving the authorship information to the end of the newsletter influences the reader’s willingness to upvote the post – perhaps an opportunity for an A/B test?
Yeah, there are a few posters who’ll get lots of upvotes regardless of the quality or utility of the post. And more comments and discussion as well, which is a kind of self-fulfilling utility of the post.
I don’t think it indicates we’re doing something wrong, just the way cults like LW work. Ok, that’s way too harsh, but if you think it’s important, that probably means you’re taking karma too seriously. Meaningless internet points are meaningless.
Good point. Strong-downvoted the post.