Voting etiquette
Not all that surprisingly, there’s quite a lot of discussion on LW about questions like
just what should get voted up or down?
what conclusions can one reasonably draw from getting downvoted?
should downvotes (or even upvotes) be accompanied by explanations?
should the way karma and voting work be changed?
This generally happens in dribs and drabs, typically in response to more specific questions of the form
Waaaa, how come my supremely insightful comment above is currently sitting at −69?
and therefore tends to clutter up discussions that are meant to be about something else. So maybe it’s worth seeing if we can arrive at some sort of consensus about the general issues, at which point maybe we can write that up and refer newcomers to it.
(The outcome may be that we find that there’s no consensus to be had. That would be useful information too.)
I’ll kick things off with a few unfocused thoughts.
What voting is for: establishing the nearest thing we have to the consensus view of the LW community, so as to (1) help readers guess what might be most worth reading and (2) help writers adjust their writing (if they wish) to please the audience more. Note that these purposes are somewhat separate from …
What karma is for: motivating people to participate, motivating people to participate well, giving readers an indication of which writers are most worth reading.
It seems to me that voting is working reasonably well—I find a reasonable correlation between comment ratings and comment quality. I’m not convinced that karma is working so well; what’s rewarded by the system is prolific posting at least as much as high-quality posting. Doing away with the auto-self-upvote (and making it impossible to upvote one’s own comments) seems likely to be an improvement. Or maybe making each comment count for (say) 1⁄4 as much as an upvote.
Explanations for votes: Lots of comments get voted up; quite a lot get voted down. The practice of explaining votes (even just downvotes) would make for cluttered threads. Also: upvotes and downvotes are anonymous, which is largely a good thing. So, here’s one possibility. (It might just be unnecessary complication). When you vote something up or down, you get the chance (or the obligation?) to write a brief explanation of why; it doesn’t go into the thread as a comment, but gets associated with the comment you voted on (without your name attached). Then hovering over a comment’s score (or something) could pop up a list of votes each way and their explanations, if any. Still anonymous; out of the way when not specifically asked for; but gives some hope of finding why something was downvoted, and also a way of distinguishing between +1 −0 and +14 −13.
Positivist’s guide to voting
Q: What is karma for?
A: You need 20 karma to write articles. That’s it.
Q: What should I upvote?
A: That which you want to see at the top of the page.
Q: What should I downvote?
A: That which you don’t want to see at the top of the page.
Anything else is meaningless.
Well said.
EDIT: Q: What should I vote down below −4? A: That which you don’t want others to see without clicking on it.
Some consequencse of CannibalSmith’s sensible rules, together with Eliezer’s sensible rule (and a converse: “What should I vote up past −4? What you think doesn’t merit hiding from others.”):
Every comment gets voted up or down by every voter. In particular …
Every comment that, on balance, people don’t want to see at the top of the page (even if they don’t otherwise mind it) gets voted down to minus infinity. Then …
Everyone follows Eliezer’s advice and votes it up again to get it past −4.
This all seems rather suboptimal.
(Yeah, I know, tongue in cheek and all that.)
I don’t think these rules imply this. Most comments shouldn’t be considered ones you want to see at the top of the page. Depending on how you define ‘top’ and ‘bottom’, it might result in every voter only voting on two comments each.
Most users will be able to answer “yes” or “no” to “Do I want to see this at the top of the page?” about most comments, I think. In which case, if they follow CS’s two rules then they will vote those comments either up or down: so most comments end up getting voted on by most users.
Therefore, any comment for which a fraction of users substantially less than 50% wants to see it at the top of the page—and, as you rightly say, that’s likely to be most of them—will get downvoted heavily if everyone follows CS’s rules; almost certainly to lower than −4, unless voters notice soon enough that this is happening and switch to following Eliezer’s rule (which they cannot do at the same time as following CS’s rule).
(I’m sure CS didn’t intend his rules to be applied so mechanically, and actually I think both his and Eliezer’s are basically correct.)
Those silly positivists! ;-)
Being more interested in voting as community feedback and less interested in getting the karmic high score, I’ve taken to removing the auto-upvote on any of my own comments that I would not have upvoted had someone else posted them, then continued to monitor the vote scores for the next few days. I’ve come to a couple conclusions:
Most people seem to be voting based on what they think the final score should be, not their personal approval independent of existing score. Most posts ended up at 1 or 2, independent of the default self-upvote.
Given roughly similar information density, people are more likely to upvote short comments.
People are far too likely to upvote bad jokes. For a while I think my highest-scored comment was an “Eliezer Yudkowsky Fact”. Oy vey.
I believe the behavior is mostly as follows. If the comment is voted neutral (currently, 1 point), people only upvote or downvote if they feel strongly about whether it’s good or bad (according to whatever valuation). If the comment is already upvoted or downvoted, people are more likely to cast an opposite vote if they feel that the comment is rated incorrectly (in the wrong direction), to “fix” its rating. Thus, if someone upvotes a 0-point comment, it doesn’t mean that the comment is considered worthy of an upvote (to reach, say, 9 points), it only means that the person thought that it didn’t deserve a downvote from 1 point.
Thus, there seems to be 2 modes of voting: hard voting and soft voting.
Hard vote is supposed to push the comment all the way in the specified direction.
Soft vote is supposed to push the comment in the given direction, but only towards neutral rating, and not beyond that. It’s intended only to eliminate the opposing votes, not to set the vote.
Yes. The quandary seems, to me, that the voting system is designed for hard voting, but in practice more people are using soft voting.
So the solution is either to change the system’s design, or change the user’s behavior? The latter seems unlikely, so what would a system designed to utilize soft voting look like?
Hacker News solved the problem by not displaying comment karma, except to the owner of the comment.
My highest-scored comment was also my worst. (It wouldn’t have been awful in a different context; but I might not have gotten so many up-votes in an appropriate context.)
My comment just above, though, sets a new personal speed record. It was up for about 10 seconds before someone down-voted it.
I am curious as to why they down-voted it. It would be nice to require people to give a brief (possibly anonymous; possibly not displayed by default) explanation when they downvote something.
A ranking preference is expressed as a vote. An explanation is expressed as a reply. In the system as it stands, these are two very discrete actions. How often and in what circumstances do people use them in combination? What would be the effects of explicitly linking them?
Current proposal @ http://code.google.com/p/lesswrong/issues/detail?id=144 calls for switching to a zero-based system with no self-upvotes or downvotes, and showing ratios of upvotes to downvotes (so making comments in old threads, that won’t be much upvoted or downvoted, won’t count against your “average” quality).
I’m beginning to think we may also want to go ahead and switch to that brilliant suggestion for distinguishing the quality upvote/downvote, and the agreement upvote/downvote.
However, all this requires programming work and isn’t the top priority right now. An experienced Python programmer willing to volunteer a noticeable block of time would probably get around to it a lot sooner.
I don’t know about ratios, but I’d like to see the total upvotes and downvotes on a comment. It would be very useful to know if I got 5 upvotes and 5 downvotes or if the comment just sat there getting nothing.
I may not understand this proposal, but… I think the current system of comments starting at +1 is good because people react badly to negative numbers and zero. That self-vote counting for total karma is bad, but that is easily fixed by subtracting 1 before computing total karma.
I don’t see any reason why karma couldn’t be calculated based on votes, disallowing self votes, but the default comment score be something other than 0.
Also in re: karma— is anyone currently worried about sockpuppet upvotes? I suppose that only the 20-point threshold means enough for that to be worth it at the moment, and we haven’t yet had to deal with serious trolling. But someone should be on the watch for certain kinds of misbehavior.
Great idea! I’ll add that to my upcoming “how to troll on lw” article. :)
We could also run some simple statistics. For instance, what is the average score of one’s posts and comments overall, as a simple metric of one’s quality of posting, versus sheer quantity (or at least how well those posts are received on average).
But I’m really more interested in knowing why people up or downvote posts, I’m here to learn stuff too, and sometimes it may not be obvious why a post was up or downvoted, be it mine or someone’s else. Even though the voting score represents the community consensus, something is amiss. In some cases, one of us may simply lack a necessary insight as to one topic in particular, to understand what that consensus judges to be obvious and straightforward in that case. If our goal is to learn and teach, then it’s our duty to make it as easy and accessible for everyone to grasp those insights, and understand the reasoning behind the consensus.
The way slashdot used to handle this is, an upvote or downvote had a name attached to it—you’d select ‘insightful’ or ‘funny’ or ‘interesting’ or ‘troll’ (amongst others) so there would be at least some reason attached to each vote.
Right now we don’t have the programming resources to do this. Tricycle is currently working on other parts of the system. Python volunteer anyone?
I’ve got quite a lot of Python-fu and sufficient web-fu, but not very much time. If there’s anything like consensus that some change to the voting/karma system would be beneficial, then I’ll see whether I can easily get the LW codebase running on my system, and have a go at doing the necessary implementation. But (1) I’d rather wait until there’s some evidence that it’s actually (a) a good idea and (b) seen to be a good idea by the Masters of the Codebase, and (2) if there are people in the LW community who’ve worked with Reddit stuff before then they’re likely to be better placed to improve things than me.
I have some modest Python skills, a forked and working copy of the LW codebase on my local machine, and the potential to block out some time to implement features.
The main things stopping me are existing time commitments for the next 1-2 weeks and a lack of consensus (from community and the Powers That Be) about how it ought to work. For instance, I’m not even sure which part(s) of gjm’s post you’re referring to with “do this”. I’d love to help but I don’t want to spend time implementing things that noone wants.
Well, we’ve got a pretty wide consensus that upvoting your own comments should not increase your karma. At least it looks like a wide consensus to me, based on admittedly biased and small sampling data. With any luck this is simple—with any unluck it’s not.
One problem with fixing this is that a quick hack to remove the karma from a self-upvote might break if/when self-upvoting is removed entirely. Of course, this might be avoided merely by the relevant programmer bearing this possibility in mind.
I wonder—would it be useful for people to receive karma points for programming contributions to the LW community? It sounds reasonable to me.
An interesting question is, how do you determine the number of karma points the work deserves? One approach would be that one of the site admins could assign it a value. Another would be that it could be voted upon.
Essentially the description of the ‘feature’ to be added would be a post, and it’s score would be the number of karma points to be awarded for implementing it. Vote up if you think that score is too little, vote down if you think it is too much. This would also give you a way to rank the ‘feature requests’ - those with the highest scores are the ones the community cares about most (of course that may not matter much if there’s only the occasional bit of programming work to be done).
I realise that there’d be costs and effort required to get any system like this going. E.g. you probably want such feature request ‘posts’ on a different part of the site, and you’d have to explain the scheme to people, etc.
This idea of providing karma points like this wouldn’t have to apply to just programming tasks—it could be anything else that isn’t a post or a comment but which is nonetheless a contribution to the community.
upon reflection, a poll might be better. along the lines of:
how many points is the implementation of this feature worth?
10
20
50
100
150
A suggestion on voting:
Get rid of down votes. Only allow up-votes, and suggest the ground for upvoting would be that the voter believes the comment or post should be higher up the sort, even if the voter disagrees with it. This would be a simple rule, easy to apply, easy to understand. The fact that a comment had only one point would indicate that only its author thought it interesting or worthwhile.
Comments made late to older posts would have a lower value simply because fewer people had read them. This is a flaw in this proposal. However all sets of rules will have flaws, and the more complex the rules, or the voting system, the more likely it is that people will break the rules or fail to apply them correctly.
Note that my first five words indicate what the comment is about, so may attract attention when it is on the “recent comments” sidebar.
The moderators would need to alter the karma rule for making a post: perhaps karma of 100 or 200.
Edit: I notice for the first time from this comment that merely commenting does not give me a point, and that I may no longer up vote or down vote my own comment.
How did I miss this post entirely? Somehow I’ve managed to write a similar post after this one without referring to it, after scouring the site for any references to these sorts of issues.
I thought the karma system was inherited from Reddit. It might be a lot of work to change it.
How about starting by someone finding out how the current karma system works? Even Reddit has no documentation on it; they just say it’s constantly changing.
OTOH, we get some security through obscurity by not knowing how it works. Hopefully no one has analyzed their karmic rise and fall to reverse-engineer the karma rules in order to exploit them.
(I am aware that hope is not a secure safeguard.)
Considering that the source code can be browsed freely on the net, I wouldn’t count much on security by obscurity :)
Ditto on this. I also suggest that only post points should count for “Top Contributors”; time spent in the comments is certainly worthy, but it seems to be much cheaper to get points that way.
A very important factor in your post points is whether Eliezer decides to promote your post. If he doesn’t, it doesn’t show up on the home page.
For this reason, I would be more in favor of this idea if lesswrong.com pointed to ‘new’ instead of ‘promoted’.
I’m not saying that I want lesswrong.com to point to ‘new’. I’m aware that there’s a reason for pointing it to ‘promoted’. I’m explaining that my utility function incorporates a dependency between these two decisions.
Set your favorites to point to new, that’s what I do.
I’ve filed this request with Tricycle, but right now they’re working on other things. If you can program in Python, then this can get done faster.