On Comments, Voting, and Karma—Part I

There has been a great deal of dis­cus­sion here about the proper meth­ods of vot­ing on com­ments and on how karma should be as­signed. I be­lieve it’s fi­nally reached the point where a post is war­ranted that cov­ers some of the is­sues in­volved. (This may be just be­cause I find my­self fre­quently in dis­agree­ment with oth­ers about it.)

The Au­to­matic Upvote

First, there is the ques­tion of whether one should be able to up­vote one’s own com­ment. This ac­tu­ally breaks apart into two re­lated con­cerns:

(1) One is able to up­vote one’s own com­ments, and

(2) One gains a point of karma just for post­ing a com­ment.

Th­ese need not be tied. We could have (2) with­out (1) by award­ing a point of karma for com­ment­ing, with­out chang­ing the com­ment’s score. We could also have (1) with­out (2) by sim­ply not count­ing self-up­votes for karma.

I am in fa­vor of (2). The main ar­gu­ment against (2) is that it re­wards quan­tity over qual­ity. The main ar­gu­ment for (2) is that it offers an au­to­matic in­cen­tive to post com­ments; that is, it re­wards com­ment­ing over silence. As we’re com­mu­nity-build­ing, I think the lat­ter in­cen­tive is more im­por­tant than the former. But I’m not sure this is worth ar­gu­ing fur­ther—it serves as a dis­trac­tion from the benefits of (1).

I am also in fa­vor of (1). As a de­fault, all com­ments have a base rat­ing of 0. Since one is al­lowed to vote on one’s own com­ments, and up­vot­ing is the de­fault for one’s own com­ments, this makes com­ments effec­tively start at a rat­ing of 1. The ar­gu­ment against this is that it makes more sense for com­ments to start with a rat­ing of 0, so that some­one else lik­ing a com­ment gives it a pos­i­tive rat­ing, while some­one dis­lik­ing it gives it a nega­tive rat­ing. I dis­agree with this as­sess­ment.

If I post a com­ment, it’s be­cause it’s the best com­ment I could think of to add to the dis­cus­sion. I will usu­ally not bother say­ing some­thing if I don’t think it’s the sort of thing that I would up­vote. When I see some­one else’s com­ment that I don’t think is very good, I down­vote it. Since they already up­voted it, I’m in effect dis­agree­ing that this was some­thing worth say­ing. The score now re­flects this—a score of 0 shows that one per­son thought it was a worth­while com­ment, and one per­son did not.

Fur­ther­more, if I was not able to vote on my own com­ments, I would be much more re­luc­tant to up­vote. Since I would not be able to up­vote my com­ment, up­vot­ing some­one else’s com­ment would sug­gest that I think their com­ment is bet­ter than my own. But by hy­poth­e­sis, I thought my com­ment was nearly the best thing that could be said on the sub­ject; thus, up­votes will be rare.

And so I say that we im­ple­ment a com­pro­mise - (1) and not (2).

What should up­vote/​down­vote mean?

I think it is es­tab­lished pretty well that up­vote means “High qual­ity com­ment” or “I would like to see more com­ments like this one”, while down­vote means “Low qual­ity com­ment” or “I would like to see fewer com­ments like this one”. How­ever, this defi­ni­tion still re­tains a good bit of am­bi­guity.

It is too easy to think of up­vote and down­vote as ‘agree’ and ‘dis­agree’. Even guard­ing my­self against this be­hav­ior I find the cur­sor drift­ing to down­vote as soon as I think, “Well that’s ob­vi­ously wrong”. But that’s clearly not what the con­cept is there for. Com­ments voted up ap­pear higher on the page (on cer­tain views), which al­lows ca­sual read­ers to see the best com­ments and dis­cus­sions on any par­tic­u­lar post. If we use up­vote and down­vote to mean ‘agree’ and ‘dis­agree’, then this is effec­tively an echo cham­ber, where the only com­ments to float to the top are the ones that jive with the group­think.

In­stead, up­vote and down­vote should re­flect over­all qual­ity of a com­ment. There are sev­eral crite­ria I tend to use to judge a good com­ment (this list is not all-in­clu­sive):

  1. Did the com­ment add any in­for­ma­tion, or did it just add to the noise? (+)

  2. Does the com­ment in­clude refer­ences or links to rele­vant in­for­ma­tion? (+)

  3. Does the com­ment re­flect a gen­uinely held point-of-view that adds to the dis­cus­sion? (+)

  4. Is the com­ment part of a dis­cus­sion that might lead some­where in­ter­est­ing? (+)

  5. Is the com­ment ob­vi­ous spam /​ trol­ling? (-)

  6. Is the com­ment off-topic? (-)

Since we feel the need to voice whether we agree or dis­agree with com­ments, but ‘I agree’ and ‘I dis­agree’ com­ments are noisy, it’s been sug­gested that there should be sep­a­rate but­tons to in­di­cate agree­ment and dis­agree­ment. Thus, some­one post­ing a well-ar­gued on-topic defense of the­ism can get the up­vote and ‘dis­agree’, while some­one post­ing an off-topic ‘phys­i­cal­ism is true’ can get the down­vote and ‘agree’. Pre­sum­ably, we’d only count up­votes and down­votes for karma, but we could use ‘agree’ and ‘dis­agree’ for “most con­tro­ver­sial” or other views/​met­rics.

Whether votes should re­quire an explanation

It has been sug­gested that votes, or down­votes speci­fi­cally, should re­quire an ex­pla­na­tion. I dis­agree with both sen­ti­ments. First, re­quiring ex­pla­na­tions for down­votes but not up­votes would bias the vot­ing pos­i­tively, which would have the effect of re­ward­ing quan­tity over qual­ity and de­crease the im­pact of down­votes.

But re­quiring ex­pla­na­tions for votes is in gen­eral a bad idea. This site is already a bur­den to keep up with; for those of us that do a lot of vot­ing, writ­ing an ex­pla­na­tion for each one would be too much time and effort. Re­quiring an ex­pla­na­tion for ev­ery vote would doubtless re­sult in a lot less vot­ing. Also, ex­plain­ing votes is al­most always off-topic, so adds to the noise here with­out re­ally con­tribut­ing to the dis­cus­sion.

Note Yvain’s more per­sonal ra­tio­nale:

I’m not pre­pared to write an es­say ex­plain­ing ex­actly was wrong with each of them, es­pe­cially if the origi­nal com­menter wasn’t pre­pared to take three sec­onds to write a halfway de­cent re­sponse.

Ad­ding to the bur­den of those already perform­ing the ser­vice of vot­ing un­duly pe­nal­izes those who are do­ing good, to the end of ap­peas­ing those who are con­tribut­ing to the noise here.

Rele­vant Comments

For refer­ence, some links to rele­vant posts and sec­tions of com­ments. I tried to be in­clu­sive, since there have been a lot of dis­cus­sions about these is­sues—more rele­vant ones hope­fully near the top. (Please com­ment if you know of any other rele­vant dis­cus­sions)

1 2 3 what up­vote and down­vote should mean and whether there should be agree/​dis­agree buttons

4 whether karma should be the sum of in­di­vi­d­ual post scores, or (per­haps) an average

5 su­per-votes

6 The util­ity of com­ment karma

7 whether one should un­s­e­lect the self-upvote

8 9 whether down­votes should re­quire explanation

10 whether Eliezer Yud­kowsky gets fewer up­votes than others

11 whether karma can be used to gauge rationality

12 whether peo­ple down­vote for dis­agree­ing with groupthink

13 whether karma pro­motes a closed-gar­den effect

14 whether ad­minis­tra­tors should delete com­ments entirely

15 Less­wrong An­tik­ib­itzer: tool for hid­ing com­ment au­thors and vote counts

ETA: I might con­cede that this post is pos­si­bly off-topic for Less Wrong—but the blog/​com­mu­nity site about “Less Wrong” does not ex­ist yet, so this seems like the best place to post it.

ETA2: Public records of up­votes/​down­votes might solve some of these prob­lems; dis­cuss.