(Note) This mostly has to do with karma with a minor rant/point at the end. If that doesn’t sound interesting, it probably won’t be.
Because I really do honestly think that if you want to downvote a comment that seems low-quality… and yet you hesitate, wondering if maybe you’re downvoting just because you disagree with the conclusion or dislike the author… feeling nervous that someone watching you might accuse you of groupthink or echo-chamber-ism or (gasp!) censorship… then nine times of ten, I bet, nine times out of ten at least, it is a comment that really is low-quality.
Some of the most interesting things I have registered about LessWrong thus far have to do with the karma game. I am convinced that there are huge swaths of information that can be learned if the karma data was opened for analysis.
If I had to guess at the weaknesses of the karma system I would peg two big problems. The first is that (some? most? many?) people are trying to assign an integer value to a post that is something outside of the range [-1,1] and then adjust their vote to affect a post’s score toward their chosen value. This seems to have the effect that everything is drawn toward 0 unless it is an absolutely stellar post. Then it just drifts up. I think the highest comment I have seen was in the high teens. I know there are more than twenty people visiting the site. Do they not read comments? Do they not vote on them?
The second problem spot is that I find it hard to actually use the feedback of karma. I have no way of knowing how well I am doing other a number. I have noticed that my karma has jumped lately and this leads me to believe I have made a change for the better. Unfortunately, I have no easy way of seeing which comments did well and which did poorly. Was it my tone? Did I get wiser? Are my comments more useful? Since I am new, my comment level is low and I can dig through what is there and learn, but this will simply get harder as time goes on. The karma system seems to work well on a comment by comment basis but not so much as a teaching tool. I see this as a problem because this is exactly what I need and I feel like I am squeezing a square peg into a round hole. It makes me think I am not using it correctly.
I find both of the above problems frustrating to me personally. I see a comment get voted down and think, “Okay, that was bad.” If I ask for clarification, it goes back up, which just makes it confusing. “Uh, so was it bad or not bad?” The difference between the highest rated comment of mine and the lowest is less than 10. I think the highest is 5 and the lowest was at −2 before I deleted it.
Now, don’t get me wrong, I am not complaining that my super-great-excellent posts are not voted to 20 karma in a single weekend. I am complaining that my crappy posts are all sitting at 0 and −1. I just started posting here and already have over 50 karma and the dark secret is that I am a complete poser. I barely even know the terms you guys use. I have not read much of Overcoming Bias and if you gave me a test on key points of rationality I would probably limp through the guessable stuff and start failing once the questions got hard. I can pick apart the logic within a given post, but the only real contributions I have made are exposing flaws in other comments. How in the world am I succeeding? I do not know.
To put this back into the original point, if people are shy about telling me my posts are low quality I can (a) never learn the difference between “mediocre” and “bad” and (b) any fool can limp by with comments that just repeat basic logic and use key terms in the right order. The chances of that being fun are low. One of my great paranoias is that I am the fool and no one pointed it out; I am the elephant in the room but have no mirror. I don’t want to trample on your garden and smush the roses. I want to partake in what seems to be a really awesome, fun community. If I don’t fit, kick me out.
(To be a little less harsh on myself, I do not consider myself a fool nor am I trying to play the role of a fool. If I am one, please let me know because I apparently have not figured it out yet.)
The karma system is a integral part of the Reddit base code that this site is built on top of. It’s designed to do one thing—increase the visibility of good content—and it does that one thing very well.
I agree, though, that there is untapped potential in the karma system. Personally I would love to see—if not by whom—at least when my comments are up/down voted.
Ah, that is good to remember. This seems to tilt my problem further toward fitting a square peg into the round hole. I guess that would be my own fault. :(
I have the same apprehension. I’m somewhere between “complete poser” and “well-established member of the community,” I just sort of found out about this movement about 50 days ago, started reading things and lurking, and then started posting. When I read the original post, I felt a little pang of guilt. Am I a fool running through your garden?
I’m doing pretty well for myself in the little Karma system, but I find that often I will post things that no one responds to, or that get up-voted or down-voted once and then left alone. I find that the only things that get down-voted more than once or twice are real attempts at trolling or otherwise hostile comments. Then again, many posts that I find insightful and beneficial to the discussion rarely rise about 2 or 3 karma points. So I’m left to wonder if my 1-point posts are controversial but good, above average but nothing special, or just mediocre and uninteresting.
Something that shows the volume of up- and down-votes as well as the net point score might provide more useful feedback.
I know there are more than twenty people visiting the site. Do they not read comments? Do they not vote on them?
I usually don’t vote because I don’t feel comfortable enough in my own understanding of these discussions to have an opinion about the relative value of a particular comment. Probably if I saw something that gave me an immediate and strong reaction, I’d be more likely to vote one way or another.
I know someone else who reads posts but seldom reads the comments.
I barely even know the terms you guys use. I have not read much of Overcoming Bias and if you gave me a test on key points of rationality I would probably limp through the guessable stuff and start failing once the questions got hard
We keep coming back to this: we very much need a “start here” document we can point people to, and say “please read this set of documents before posting”.
What I would like to see is a book that goes through all of the major biases and gives examples of each as well as heuristics for calibrating yourself better.
Our wiki article on Bias references the Wikipedia and Psychology Wiki lists of biases, and provides an outline of most of the specific biases discussed on OB.
Personally, I consider it my own responsibility to learn the terms. And I am learning them, I just have other stuff to do in the meantime. A “start here” would be useful and the place I started was the about page. Since then I think of a topic I think is relevant and then search OB and LW for topics already about that subject. More often than not, someone else has already said what I am thinking. That, mixed with reading comments, has gotten me as far as I am now.
Of course, a list would have made it a little easier. :)
I guess that CronoDAS had the people who have been on the site at least awhile in mind when he wrote “us.” If you see jargon being used that doesn’t already have an explanation at hand, you could always just reply to the comment that used the term and ask. The jargon page he alluded to is at http://wiki.lesswrong.com/wiki/Jargon
I have no easy way of seeing which comments did well and which did poorly
If you click on your username (or any other user’s), you get a history page with only your posts. That saves you the trouble of digging through all the stories you commented on, and lets you look at all your scores in one place.
The user info in the sidebar now has an envelope which is a link to a users inbox. The envelope is red if there are new messages, otherwise it is gray.
The inbox and sent pages are now styled similar to the rest of lesswrong. In addition they now also have the sidebar.
I have an enhancement suggestion: have two colors for the “Inbox” icon, one to indicate that there are only comment replies (green color?), and another one for private messages (orange). This way, I won’t need to check the inbox for the comments, if I know that I have read them anyway, but I won’t miss private messages as a result of not checking it when new comments arrive.
The inbox is a feature that came for free with the Reddit codebase but it was “lost” when the site was restyled. You will notice that the formatting of inbox page is totally messed up, this is also because it wasn’t included in redesign. Notification of replies is on the list of things to implement but there’s some higher priority work going on at the moment. Since it is a small change and many people seem to be requesting it I hope that we will get to it soon.
Thank you for the analysis. Would it help if you saw who, in particular, downvoted/upvoted each of your comments? There is this feature “make my votes public”, but it’s virtually unusable in its current implementation (as it’s scoped by voters, not by articles that are being voted for), and it doesn’t seem to apply to comments at all. If the list of people who publicly voted about your comments (not everyone else’s) is directly visible next to the comments, I expect that to be useful to the newcommers, and it won’t clutter the interface overly much.
I would find just knowing the total up and down to accomplish more. The only reason I would want to know who voted is to see if the immediate replies are voted up or down. I have noticed a few people who will reply in disagreement and vote down. (This is not a problem; it is just a pattern I noticed.)
The karma system isn’t enough for the purpose of learning; I fully agree to that. And to the point of this article, I usually don’t downvote people, rather I try to correct them if I see something wrong. That, if anything, seems more appropriate to me. If I see an issue somewhere, it isn’t enough to point it, I must be able to explain why it is an issue, and should propose a way to solve it.
But Eliezer has me swayed on that one. Now I’ll downvote, even though I am, indeed, very uncertain of my own ability to correctly judge whether a post deserves to be downvoted or not. For that matter, I am very uncertain about the quality of my own contributions as well, so there too I can relate to your experience. Sometimes, I feel like I’m just digging myself deeper and deeper, that I am not up to the necessary quality required to post in here.
Now, if I was told what, in my writings, correlates with high karma, and what does, with low karma, I think I might be tempted to optimize my posting to karma—gathering, rather than adapting them to the purpose of making high quality, useful contributions.
That’s a potential issue. Karma is correlated to quality and usefulness, but ultimately, other things than quality alone can come into play, and we don’t want to elicit people’s optimizing for those for their own sake alone (like, persuasiveness, rhetorics, seductive arguments, well written, soul sucking texts, etc.).
We really need to get beyond the karma system. But apparently none of the ways so far proposed would be workable, for lack of programming resources. We’ll need to be vigilant till then.
But Eliezer has me swayed on that one. Now I’ll downvote, even though I am, indeed, very uncertain of my own ability to correctly judge whether a post deserves to be downvoted or not.
I disagree, I don’t think you should downvote what you don’t understand. This will only pull the discussion to the level of the least competent people.
if people downvote what they don’t understand, and it’s a good comment, then it should have more upvotes than downvotes if most people understand it. If it has more downvotes than upvotes in this scenario, then it was not explained well enough for the majority of readers.
These are generalizations, of course, and depend largely on actual voting habits. But so was the note that it will pull the discussion to the level of the ‘least competent people’ - possibly the same observation could be stated as pulling the discussion to the level of the majority of the readership.
That was my first idea. But I am not the only player here. I know I overcompensate for my uncertainty, and so I tend to never downvote anything. Other people may not have the same attitude, for down, and upvoting. Who are they ? Is their opinion more educated than mine ? If we all are too scrupulous to vote when our opinion is in fact precious, then our occasional vote may end up drowned in a sea of poorly decided, hastily cast ones.
Besides, I am still only going to downvote if I can think of a good reason to do so. For sometimes, I have a good reason to downvote, but no still no good reasons, or even no time, to reply to all ideas I think need a fix, or those which are simply irrelevant to the current debate.
You are trying to fight fools with your intuition. How much confidence do you place in it? Is your intuition more informed than the decisions of average voters? Hard to say, I wouldn’t be so sure in this compound statement. It only becomes clear where you know yourself to be competent or ignorant, above or below the “average voter”. At least abstaining from voting has clear semantics, you don’t introduce your judgment at all. On the other hand, in many cases it should be easy to recognize poor quality.
I don’t place any confidence in my intuition as a general, indiscriminately good-for-everything case. I try to only have confidence on a case by case basis. I try to pay attention to all potential bias that could screw my opinion, like anchoring. And try to not pay attention to who wrote what I’m voting upon. Then I have to have a counterargument. Even if I don’t elaborate it, even if I don’t lay it down, I have to know that if I had the time or motivation, I could rather reply, and say what was wrong or right in that post.
My decisions and arguments, could, or could not be more informed than those of the average voter. But if I add my own in the pool of votes, then we have a new average. Which will only be slightly worse, or slightly better. Could we try to adapt something of decision markets there ? The way they’re supposed to self correct, under the right conditions, makes me wonder if we could dig a solution in them.
And maybe someone could create an article, collecting all the stuff that could help people make more informed votes on LW, that’d help too. Like the biases they’d have to take into account, stuff like the antikibitzer, or links to articles such as the one about aumann voting or this very one.
(Note) This mostly has to do with karma with a minor rant/point at the end. If that doesn’t sound interesting, it probably won’t be.
Some of the most interesting things I have registered about LessWrong thus far have to do with the karma game. I am convinced that there are huge swaths of information that can be learned if the karma data was opened for analysis.
If I had to guess at the weaknesses of the karma system I would peg two big problems. The first is that (some? most? many?) people are trying to assign an integer value to a post that is something outside of the range [-1,1] and then adjust their vote to affect a post’s score toward their chosen value. This seems to have the effect that everything is drawn toward 0 unless it is an absolutely stellar post. Then it just drifts up. I think the highest comment I have seen was in the high teens. I know there are more than twenty people visiting the site. Do they not read comments? Do they not vote on them?
The second problem spot is that I find it hard to actually use the feedback of karma. I have no way of knowing how well I am doing other a number. I have noticed that my karma has jumped lately and this leads me to believe I have made a change for the better. Unfortunately, I have no easy way of seeing which comments did well and which did poorly. Was it my tone? Did I get wiser? Are my comments more useful? Since I am new, my comment level is low and I can dig through what is there and learn, but this will simply get harder as time goes on. The karma system seems to work well on a comment by comment basis but not so much as a teaching tool. I see this as a problem because this is exactly what I need and I feel like I am squeezing a square peg into a round hole. It makes me think I am not using it correctly.
I find both of the above problems frustrating to me personally. I see a comment get voted down and think, “Okay, that was bad.” If I ask for clarification, it goes back up, which just makes it confusing. “Uh, so was it bad or not bad?” The difference between the highest rated comment of mine and the lowest is less than 10. I think the highest is 5 and the lowest was at −2 before I deleted it.
Now, don’t get me wrong, I am not complaining that my super-great-excellent posts are not voted to 20 karma in a single weekend. I am complaining that my crappy posts are all sitting at 0 and −1. I just started posting here and already have over 50 karma and the dark secret is that I am a complete poser. I barely even know the terms you guys use. I have not read much of Overcoming Bias and if you gave me a test on key points of rationality I would probably limp through the guessable stuff and start failing once the questions got hard. I can pick apart the logic within a given post, but the only real contributions I have made are exposing flaws in other comments. How in the world am I succeeding? I do not know.
To put this back into the original point, if people are shy about telling me my posts are low quality I can (a) never learn the difference between “mediocre” and “bad” and (b) any fool can limp by with comments that just repeat basic logic and use key terms in the right order. The chances of that being fun are low. One of my great paranoias is that I am the fool and no one pointed it out; I am the elephant in the room but have no mirror. I don’t want to trample on your garden and smush the roses. I want to partake in what seems to be a really awesome, fun community. If I don’t fit, kick me out.
(To be a little less harsh on myself, I do not consider myself a fool nor am I trying to play the role of a fool. If I am one, please let me know because I apparently have not figured it out yet.)
The karma system is a integral part of the Reddit base code that this site is built on top of. It’s designed to do one thing—increase the visibility of good content—and it does that one thing very well.
I agree, though, that there is untapped potential in the karma system. Personally I would love to see—if not by whom—at least when my comments are up/down voted.
Ah, that is good to remember. This seems to tilt my problem further toward fitting a square peg into the round hole. I guess that would be my own fault. :(
I have the same apprehension. I’m somewhere between “complete poser” and “well-established member of the community,” I just sort of found out about this movement about 50 days ago, started reading things and lurking, and then started posting. When I read the original post, I felt a little pang of guilt. Am I a fool running through your garden?
I’m doing pretty well for myself in the little Karma system, but I find that often I will post things that no one responds to, or that get up-voted or down-voted once and then left alone. I find that the only things that get down-voted more than once or twice are real attempts at trolling or otherwise hostile comments. Then again, many posts that I find insightful and beneficial to the discussion rarely rise about 2 or 3 karma points. So I’m left to wonder if my 1-point posts are controversial but good, above average but nothing special, or just mediocre and uninteresting.
Something that shows the volume of up- and down-votes as well as the net point score might provide more useful feedback.
I usually don’t vote because I don’t feel comfortable enough in my own understanding of these discussions to have an opinion about the relative value of a particular comment. Probably if I saw something that gave me an immediate and strong reaction, I’d be more likely to vote one way or another.
I know someone else who reads posts but seldom reads the comments.
We keep coming back to this: we very much need a “start here” document we can point people to, and say “please read this set of documents before posting”.
In the mean time, here is a list of Eliezer’s posts to Overcoming Bias.
What I would like to see is a book that goes through all of the major biases and gives examples of each as well as heuristics for calibrating yourself better.
Do we even have a ready at hand list of the major biases? That would be a good wiki article.
http://en.wikipedia.org/wiki/List_of_cognitive_biases
Our wiki article on Bias references the Wikipedia and Psychology Wiki lists of biases, and provides an outline of most of the specific biases discussed on OB.
Personally, I consider it my own responsibility to learn the terms. And I am learning them, I just have other stuff to do in the meantime. A “start here” would be useful and the place I started was the about page. Since then I think of a topic I think is relevant and then search OB and LW for topics already about that subject. More often than not, someone else has already said what I am thinking. That, mixed with reading comments, has gotten me as far as I am now.
Of course, a list would have made it a little easier. :)
When you see a term that you don’t immediately understand, let us know, so we can add it to the wiki.
Better still, ask for the page to be created by following the instructions under “Getting help” on the front page of the wiki.
Who is “us”? How should one let you know?
I guess that CronoDAS had the people who have been on the site at least awhile in mind when he wrote “us.” If you see jargon being used that doesn’t already have an explanation at hand, you could always just reply to the comment that used the term and ask. The jargon page he alluded to is at http://wiki.lesswrong.com/wiki/Jargon
Thank you.
If you click on your username (or any other user’s), you get a history page with only your posts. That saves you the trouble of digging through all the stories you commented on, and lets you look at all your scores in one place.
Thanks. Is there anyway to see which comments have replies from that page?
no, but you can see from your inbox, which, for some odd reason, is not linked to anywhere.
ETA: Well, not linked to anywhere is a stretch. You can navigate there as follows:
click some user’s name
click “send a message” (off to the right near your and their karma scores)
there’ll be a menu under the site logo with “compose, inbox, sent”
I find it’s easier to just bookmark the inbox page, or let your browser start autocompleting it for you
The user info in the sidebar now has an envelope which is a link to a users inbox. The envelope is red if there are new messages, otherwise it is gray.
The inbox and sent pages are now styled similar to the rest of lesswrong. In addition they now also have the sidebar.
Thanks!
I have an enhancement suggestion: have two colors for the “Inbox” icon, one to indicate that there are only comment replies (green color?), and another one for private messages (orange). This way, I won’t need to check the inbox for the comments, if I know that I have read them anyway, but I won’t miss private messages as a result of not checking it when new comments arrive.
Thank you!
The inbox is a feature that came for free with the Reddit codebase but it was “lost” when the site was restyled. You will notice that the formatting of inbox page is totally messed up, this is also because it wasn’t included in redesign. Notification of replies is on the list of things to implement but there’s some higher priority work going on at the moment. Since it is a small change and many people seem to be requesting it I hope that we will get to it soon.
Whoa, that is the most useful feature yet. Fantastic; thank you.
no problem =)
http://lesswrong.com/message/inbox
(Is there a navigation link to this? I only knew about it from the welcome post)
ETA: This was in response to a question about whether there’s any way to navigate to the inbox
sort of...
click some user’s name
click send a message
there’ll be a menu under the site logo with “compose, inbox, sent”
I find it’s easier to just bookmark the inbox page, or let your browser start autocompleting it for you
Thank you for the analysis. Would it help if you saw who, in particular, downvoted/upvoted each of your comments? There is this feature “make my votes public”, but it’s virtually unusable in its current implementation (as it’s scoped by voters, not by articles that are being voted for), and it doesn’t seem to apply to comments at all. If the list of people who publicly voted about your comments (not everyone else’s) is directly visible next to the comments, I expect that to be useful to the newcommers, and it won’t clutter the interface overly much.
I would find just knowing the total up and down to accomplish more. The only reason I would want to know who voted is to see if the immediate replies are voted up or down. I have noticed a few people who will reply in disagreement and vote down. (This is not a problem; it is just a pattern I noticed.)
The karma system isn’t enough for the purpose of learning; I fully agree to that. And to the point of this article, I usually don’t downvote people, rather I try to correct them if I see something wrong. That, if anything, seems more appropriate to me. If I see an issue somewhere, it isn’t enough to point it, I must be able to explain why it is an issue, and should propose a way to solve it.
But Eliezer has me swayed on that one. Now I’ll downvote, even though I am, indeed, very uncertain of my own ability to correctly judge whether a post deserves to be downvoted or not. For that matter, I am very uncertain about the quality of my own contributions as well, so there too I can relate to your experience. Sometimes, I feel like I’m just digging myself deeper and deeper, that I am not up to the necessary quality required to post in here.
Now, if I was told what, in my writings, correlates with high karma, and what does, with low karma, I think I might be tempted to optimize my posting to karma—gathering, rather than adapting them to the purpose of making high quality, useful contributions.
That’s a potential issue. Karma is correlated to quality and usefulness, but ultimately, other things than quality alone can come into play, and we don’t want to elicit people’s optimizing for those for their own sake alone (like, persuasiveness, rhetorics, seductive arguments, well written, soul sucking texts, etc.).
We really need to get beyond the karma system. But apparently none of the ways so far proposed would be workable, for lack of programming resources. We’ll need to be vigilant till then.
I disagree, I don’t think you should downvote what you don’t understand. This will only pull the discussion to the level of the least competent people.
if people downvote what they don’t understand, and it’s a good comment, then it should have more upvotes than downvotes if most people understand it. If it has more downvotes than upvotes in this scenario, then it was not explained well enough for the majority of readers.
These are generalizations, of course, and depend largely on actual voting habits. But so was the note that it will pull the discussion to the level of the ‘least competent people’ - possibly the same observation could be stated as pulling the discussion to the level of the majority of the readership.
That was my first idea. But I am not the only player here. I know I overcompensate for my uncertainty, and so I tend to never downvote anything. Other people may not have the same attitude, for down, and upvoting. Who are they ? Is their opinion more educated than mine ? If we all are too scrupulous to vote when our opinion is in fact precious, then our occasional vote may end up drowned in a sea of poorly decided, hastily cast ones.
Besides, I am still only going to downvote if I can think of a good reason to do so. For sometimes, I have a good reason to downvote, but no still no good reasons, or even no time, to reply to all ideas I think need a fix, or those which are simply irrelevant to the current debate.
You are trying to fight fools with your intuition. How much confidence do you place in it? Is your intuition more informed than the decisions of average voters? Hard to say, I wouldn’t be so sure in this compound statement. It only becomes clear where you know yourself to be competent or ignorant, above or below the “average voter”. At least abstaining from voting has clear semantics, you don’t introduce your judgment at all. On the other hand, in many cases it should be easy to recognize poor quality.
I don’t place any confidence in my intuition as a general, indiscriminately good-for-everything case. I try to only have confidence on a case by case basis. I try to pay attention to all potential bias that could screw my opinion, like anchoring. And try to not pay attention to who wrote what I’m voting upon. Then I have to have a counterargument. Even if I don’t elaborate it, even if I don’t lay it down, I have to know that if I had the time or motivation, I could rather reply, and say what was wrong or right in that post.
My decisions and arguments, could, or could not be more informed than those of the average voter. But if I add my own in the pool of votes, then we have a new average. Which will only be slightly worse, or slightly better. Could we try to adapt something of decision markets there ? The way they’re supposed to self correct, under the right conditions, makes me wonder if we could dig a solution in them.
And maybe someone could create an article, collecting all the stuff that could help people make more informed votes on LW, that’d help too. Like the biases they’d have to take into account, stuff like the antikibitzer, or links to articles such as the one about aumann voting or this very one.