Instead of trying to stop noise, you can filter it. Instead of designing to prevent errors, you can design to be robust to them.
I’ll repeat something I said in the other thread:
To the extent that all the griping over signal to noise is about a desire to control what you see, and not control what others see or say, there are decades old solutions to discussion filtering. The fancy shmancy Web has been a marked deevolution of capabilities in this regard. It’s pitiful. No web discussion forum I know of has filtering capabilities even in the ball park of Usenet, which was available in the 80s. Pitiful.
I also suggest that any solution which is not fundamentally about user customization is a failure under my assumption above, because one man’s noise is another man’s signal.
You’ve made me understand the root of one of my own dissatisfactions with the current system. If I look through my post history and roughly group my posts into bins based on how I would summarize them, this is what I see:
Silly posts in the HP:MOR threads: ~ +20 karma
Posts of mine having little content except to express agreement with other high-karma posts: ~ +10 karma
Important information or technical corrections in serious discussions: ~ +1 karma
Posts which I try to say something technical which I retrospectively realize were poorly worded but could have been clarified if someone pointed out an issue instead of just downvoting: ~ −5 karma
Perhaps I exaggerate slightly but my point is that if I were to formulate a posting strategy aimed at obtaining karma, then I would avoid saying anything technical or discussing anything serious and stick to applause lights and fluff.
On top of this, I tend to watch how the karma of my most recent comments behaves, and so I notice that, for example, a comment might have +5 upvotes and −3 downvotes, with no replies. This is just baffling to me. Was there something wrong with the post that three people noticed? Were the three separate things wrong with it? Was it just a response to the tone? What about the upvotes, is it being upvoted because of the witticism at the end, or because of the technical content in the middle? My point is something like Slashdot has a system where things are voted “funny” or “insightful” would be infinitely more useful.
Perhaps I exaggerate slightly but my point is that if I were to formulate a posting strategy aimed at obtaining karma, then I would avoid saying anything technical or discussing anything serious and stick to applause lights and fluff.
That’s about right. Also, stick to high traffic threads. Hit the HPMOR threads hard!
As I pointed out that people want different things out of the list, you finish by pointing out that the karma votes themselves are clearly used differently by different people. They’re also used to a different extent by different people.
One nice thing that Slashdot does is limit your karma votes. That keeps individual Karma Kops from have a disproportionate effect on total score. But I don’t think the Slashdot system of multiple scores is that helpful.
From my experience in the grand old days of Usenet, the most useful filters were on people, and the important ease of use features were a single screen view of all threads, expand and contract, sort by date or thread, and sort by date for a subset of threads.
I think you might be falling prey to a sort of fundamental attribution error for comments… thinking of all votes on a comment as being about the internal traits of the comment itself.
I generally vote to enact a total ordering on all current content, aiming to raise valuable/unique/informative/pro-social content to reader-serving prominence. This involves determining an ideal arrangement of content and voting everything down that is too high, and voting up everything that is too low… except, I try to keep the floor at zero total except where content is (sadly) above the sanity waterline of LW, as with some discussions of gender relations and politics.
About the only “pure knee jerk voting” I engage in is upvoting of content that isn’t mind-killing in itself but that has a negative total. Sometimes I upvote a comment simply because someone said something really awesome as a rebuttal to it, and that comment/rebuttal pair is worth reading, and the way to give the conversation the order-of-reading-attention it deserves is to upvote the parent.
(+1) I rarely downvote, but from now on, I will accompany any downvote with a reply stating “-1: reason for downvote.”
I will downvote every such comment because I oppose this being used as a general policy. I don’t want to see the spam and sometimes it just isn’t useful to criticize explicitly. Even when downvoting is accompanied by such an criticism it is sometimes better to just speak directly rather than dragging in talk about your downvotes as part of the conversation.
This seems like a reasonable approach. The reason for the downvote could force a defamatory statement, which I prefer to avoid. Otherwise, you are right that dragging in a downvote mention doesn’t add anything to just saying what you want to say. Thanks for the comment, by the way.
I was thinking that upon downvoting, maybe an option (not a requirement) should be given to state a reason why. Then I realized that there is no need to program such a thing; this option exists already.
Too much information can be ignored, too little information is sometimes annoying. I’d always welcome your reason for explaining your downvote, especially if it seems legitimate to me.
If we were going to get highly technical, a somewhat interesting thing to do would be to allow a double click to differentiate your downvote, and divide it into several “slider bars.” People who didn’t differentiate their downvotges would be listed as “general downvote” Those who did differentiate would be listed as a “specific reason downvote.” A small number of “common reasons for downvoting that don’t merit an individualized comment” on LessWrong would be present, plus an “other” box. If you clicked on the light gray “other”, it would be replaced with a dropdown selection box, one whose default position you could type into, limited to 140 characters. Other comments could be “poorly worded, but likely to be correct” “Poorly constructed argument,” “well-worded but likely incorrect” “ad hominem attack” “contains logical fallacies” “bad grammar” “bad formatting” “ignores existing body of thought, seems unaware of existing work on the subject” “anti-consensus, likely wrong” “anti-consensus, grains of truth.”
There could also be a “reason for upranking,” including polar opposite options that were the opposites of the prior options, so one need only adjust one slider bar for “positive and negative” common reasons. This would allow a + and—value to be associated with comments, to obtain a truer picture of the comment more quickly. “Detailed rankings” (listed next to the general ranking) could give commentators a positive and a negative for various reasons, dividing up two possible points, and adjusting remaining percentages for remaining portions of a point as the slider bar was raised. “General argument is true” could be the positive “up” value, “general argument is false” could be its polar opposite.
It also might be interesting to indicate how long people took to write their comments, if they were written in the edit window, and not copied and pasted. A hastily written comment could be downranked as “sloppily written” unless it was an overall optimal comment.
Then, when people click on the comment ranking numbers, they could see a popup window with all the general up and downvotes, and with many of them providing specific reasoning behind them. clicking on a big “X” would close the window.
I also like letting unregistered users voting in a separate “unregistered users” ranking. Additionally, it would be interesting to create a digital currency for the site that can be traded or purchased, in order to create market karma. Anyone who produces original work for LW could be paid corresponding to the importance of the work, according to their per hour payscale and the number of hours (corresponding to “real world pay” from the CFAR, or other cooperating organizations).
A friend of mine made $2M off of an initial small investment in bitcoin, and never fails to rub that in when I talk to him. I’d like it if a bunch of LW people made similar profits off of ideas they almost inherently understand. Additionally, it would be cool to get paid for “intellectual activity” or “actual useful intellectual work” (depending on one’s relationship with the site) :)
No web discussion forum I know of has filtering capabilities even in the ball park of Usenet, which was available in the 80s. Pitiful.
I strongly share your opinion on this. LW is actually one of the better fora I’ve come across in terms of filtering, and it still is fairly primitive. (Due to the steady improvement of this forum based on some of the suggestions that I’ve seen here, I don’t want to be too harsh.)
It might be a good idea to increase comment-ranking values for people who turn on anti-kibbitzing. (I’m sure other people have suggested this, so I claim no points for originality.) …What a great feature!
(Of course, then that option of “stronger karma for enabled anti-kibbitzers” would give an advantage the malevolent people who want to “game the system” who could turn it on and off, or turn it on on another device, see the information necessary to “send out their political soldiers” and use that to win arguments at a higher-ranking karma. Of course, one might want to reward malevolent players, because they are frequent users of the site, who thus increase the overall activity level, even if they do so dishonestly. They then become “invested players,” for when the site is optimized further. Also, robust sites should be able to filter even malevolent players, emphasizing constructive information flow. So, even though I’m a “classical liberal” or “small-L libertarian,” this site could theoretically be made stronger if there were a lot of paid government goons on it, purposefully trying to prevent benevolent or “friendly” AGI that might interfere with their plans for continuing domination.)
A good way to defeat this would be to “mine” for “anti-kibbitzing” karma. Another good idea would be to allow users to “turn off karma.” Another option would be to allow those with lots of karma to turn off their own karma, and show a ratio of “possible karma” next to “visible karma,” as an ongoing vote for what system makes the most sense, from those in a position of power to benefit from the system. This still wouldn’t tell you if it was a good system, but everyone exercizing the option would indicate that the karma-based system was was a bad one.
Also, I think that in a perfect world, Karma in its entirety should be eliminated here. “One man’s signal is another man’s noise,” indeed! If a genius level basement innovator shows up tomorrow and begins commenting here, I’d like him to stick around. (Possibly because I might be one myself, and have noticed that some of the people who most closely agree with certain arguments of mine are here briefly as “very low karma” partipants, agree with one or two points I make, and then leave. Also, if I try to encourage them but others vote them down, I’m encouraged to eliminate dissent, in the interest of eliminating “noise.” Why not just allow users to automatically minimize anyone who comments on a heavily-downranked already minimized comment? Problem solved.)
LessWrong is at risk of becoming another “unlikeliest cult,” to the same extent that Ayn Rand Institute became an “unlikely cult.” (ARI did, to some extent, become a cult, and that made it less successful at its intended goal, which was similar to the stated goal of LessWrong. It became too important what Ayn Rand personally thought about an idea, and too unimportant what hierarchical importance there inherently was to the individual ideas themselves. Issues became “settled” once she had an opinion on them. Much the way that “mind-killing” is now used to “shut down” political debate, or debate over the importance of political engagement, and thus cybernetics, itself.)
There are certain subjects that “most humans in general” have an incredibly difficult time discussing, and unthinking agreement with respected members of the community is precisely what makes it “safe” to disregard novel “true” or “valuable” solutions or problem-solving ideas, …rare as they may admittedly be.
Worse still, any human network is more likely to benefit from solutions outside of its own area of expertise. After all, the experts congregate in the same place, and familiarize themselves with the same incremental pathways toward the solution of their problems. In any complex modern discipline this requires immense knowledge and discipline. But what if there is a more direct but unanticipated solution that can arise from outside of that community? This is frequently the case, as indicated in Kurzweil’s quote of Weiner’s “Cybernetics” in “How to Create a Mind.”
It may be that the rise of a simple algorithm designed by a nanotech pioneer rapidly builds a better brain than AGI innovators can build, and that this brain “slashes the gordian knot,” by out-thinking humans and building better and better brains that ultimately are highly-logical, highly-rational, and highly-benevolent AGI. This constitutes some of the failure of biologists and computer scientists to understand the depth of each others’ points in a recent Singularity Summit meeting.
http://www.youtube.com/watch?v=kQ2snfsnroM -Dennis Bray on the Complexity of Biological Systems (Author of “Wetware” describing computational processes within cells).
Also, if someone can be a “troll” and bother other people with his comments, he’s doing you a small favor, because he’s showing that there are weaknesses in your commenting system that actually rise to the level of interfering with your mission. If we were all being paid minimum wage to be here, that might represent significant losses. (And shouldn’t we put a price on how valuable this time is to us?) The provision of garbled blather as a steady background of “chatter” can be analyzed by itself, and I believe it exists on a fluid scale from “totally useless” to “possibly useful” to “interesting.” Also, it indicates a partial value: the willingness to engage. Why would someone be willing to engage a website about learning an interesting subject, but not actually learn it? They might be unintelligent, which then gives you useful information about what people are willing to learn, and what kinds of minds are drawn to the page without the intelligence necessary to comprehend it, but with the willingness to try to interact with it to gain some sort of value. (Often these are annoying religious types who wish to convert you to their religion, who are unfamiliar with the reasons for unbelief. However, occasionally there’s someone who has logic and reason on their side, even though they are “unschooled.” I’m with Dawkins on this one: A good or bad meme can ride an unrelated “carrier meme” or “vehicle.”)
Site “chatter” might normally not be too interesting, and I admit it’s sub-optimal next to people who take the site seriously, but it’s also a little bit useful, and a little bit interesting, if you’re trying to build a network that applies rationality.
For example, there are, no doubt, people who have visited this website who are marketing majors, or who were simply curious about the current state of AGI due to a question about when will a “Terminator” or “skynet”-like scenario be possible, (if not likely). Some of them might have been willing participants in the more mindless busywork of the site, if there had been an avenue for them to pursue in that direction. There are very few such avenues on this “no nonsense” (but also no benevolent mutations) version of the site.
There also doesn’t appear to be much of an avenue for people who hold significant differences of opinion that contradict or question the consensus. Such ideas will be downvoted, and likely out of destructive conformity. As such, I agree that it’s best to allow users to eliminate or “minimize” their own conceptions of “what constitutes noise” and “what constitutes bias.”
Instead of trying to stop noise, you can filter it. Instead of designing to prevent errors, you can design to be robust to them.
I’ll repeat something I said in the other thread:
I also suggest that any solution which is not fundamentally about user customization is a failure under my assumption above, because one man’s noise is another man’s signal.
You’ve made me understand the root of one of my own dissatisfactions with the current system. If I look through my post history and roughly group my posts into bins based on how I would summarize them, this is what I see:
Silly posts in the HP:MOR threads: ~ +20 karma
Posts of mine having little content except to express agreement with other high-karma posts: ~ +10 karma
Important information or technical corrections in serious discussions: ~ +1 karma
Posts which I try to say something technical which I retrospectively realize were poorly worded but could have been clarified if someone pointed out an issue instead of just downvoting: ~ −5 karma
Perhaps I exaggerate slightly but my point is that if I were to formulate a posting strategy aimed at obtaining karma, then I would avoid saying anything technical or discussing anything serious and stick to applause lights and fluff.
On top of this, I tend to watch how the karma of my most recent comments behaves, and so I notice that, for example, a comment might have +5 upvotes and −3 downvotes, with no replies. This is just baffling to me. Was there something wrong with the post that three people noticed? Were the three separate things wrong with it? Was it just a response to the tone? What about the upvotes, is it being upvoted because of the witticism at the end, or because of the technical content in the middle? My point is something like Slashdot has a system where things are voted “funny” or “insightful” would be infinitely more useful.
That’s about right. Also, stick to high traffic threads. Hit the HPMOR threads hard!
As I pointed out that people want different things out of the list, you finish by pointing out that the karma votes themselves are clearly used differently by different people. They’re also used to a different extent by different people.
One nice thing that Slashdot does is limit your karma votes. That keeps individual Karma Kops from have a disproportionate effect on total score. But I don’t think the Slashdot system of multiple scores is that helpful.
From my experience in the grand old days of Usenet, the most useful filters were on people, and the important ease of use features were a single screen view of all threads, expand and contract, sort by date or thread, and sort by date for a subset of threads.
I think you might be falling prey to a sort of fundamental attribution error for comments… thinking of all votes on a comment as being about the internal traits of the comment itself.
I generally vote to enact a total ordering on all current content, aiming to raise valuable/unique/informative/pro-social content to reader-serving prominence. This involves determining an ideal arrangement of content and voting everything down that is too high, and voting up everything that is too low… except, I try to keep the floor at zero total except where content is (sadly) above the sanity waterline of LW, as with some discussions of gender relations and politics.
About the only “pure knee jerk voting” I engage in is upvoting of content that isn’t mind-killing in itself but that has a negative total. Sometimes I upvote a comment simply because someone said something really awesome as a rebuttal to it, and that comment/rebuttal pair is worth reading, and the way to give the conversation the order-of-reading-attention it deserves is to upvote the parent.
(+1) I rarely downvote, but from now on, I will accompany any downvote with a reply stating “-1: reason for downvote.”
I will downvote every such comment because I oppose this being used as a general policy. I don’t want to see the spam and sometimes it just isn’t useful to criticize explicitly. Even when downvoting is accompanied by such an criticism it is sometimes better to just speak directly rather than dragging in talk about your downvotes as part of the conversation.
This seems like a reasonable approach. The reason for the downvote could force a defamatory statement, which I prefer to avoid. Otherwise, you are right that dragging in a downvote mention doesn’t add anything to just saying what you want to say. Thanks for the comment, by the way.
I was thinking that upon downvoting, maybe an option (not a requirement) should be given to state a reason why. Then I realized that there is no need to program such a thing; this option exists already.
Too much information can be ignored, too little information is sometimes annoying. I’d always welcome your reason for explaining your downvote, especially if it seems legitimate to me.
If we were going to get highly technical, a somewhat interesting thing to do would be to allow a double click to differentiate your downvote, and divide it into several “slider bars.” People who didn’t differentiate their downvotges would be listed as “general downvote” Those who did differentiate would be listed as a “specific reason downvote.” A small number of “common reasons for downvoting that don’t merit an individualized comment” on LessWrong would be present, plus an “other” box. If you clicked on the light gray “other”, it would be replaced with a dropdown selection box, one whose default position you could type into, limited to 140 characters. Other comments could be “poorly worded, but likely to be correct” “Poorly constructed argument,” “well-worded but likely incorrect” “ad hominem attack” “contains logical fallacies” “bad grammar” “bad formatting” “ignores existing body of thought, seems unaware of existing work on the subject” “anti-consensus, likely wrong” “anti-consensus, grains of truth.”
There could also be a “reason for upranking,” including polar opposite options that were the opposites of the prior options, so one need only adjust one slider bar for “positive and negative” common reasons. This would allow a + and—value to be associated with comments, to obtain a truer picture of the comment more quickly. “Detailed rankings” (listed next to the general ranking) could give commentators a positive and a negative for various reasons, dividing up two possible points, and adjusting remaining percentages for remaining portions of a point as the slider bar was raised. “General argument is true” could be the positive “up” value, “general argument is false” could be its polar opposite.
It also might be interesting to indicate how long people took to write their comments, if they were written in the edit window, and not copied and pasted. A hastily written comment could be downranked as “sloppily written” unless it was an overall optimal comment.
Then, when people click on the comment ranking numbers, they could see a popup window with all the general up and downvotes, and with many of them providing specific reasoning behind them. clicking on a big “X” would close the window.
I also like letting unregistered users voting in a separate “unregistered users” ranking. Additionally, it would be interesting to create a digital currency for the site that can be traded or purchased, in order to create market karma. Anyone who produces original work for LW could be paid corresponding to the importance of the work, according to their per hour payscale and the number of hours (corresponding to “real world pay” from the CFAR, or other cooperating organizations).
A friend of mine made $2M off of an initial small investment in bitcoin, and never fails to rub that in when I talk to him. I’d like it if a bunch of LW people made similar profits off of ideas they almost inherently understand. Additionally, it would be cool to get paid for “intellectual activity” or “actual useful intellectual work” (depending on one’s relationship with the site) :)
I strongly share your opinion on this. LW is actually one of the better fora I’ve come across in terms of filtering, and it still is fairly primitive. (Due to the steady improvement of this forum based on some of the suggestions that I’ve seen here, I don’t want to be too harsh.)
It might be a good idea to increase comment-ranking values for people who turn on anti-kibbitzing. (I’m sure other people have suggested this, so I claim no points for originality.) …What a great feature!
(Of course, then that option of “stronger karma for enabled anti-kibbitzers” would give an advantage the malevolent people who want to “game the system” who could turn it on and off, or turn it on on another device, see the information necessary to “send out their political soldiers” and use that to win arguments at a higher-ranking karma. Of course, one might want to reward malevolent players, because they are frequent users of the site, who thus increase the overall activity level, even if they do so dishonestly. They then become “invested players,” for when the site is optimized further. Also, robust sites should be able to filter even malevolent players, emphasizing constructive information flow. So, even though I’m a “classical liberal” or “small-L libertarian,” this site could theoretically be made stronger if there were a lot of paid government goons on it, purposefully trying to prevent benevolent or “friendly” AGI that might interfere with their plans for continuing domination.)
A good way to defeat this would be to “mine” for “anti-kibbitzing” karma. Another good idea would be to allow users to “turn off karma.” Another option would be to allow those with lots of karma to turn off their own karma, and show a ratio of “possible karma” next to “visible karma,” as an ongoing vote for what system makes the most sense, from those in a position of power to benefit from the system. This still wouldn’t tell you if it was a good system, but everyone exercizing the option would indicate that the karma-based system was was a bad one.
Also, I think that in a perfect world, Karma in its entirety should be eliminated here. “One man’s signal is another man’s noise,” indeed! If a genius level basement innovator shows up tomorrow and begins commenting here, I’d like him to stick around. (Possibly because I might be one myself, and have noticed that some of the people who most closely agree with certain arguments of mine are here briefly as “very low karma” partipants, agree with one or two points I make, and then leave. Also, if I try to encourage them but others vote them down, I’m encouraged to eliminate dissent, in the interest of eliminating “noise.” Why not just allow users to automatically minimize anyone who comments on a heavily-downranked already minimized comment? Problem solved.)
LessWrong is at risk of becoming another “unlikeliest cult,” to the same extent that Ayn Rand Institute became an “unlikely cult.” (ARI did, to some extent, become a cult, and that made it less successful at its intended goal, which was similar to the stated goal of LessWrong. It became too important what Ayn Rand personally thought about an idea, and too unimportant what hierarchical importance there inherently was to the individual ideas themselves. Issues became “settled” once she had an opinion on them. Much the way that “mind-killing” is now used to “shut down” political debate, or debate over the importance of political engagement, and thus cybernetics, itself.)
There are certain subjects that “most humans in general” have an incredibly difficult time discussing, and unthinking agreement with respected members of the community is precisely what makes it “safe” to disregard novel “true” or “valuable” solutions or problem-solving ideas, …rare as they may admittedly be.
Worse still, any human network is more likely to benefit from solutions outside of its own area of expertise. After all, the experts congregate in the same place, and familiarize themselves with the same incremental pathways toward the solution of their problems. In any complex modern discipline this requires immense knowledge and discipline. But what if there is a more direct but unanticipated solution that can arise from outside of that community? This is frequently the case, as indicated in Kurzweil’s quote of Weiner’s “Cybernetics” in “How to Create a Mind.”
It may be that the rise of a simple algorithm designed by a nanotech pioneer rapidly builds a better brain than AGI innovators can build, and that this brain “slashes the gordian knot,” by out-thinking humans and building better and better brains that ultimately are highly-logical, highly-rational, and highly-benevolent AGI. This constitutes some of the failure of biologists and computer scientists to understand the depth of each others’ points in a recent Singularity Summit meeting. http://www.youtube.com/watch?v=kQ2snfsnroM -Dennis Bray on the Complexity of Biological Systems (Author of “Wetware” describing computational processes within cells).
Also, if someone can be a “troll” and bother other people with his comments, he’s doing you a small favor, because he’s showing that there are weaknesses in your commenting system that actually rise to the level of interfering with your mission. If we were all being paid minimum wage to be here, that might represent significant losses. (And shouldn’t we put a price on how valuable this time is to us?) The provision of garbled blather as a steady background of “chatter” can be analyzed by itself, and I believe it exists on a fluid scale from “totally useless” to “possibly useful” to “interesting.” Also, it indicates a partial value: the willingness to engage. Why would someone be willing to engage a website about learning an interesting subject, but not actually learn it? They might be unintelligent, which then gives you useful information about what people are willing to learn, and what kinds of minds are drawn to the page without the intelligence necessary to comprehend it, but with the willingness to try to interact with it to gain some sort of value. (Often these are annoying religious types who wish to convert you to their religion, who are unfamiliar with the reasons for unbelief. However, occasionally there’s someone who has logic and reason on their side, even though they are “unschooled.” I’m with Dawkins on this one: A good or bad meme can ride an unrelated “carrier meme” or “vehicle.”)
Site “chatter” might normally not be too interesting, and I admit it’s sub-optimal next to people who take the site seriously, but it’s also a little bit useful, and a little bit interesting, if you’re trying to build a network that applies rationality.
For example, there are, no doubt, people who have visited this website who are marketing majors, or who were simply curious about the current state of AGI due to a question about when will a “Terminator” or “skynet”-like scenario be possible, (if not likely). Some of them might have been willing participants in the more mindless busywork of the site, if there had been an avenue for them to pursue in that direction. There are very few such avenues on this “no nonsense” (but also no benevolent mutations) version of the site.
There also doesn’t appear to be much of an avenue for people who hold significant differences of opinion that contradict or question the consensus. Such ideas will be downvoted, and likely out of destructive conformity. As such, I agree that it’s best to allow users to eliminate or “minimize” their own conceptions of “what constitutes noise” and “what constitutes bias.”