The LW team has spent the last few weeks developing alternative voting systems. We’ve enabled two-axis voting on this post. The two dimensions are:
Overall: what is your overall feeling about the comment? Does it contribute positively to the conversation? Do you want to see more comments like this?
Agreement: do you agree with the position of this comment?
Separating these out allows for you to express more nuanced reactions to comments such as “I still disagree with what you’re arguing for, but you’ve raised some interesting and helpful points” and “although I agree with what you’re saying, I think this is a low-quality comment”.
Edited to Add: I checked with Jessica first whether she was happy for us to try this experiment with her post.
This will be one experiment among several. This is an experiment so bugs are possible. We’re interested in what effect this has on the quality of conversation, what the experience of voting in this system is like, and what the experience of skimming a thread and seeing these scores is like.
Agreement-votes and related code will not necessarily be kept forever, if we don’t go with that as the overall voting system for LW. Agreement-votes and agreement-scores will be kept at least for as long as any thread using that voting system is active.
GreaterWrong, and some areas of the site which aren’t especially integrated with the two-axis voting, will show only the overall score, not the agreement scores. Sorting is by overall score and only the overall score affects user karma.
Feedback: I intuitively expected the first/left vote to be “agree/disagree” and the second/right vote to be “compliance with good standards.” In reality, it’s closer to the reverse of that. Not sure how typical my experience will be.
(In my imagination, a user goes “I like this” followed by ”...but it was sketchy from an epistemic standpoint” or similar.)
I think if you swapped them, at least at this stage, you would have a bunch of people who accidentally indicated agreement because they thought they were normal voting
Yeah. Normal voting could have been left as is, with two buttons that indicate those two things. If something had an extreme score via voting, but didn’t score strongly (or in the same direction) via the other two, then voting would be capturing something else.
One of the issues with these things (like ‘agree’) - whatever that refers to (i.e. agree with what?), is that the longer, and more parts, a comment has, the less a single score captures that well, for any dimension.
One of the issues with these things (like ‘agree’) - whatever that refers to (i.e. agree with what?), is that the longer, and more parts, a comment has, the less a single score captures that well, for any dimension.
This thread The comments on this post in particular are a great example of this. Lots of people taking pieces of the post and going ‘I disagree with this’ or ‘this is not true’.
I guess the first vote means “whatever my vote would be under the old system”.
The second vote… I am not sure how to apply it e.g. to your comment. If I click “agree”, what does it mean?
I agree that Duncan intuitively expected the first/left vote to be agree/disagree, and the second/right vote to be good standards? [yes]
It also seems to me that the first/left vote is agree/disagree, and the second/right vote is good standards? [no]
I guess I am just going to use the first vote as usual (now if you vote “agree” on this comment, does it mean “yes, I believe this is exactly what Viliam will do” or “yes, I will do the same thing”?), and the second one only in situations that seem unambiguous.
I’ve been doing a lot of ‘overall’ voting based on “all things considered, am I happy this comment exists?” or “would I like to see more comments like this in the future?” and ‘agreement’ voting on specifically “do I endorse its contents?”
For a non-charged example, I upvoted Duncan’s comment suggesting that the buttons be swapped, because I think that a good kind of feedback to give on an experiment, and voted disagree on it because I don’t think swapping would have the effect he thinks it would have.
Another example: if two people are having a back and forth where they seem to remember different things, I’ll normal vote for both of them because I’m glad they’re hashing it out, but I won’t agree/disagree with any of them because I don’t have any inside information on what happened.
Cool experiment. A note: I just clicked “agree” to a comment and noticed that it gave two points, and was somewhat surprised (with a bad valence). Maybe it makes sense, but somehow I expected the agree thing to mean literally “this many people clicked agree”. (Haven’t thought about it, just a reaction.)
Just some immediate feedback on this—There is a big noticeable phenomenon, which is that I agree and disagree with many comments, even though I frequently think the comment is just “OK”, so I am making many more “agreement” votes than “overall” votes.
Really! I just encountered this feature, and have been more reluctant to agree than to upvote. Admittedly, the topic has mostly concerned conversations which I didn’t hear.
This post discusses suicides and psychosis of named people. I think it’s an inappropriate place to experiment with a new voting system. I think you could choose a less weighty post for initial experiments.
Also, I don’t get the impression that this experiment was done with jessicata’s explicit agreement, and I’m worried that this post is being singled out for special treatment because of its content.
We did ask Jessica first whether she would want to participate in the experiment. Also, the reason why we want to experiment with additional voting systems is because specifically threads like this seem like they kind of tend to go badly using our current voting system, and switching it out is an attempt at making them go better, in a way that I think Jessica also wants.
Recording my reaction to the new system: I looked back at the comment and saw it got a downvote on agreement and got that small twinge of negative affect that I sometimes get from seeing a downvote on my own comment, then realized it probably just means that person doesn’t have the same bug, and it passed. It would be interesting to see if this instinct changes after some time getting used to this system.
I also see the CSS as a bit wonky, at least on mobile, though not as wonky as that. I see the agreement box as about one pixel higher than the overall box.
I was noticing different users having different patterns for upvotes versus agreement (partly because mine seemed to be skewed toward agreement…) and I wanted to play with it a little more. Here’s a script that extracts the votes from this page. Expand all comments (⌘F) before running.
function author(meta) {
return meta.children[1].innerText;
}
function votes(meta) {
return meta.children[3].innerText.split("\n");
}
const metas = document.getElementsByClassName("CommentsItem-meta");
const output = {};
for (let i = 0; i < metas.length; i++) {
const meta = metas[i];
output[author(meta)] = output[author(meta)] || {
comments: 0,
upvotes: 0,
agreement: 0,
};
output[author(meta)].comments++;
output[author(meta)].upvotes += Number(votes(meta)[0]);
output[author(meta)].agreement += Number(votes(meta)[1]);
}
Here’s the total agreement:upvotes ratio for this thread, by user:
This is a good example of a comment that is not a contribution to the discussion, but which lots of people here agree with, illustrating the power and flexibility of the two-axis voting system. Except, if you meant it as such, it’s a valuable contribution to the discussion, and shouldn’t be downvoted. But if it weren’t downvoted it wouldn’t properly show off the voting system, and would no longer be valuable. You’ve tied us in knots!
Yep, I wanted to experiment with a central example of a comment that should be in the “downvote/agree” quadrant, since that seemed like the least likely to occur naturally. It’s nice to see the voting system is working as intended.
Experimental Two-Axis Voting: “Overall” & “Agreement”
The LW team has spent the last few weeks developing alternative voting systems. We’ve enabled two-axis voting on this post. The two dimensions are:
Overall: what is your overall feeling about the comment? Does it contribute positively to the conversation? Do you want to see more comments like this?
Agreement: do you agree with the position of this comment?
Separating these out allows for you to express more nuanced reactions to comments such as “I still disagree with what you’re arguing for, but you’ve raised some interesting and helpful points” and “although I agree with what you’re saying, I think this is a low-quality comment”.
Edited to Add: I checked with Jessica first whether she was happy for us to try this experiment with her post.
A few notes:
This will be one experiment among several. This is an experiment so bugs are possible. We’re interested in what effect this has on the quality of conversation, what the experience of voting in this system is like, and what the experience of skimming a thread and seeing these scores is like.
Agreement-votes and related code will not necessarily be kept forever, if we don’t go with that as the overall voting system for LW. Agreement-votes and agreement-scores will be kept at least for as long as any thread using that voting system is active.
GreaterWrong, and some areas of the site which aren’t especially integrated with the two-axis voting, will show only the overall score, not the agreement scores. Sorting is by overall score and only the overall score affects user karma.
Feedback: I intuitively expected the first/left vote to be “agree/disagree” and the second/right vote to be “compliance with good standards.” In reality, it’s closer to the reverse of that. Not sure how typical my experience will be.
(In my imagination, a user goes “I like this” followed by ”...but it was sketchy from an epistemic standpoint” or similar.)
I think if you swapped them, at least at this stage, you would have a bunch of people who accidentally indicated agreement because they thought they were normal voting
Yeah. Normal voting could have been left as is, with two buttons that indicate those two things. If something had an extreme score via voting, but didn’t score strongly (or in the same direction) via the other two, then voting would be capturing something else.
One of the issues with these things (like ‘agree’) - whatever that refers to (i.e. agree with what?), is that the longer, and more parts, a comment has, the less a single score captures that well, for any dimension.
This threadThe comments on this post in particular are a great example of this. Lots of people taking pieces of the post and going ‘I disagree with this’ or ‘this is not true’.I am confused.
I guess the first vote means “whatever my vote would be under the old system”.
The second vote… I am not sure how to apply it e.g. to your comment. If I click “agree”, what does it mean?
I agree that Duncan intuitively expected the first/left vote to be agree/disagree, and the second/right vote to be good standards? [yes]
It also seems to me that the first/left vote is agree/disagree, and the second/right vote is good standards? [no]
I guess I am just going to use the first vote as usual (now if you vote “agree” on this comment, does it mean “yes, I believe this is exactly what Viliam will do” or “yes, I will do the same thing”?), and the second one only in situations that seem unambiguous.
I’ve been doing a lot of ‘overall’ voting based on “all things considered, am I happy this comment exists?” or “would I like to see more comments like this in the future?” and ‘agreement’ voting on specifically “do I endorse its contents?”
For a non-charged example, I upvoted Duncan’s comment suggesting that the buttons be swapped, because I think that a good kind of feedback to give on an experiment, and voted disagree on it because I don’t think swapping would have the effect he thinks it would have.
Another example: if two people are having a back and forth where they seem to remember different things, I’ll normal vote for both of them because I’m glad they’re hashing it out, but I won’t agree/disagree with any of them because I don’t have any inside information on what happened.
Cool experiment. A note: I just clicked “agree” to a comment and noticed that it gave two points, and was somewhat surprised (with a bad valence). Maybe it makes sense, but somehow I expected the agree thing to mean literally “this many people clicked agree”. (Haven’t thought about it, just a reaction.)
I think that makes sense but on the other hand both vote counts being directly comparable seems good
Just some immediate feedback on this—There is a big noticeable phenomenon, which is that I agree and disagree with many comments, even though I frequently think the comment is just “OK”, so I am making many more “agreement” votes than “overall” votes.
Really! I just encountered this feature, and have been more reluctant to agree than to upvote. Admittedly, the topic has mostly concerned conversations which I didn’t hear.
This post discusses suicides and psychosis of named people. I think it’s an inappropriate place to experiment with a new voting system. I think you could choose a less weighty post for initial experiments.
Also, I don’t get the impression that this experiment was done with jessicata’s explicit agreement, and I’m worried that this post is being singled out for special treatment because of its content.We did ask Jessica first whether she would want to participate in the experiment. Also, the reason why we want to experiment with additional voting systems is because specifically threads like this seem like they kind of tend to go badly using our current voting system, and switching it out is an attempt at making them go better, in a way that I think Jessica also wants.
The agreement box got a bit excited :)
Recording my reaction to the new system: I looked back at the comment and saw it got a downvote on agreement and got that small twinge of negative affect that I sometimes get from seeing a downvote on my own comment, then realized it probably just means that person doesn’t have the same bug, and it passed. It would be interesting to see if this instinct changes after some time getting used to this system.
I try to imagine the two numbers as parts of a complex number, like “37+8i”.
From that perspective, both “37+8i” and “37-8i” feel positive.
I also see the CSS as a bit wonky, at least on mobile, though not as wonky as that. I see the agreement box as about one pixel higher than the overall box.
Yay for experimenting!
I was noticing different users having different patterns for upvotes versus agreement (partly because mine seemed to be skewed toward agreement…) and I wanted to play with it a little more. Here’s a script that extracts the votes from this page. Expand all comments (⌘F) before running.
Here’s the total agreement:upvotes ratio for this thread, by user:
Yudkowsky is so awesome!!
This is a good example of a comment that is not a contribution to the discussion, but which lots of people here agree with, illustrating the power and flexibility of the two-axis voting system. Except, if you meant it as such, it’s a valuable contribution to the discussion, and shouldn’t be downvoted. But if it weren’t downvoted it wouldn’t properly show off the voting system, and would no longer be valuable. You’ve tied us in knots!
Yep, I wanted to experiment with a central example of a comment that should be in the “downvote/agree” quadrant, since that seemed like the least likely to occur naturally. It’s nice to see the voting system is working as intended.