Still somewhat sad about this, as it feels to me like a half-solution that’s pumping against human nature.
My claims in previous discussions about alternate voting systems were that:
It was going to be really important to have a single click, not multiple clicks (i.e. any two-separate-votes system was going to have people overwhelmingly just using one of the votes and largely ignoring the second one)
It was going to be really important to use visual cues and directionality and not just have two things side by side
I wanted something like the following:
… where users would single-click one of the four buttons and could click-hold to strong vote, but would with a single click show:
Upvote this and also it’s true (the dark blue one)
This seems true but slight downvote/it’s not helping/it’s making things worse (the light blue one)
This seems false or sketchy but slight upvote/it’s helping/I’m glad it’s here (the light orange one)
Downvote this and also it’s false (the dark orange one).
True-false is on the forward/backward axis, in other words, and good/bad is on the vertical axis, as usual.
The display for aggregation could look a lot of different ways; please don’t hate the below for its ugly first-draft nature:
For example, maybe the numbers are invisible unless you hover, or something. In this example, the blue shows which one I clicked.
This would also allow you (if you wanted) to display an aggregated “what’s this user’s rep” function—you could make some hover-over thing or put a place on people’s profile pages where you could see whether a given user was generally given dark blue, light blue, light orange, or dark orange feedback.
The reason I’m sad about the thing that’s happening here is that:
I suspect it’s a half-solution that will decay back to mostly-people-just-use-the-first-vote
I don’t think it lets me grok the quality of the reaction to a comment at a glance; I keep having to effortfully process “okay, what does—okay, this means that people like it but think it’s slightly false, unless they—hmm, a lot more people voted up-down than true-false, unless they all strong voted up-down but weak-voted tru—you know what, I can’t get any meaningful info out of this.”
I suspect it will sap whatever momentum there was for a truly better voting system, such that if it turns out I’m right and a year from now this didn’t really help, it’ll be even less likely that we can muster for another attempt.
I agree with the sentiment here, but I think you have too little faith in some people’s willingness to be disagreeable… especially on LessWrong! Personally I’d feel fine/great about having a high karma and a low net-agreement score, because it means I’m adding a unique perspective to the community that people value.
… and, I’d go so far as to bet that the large number of agreement with your comment here is representative of a bunch of users that would feel similarly, but I’m putting this in a separate comment so accrues a separate agree/disagree score. If lots of people disagree, I’ll update :)
It was going to be really important to have a single click, not multiple clicks (i.e. any two-separate-votes system was going to have people overwhelmingly just using one of the votes and largely ignoring the second one)
I feel like it’s slightly less work for me to consider one axis and up/downvote it, and then consider the second axis and up/downvote it, than it’d be to vote on two axes with a single click. The former lets me consider the two separately, “make one decision and then forget about it”, whereas the latter requires me to think about both at the same time. That means that I’m (slightly) more likely to cast two votes on a multiple-click system than on a single-click system.
Though I do also consider it a feature if the system allows me to only cast one vote rather than forcing me to do both. E.g. in situations where I want to upvote a domain expert’s comment giving an explanation about a domain that I’m not familiar with, so don’t feel qualified to cast a vote on its truth even though I want to indicate that I appreciate having the explanation.
I suspect it’s a half-solution that will decay back to mostly-people-just-use-the-first-vote
Regardless of whether it’s a bad solution in other respects, I predict that people will use the agree/disagree vote a ton, reliably, forever.
I don’t think it lets me grok the quality of the reaction to a comment at a glance; I keep having to effortfully process “okay, what does—okay, this means that people like it but think it’s slightly false, unless they—hmm, a lot more people voted up-down than true-false, unless they all strong voted up-down but weak-voted tru—you know what, I can’t get any meaningful info out of this.”
I mostly care about agree/disagree votes (especially when it comes to specifics). From my perspective, the upvotes/downvotes are less important info; they’re mostly there to reward good behavior and make it easier to find the best content fast.
In that respect, the thing that annoys me about agree/disagree votes isn’t any particular relationship to the upvotes/downvotes; it’s that there isn’t a consistent way to distinguish ‘a few people agreeing strongly’ from ‘a larger number of people agreeing weakly’, ‘everyone agrees with this but weakly’ from ‘some agree strongly but they’re being partly offset by others who disagree’, or ‘this is the author agreeing with their own comment’ from ‘this is a peer independently vouching for the comment’s accuracy’.
I think all of those things would ideally be distinguishable, at least on hover. (Or the ambiguity would be eliminated by changing how the feature works—e.g., get rid of the strong/weak distinction for agreevotes, get rid of the thing where users can agreevote their own comments, etc.)
The specific thing I’d suggest is to get rid of ‘authors can agree/disagree vote on their own comments’ (LW already has a ‘disendorse’ feature), and to replace the current UI with a tiny bar graph showing the rough relative number of strong agree, weak agree, strong disagree, and weak disagree votes (at least on hover).
I predict that people will use the agree/disagree vote a ton, reliably, forever.
I feel zero motivation to use it. I feel zero value gained from it, in its current form. I actually find it a deterrent, e.g. looking at the information coming in on my comment above gave me a noticeable “ok just never comment on LW again” feeling.
(I now fear social punishment for admitting this fact, like people will decide that me having detected such an impulse means I’m some kind of petty or lame or bad or whatever, but eh, it’s true and relevant. I don’t find downvotes motivationally deterring in the same fashion, at all.)
EDIT: this has been true in other instances of looking at these numbers on my other comments in the past; not an isolated incident.
“Okay, so it’s … it’s plus eight, on some karma meaning … something, but negative nine on agreement? What the heck does this even mean, do people think it’s good but wrong, are some people upvoting but others downvoting in a different place—I hate this. I hate everything about this. Just give up and go somewhere where the information is clear and parse-able.”
Like, maybe it would feel better if I could see something that at least confirmed to me how many people voted in both places? So I’m not left with absolutely no idea how to compare the +8 to the −9?
But overall it just hurts/confuses and I’m having to actively fight my own you’d-be-happier-not-being-here feelings, which are very strong in a way that they aren’t in the one-vote system, and wouldn’t be in either my compass rose system or Rob’s heart/X system.
do people think it’s good but wrong [...] I hate this
The parent comment serves as a counterexample to this interpretation: It seems natural to agreement-downvote your comment to indicate that I don’t share this feeling/salient-impression, without meaning to communicate that I believe your feeling-report to be false (about your own impression). And to karma-upvote it to indicate that I care for existence of this feeling to become a known issue and to incentivise corroboration from others (with visibility given by karma-upvoting) who feel similarly (which might in part be communicated with agreement-upvoting).
there is some effortful, System-2 processing that I could do
The important distinction is about existence of System-1 distillation that enables ease, which develops with a bit of exposure, and of the character of that distillation. (Is it ugly/ruinous/not-forming, despite the training data being fine?) Whether a new thing is immediately familiar is much less strategically relevant.
This function has been available, and I’ve encountered it off and on, for months. This isn’t a case of “c’mon, give it a few tries before you judge it.” I’ve had more than a bit of exposure.
If being highly upvoted yet highly disagreed with make you feel deterred and never want to comment again, wouldn’t that also be the case if you see a lot of light orange beside your comments?
Since it seems unlikely you’ll forget your own proposal nor what the colours correspond to.
In fact it may hasten your departure since bright colours are a lot more difficult to ignore than a grey number.
I do not have a model/explanation for why, but no, apparently not. I’ve got pretty decent introspection and very good predicting-future-Duncan’s-responses skill and the light orange does not produce the same demoralization as negative numbers.
Though the negative numbers also produce less demoralization if the prompt is changed in accordance with some suggestions to something like “I could truthfully say this or something close to it from my own beliefs and experience.”
From my perspective, the upvotes/downvotes are less important info
Their role is different: it’s about quality/incentives, so the appropriate way of deciding visibility (comment ordering) and aggregating into user’s overall footprint/contribution. Agreement clarifies attitude to individual comments without compromising the quality vote, in particular making it straightforward/convenient to express approval/incentivization of disagreed-with comments. In this way agreement vote improves fidelity of the more strategic quality/incentives vote, while communicating an additional tactical fact about each particular comment.
After using the new system for a couple of days, I now believe that a single-click[1] system, like the one Duncan describes, would probably be preferable for interaction efficiency / satisfaction reasons. (Having to click on two different UI widgets in two different screen locations—i.e., mouse move, click, another mouse move, click—is an annoyance.)
One downside of Duncan’s proposed widget design would be that it cannot accommodate the full range of currently permissible input values. The current two-widget system has 25 possible states (karma and agreement can each independently take on any of five values: ++, +, 0, −, −−), while the proposed “blue and orange compass rose” single-widget system has only 9 possible states (the neutral state, plus two strengths of vote × four directions).
It is not immediately obvious to me what an ideal solution would look like. The obvious solution (in terms of interaction design) would be to construct a mapping from the 25 states to the 9, wherein some of the 25 currently available input states should be impermissible, and some sets of the remainder of the 25 should each be collapsed into one of the 9 input states of the proposed widget. (I haven’t thought about the problem enough to know if a satisfactory such mapping exists.)
Or, to be more precise, a “single-click / double-click / click-and-hold, depending on implementation details and desired outcome, but in all cases a pointer interaction with only one UI widget in one location on the screen” system.
Hypothetically, you could represent all of the states by using the diamond, but adding a second ‘diamond’ or ‘shell’ around it, and making all of the vertexes and regions clickable. To express a +/+ you click in the upper right region; to express ++/++, the uppermost right region; to express 0/++, you click on the right-most tip; to express ++/0, you click on the bottom tip; and so on. The regions can be colored. (And for users who don’t get strong votes, it degrades nicely: you should omit the outer shell corresponding to the strong votes.) I’m sure I’ve seen this before in video games or something, but I’m not sure where or what it may be called (various search queries for ‘diamond’ don’t pull up anything relevant). It’s a bit like a radar chart, but discretized.
This would be easy to use (as long as the vertexes have big hit boxes) since you make only 1 click (rather than needing to click 4 times or hold long twice for a ++/++) and use the mouse to choose what the pair is (which is a good use of mice), and could be implemented even as far back as Web 1.0 with imagemaps, but somewhat hard to explain—however, that’s what tooltips are for, and this is for power-users in the first place, so some learning curve is tolerable.
I find all of the four-way graphical depictions in this subthread to be horribly confusing at an immediate glance; indeed I had to fight against reversing the axes on the one you showed. I already know what a karma system is, as imperfect as that is, and I already know what agreement and disagreement are—and being able to choose to only use one of the axes at any given moment is an engagement win for me, because (for instance) if I start out wanting to react “yes, I agree” but then have to think about “but also was the post good in a different sense” before I can record my answer, or vice versa, that means I have to perform the entire other thought-train with the first part in my working-memory stack. And my incentive and inclination to vote at all doesn’t start out very high. It’s like replacing two smaller stairs with one large stair of combined height.
A more specific failure mode there is lack of representation for null results on either axis, especially “I think this was a good contribution, but it makes no specific claims to agree or disagree with and instead advances the conversation some other way” and “I think this was a good contribution, but I will have to think for another week to figure out whether I agree or disagree with it on the object level, and vaguely holding onto the voting-intention in the back of my mind for a week gives a horrible feeling of cruftifying my already-strained attention systems”.
To try to expand the separate-axes system in this exact case, I have upvoted your comment here on the quality axis and also marked my disagreement (which I don’t like the term “downvote” for, as described elsewhere), because I think it’s a good thing that you went to the effort of thinking about this and posting about it, and I think the explanation is coherent and reasonable, but I also think the suggestion itself would be more complicated and difficult and overall worse than what’s been implemented, largely because of differing impressions of the actual world. I think this is much better than having the choice of downvoting the comment and thus indicating that I wish you hadn’t posted this, which is false, or upvoting it and risking a perception that I wanted the proposal to be implemented, which is also false.
I have meta-difficulty grasping why you find the at-a-glance compounding of info difficult under the separate-axes system. Do you feel inclined to try to explain it differently? In particular, I do not understand why “a lot more people” and “strong/weak votes” play so heavily into your reported thought process. I process the numbers as pre-aggregated information with a stock “congealed” (one-mental-step access) approximate correction-and-blurring model of the imperfections of the aggregation and selection most of the time. Trying to dig further is rare and mainly happens if I’m quite surprised by what I see initially.
I think many of the UI ideas here are potentially interesting, but one major issue is the amount of space we have to work with. The design here is particularly cool because it’s a compass rose which matches the LW logo, but… I don’t see how we could fit a version into every comment that actually worked. (Maybe if it was little but got big when you hovered over it?)
(to be clear these seem potentially fixable, just noting that that’s where my attention goes next in a problem-solving-y way)
FYI In my mind there’s still some radically different solutions that might be worth trying for agree/disagree, I’m still pretty uncertain about the whole thing.
I found the current UI intuitive. I find the four-pointed star you suggested confusing (though mayyyybe I’d like it if I got used to it?). I tend to mix up my left and my right, and I don’t associate left/right with false/true at all, nor do I associate blue with “truth”. (if anything, I associate blue more with goodness, so I might have guessed dark-blue was ‘good and true’ and light-blue was ‘good and false’)
A version of this I’m confident would be easier for me to track is, e.g.:
It’s less pretty, but:
The shapes give me an indication of what each direction means. ✔ and ✖ I think are very useful and clear in that respect: to me, they’re obviously about true/false rather than good/bad.
Green vs. red still isn’t super clear. But it’s at least clearer than blue vs. red, to me; and if I forget what the colors mean, I have clear indicators via ‘ah, there’s a green X, but no green checkmark, because the heart is the special “good on all dimensions” symbol, and because green means “good” (so it would be redundant to have a green heart and a green checkmark)’.
The left and right options are smaller and more faded. Some consequences:
(a) This makes the image as a whole feel less overwhelming, because there’s a clear hierarchy that encourages me to first pay attention to one thing, then only consider the other thing as an afterthought. In this case, I first notice the heart and X, which give me an anchor for what green, red, and X mean. Then I notice the smaller symbols, which I can then use my anchors to help interpret. This is easier than trying to parse four symbols at the exact same moment, especially when those symbols have complicated interactions rather than being primitives.
I think this points at the core reason Duncan’s proposal is harder for me to fit in my head than the status quo: my working memory can barely handle four things at once, and the four options here are really ordered pairs. At least, my brain thinks of them as ordered pairs rather than as primitives: I don’t have four distinct qualia or images or concepts for (true, good), (true, bad), (false, good), and (false,bad), I just have the dichotomies “true v. false” and “good v. bad”. Trying to compare all four options at once overloads my brain, whereas trying to compare two things (good v. bad) and then two other things (true v. false) is a lot easier for me.
Having a “heart” symbol is a step in the right direction in that respect, because it’s closer to “a unitary concept” in my mind rather than “an ordered pair”. If I had four very clearly distinct symbols for the four options, and they all made sense to me and were hard to confuse for each other, then that might more-or-less solve the problem for me.
(b) This makes it easier for me to chunk the two faded options as a separate category, and to think my way to ‘what does these mean?’ hierarchically: first I notice that these are the two ‘mixed’ options (because they’re small and faded and off to the sides), then I notice which one is ‘true mixed’ versus ‘false mixed’ (because true mixed will have a check, while false mixed has an X).
Here’s a version that’s probably closer to what would actually work for me:
Now all four are closer to being conceptual primitives for me. 💚 is ‘good on all the dimensions’; ❌ is ‘bad on all the dimensions’.
The facepalm emoji is meant to evoke a specific emotional reaction: that exasperated feeling I get when I see someone saying a thing that’s technically true but is totally irrelevant, or counter-productive. (Colored purple because purple is an ‘ambiguous but bad-leaning’ color, e.g., in Hollywood movies, and is associated with villainy and trolling.)
The shaking-head icon is meant to evoke another emotional reaction: the feeling of being a teacher who’s happy with their student’s performance, but is condescendingly shaking their head to say “No, you got the wrong answer”. (Colored blue because blue is ‘ambiguous but good-leaning’ and is associated with innocence and youthful naïveté.)
Neither of these emotional reactions capture the range of situations where I’d want to vote (true,bad) or (false,good). But my goal is to give me a vivid, salient handle at all on what the symbols might mean, at a glance; I think the hard part for me is rapidly distinguishing the symbols at all when there are so many options, not so much ‘figuring out the True Meaning of the symbol once I’ve distinguished it from the other three’.
I don’t like my own proposals, so do the disagree-votes mean that you agree with me that these are bad proposals, or do they mean you disagree with me and think they’re good? :P
With two axes, each on a scale -strong/-weak/null/weak/strong, there are 24 non-trivial possibilities. Why have you chosen these four, excluding such things as “this is an important contribution that I completely agree with”, or “this is balderdash on both axes”?
Subjective sense of what would make LessWrong both a) more a place I’m excited to be, and b) (not unrelatedly) more of a place that helps me be better according to my own goals and values.
I’m also struggling to interpret cases where karma & agreement diverge, and would also prefer a system that lets me understand how individuals have voted. E.g. Duncan’s comment above currently has positive karma but negative agreement, with different numbers of upvotes and agreement votes. There are many potential voting patterns that can have such a result, so it’s unclear how to interpret it.
Whereas in Duncan’s suggestion, a) all votes contain two bits of information and hence take a stand on something like agreement (so there’s never a divergence between numbers of votes on different axes), and b) you can tell if e.g. your score is the result of lots of voters with “begrudging upvotes”, or “conflicted downvotes” or something.
Whereas in Duncan’s suggestion, a) all votes contain two bits of information and hence take a stand on something like agreement
I didn’t notice that! I don’t want to have to decide on whether to reward or punish someone every time I figure out whether they said a true or false thing. Seems like it would also severely enhance the problem of “people who say things that most people believe get lots of karma”.
The alternative solutions you are gesturing at do communicate the problems of the current solution, but I think they are worse than the current solution, and I’m not sure there is a feasible UI change that’s significantly better than the current solution (among methods for collecting the data with the same meaning, quality/agreement score). Being convenient to use and not using up too much space are harsh constraints.
Still somewhat sad about this, as it feels to me like a half-solution that’s pumping against human nature.
My claims in previous discussions about alternate voting systems were that:
It was going to be really important to have a single click, not multiple clicks (i.e. any two-separate-votes system was going to have people overwhelmingly just using one of the votes and largely ignoring the second one)
It was going to be really important to use visual cues and directionality and not just have two things side by side
I wanted something like the following:
… where users would single-click one of the four buttons and could click-hold to strong vote, but would with a single click show:
Upvote this and also it’s true (the dark blue one)
This seems true but slight downvote/it’s not helping/it’s making things worse (the light blue one)
This seems false or sketchy but slight upvote/it’s helping/I’m glad it’s here (the light orange one)
Downvote this and also it’s false (the dark orange one).
True-false is on the forward/backward axis, in other words, and good/bad is on the vertical axis, as usual.
The display for aggregation could look a lot of different ways; please don’t hate the below for its ugly first-draft nature:
For example, maybe the numbers are invisible unless you hover, or something. In this example, the blue shows which one I clicked.
This would also allow you (if you wanted) to display an aggregated “what’s this user’s rep” function—you could make some hover-over thing or put a place on people’s profile pages where you could see whether a given user was generally given dark blue, light blue, light orange, or dark orange feedback.
The reason I’m sad about the thing that’s happening here is that:
I suspect it’s a half-solution that will decay back to mostly-people-just-use-the-first-vote
I don’t think it lets me grok the quality of the reaction to a comment at a glance; I keep having to effortfully process “okay, what does—okay, this means that people like it but think it’s slightly false, unless they—hmm, a lot more people voted up-down than true-false, unless they all strong voted up-down but weak-voted tru—you know what, I can’t get any meaningful info out of this.”
I suspect it will sap whatever momentum there was for a truly better voting system, such that if it turns out I’m right and a year from now this didn’t really help, it’ll be even less likely that we can muster for another attempt.
Agreement votes must never be aggregated, otherwise there is incentive for uncontroversial commenting.
I agree with the sentiment here, but I think you have too little faith in some people’s willingness to be disagreeable… especially on LessWrong! Personally I’d feel fine/great about having a high karma and a low net-agreement score, because it means I’m adding a unique perspective to the community that people value.
… and, I’d go so far as to bet that the large number of agreement with your comment here is representative of a bunch of users that would feel similarly, but I’m putting this in a separate comment so accrues a separate agree/disagree score. If lots of people disagree, I’ll update :)
Absolutely! Agree 100%.
I feel like it’s slightly less work for me to consider one axis and up/downvote it, and then consider the second axis and up/downvote it, than it’d be to vote on two axes with a single click. The former lets me consider the two separately, “make one decision and then forget about it”, whereas the latter requires me to think about both at the same time. That means that I’m (slightly) more likely to cast two votes on a multiple-click system than on a single-click system.
Though I do also consider it a feature if the system allows me to only cast one vote rather than forcing me to do both. E.g. in situations where I want to upvote a domain expert’s comment giving an explanation about a domain that I’m not familiar with, so don’t feel qualified to cast a vote on its truth even though I want to indicate that I appreciate having the explanation.
Regardless of whether it’s a bad solution in other respects, I predict that people will use the agree/disagree vote a ton, reliably, forever.
I mostly care about agree/disagree votes (especially when it comes to specifics). From my perspective, the upvotes/downvotes are less important info; they’re mostly there to reward good behavior and make it easier to find the best content fast.
In that respect, the thing that annoys me about agree/disagree votes isn’t any particular relationship to the upvotes/downvotes; it’s that there isn’t a consistent way to distinguish ‘a few people agreeing strongly’ from ‘a larger number of people agreeing weakly’, ‘everyone agrees with this but weakly’ from ‘some agree strongly but they’re being partly offset by others who disagree’, or ‘this is the author agreeing with their own comment’ from ‘this is a peer independently vouching for the comment’s accuracy’.
I think all of those things would ideally be distinguishable, at least on hover. (Or the ambiguity would be eliminated by changing how the feature works—e.g., get rid of the strong/weak distinction for agreevotes, get rid of the thing where users can agreevote their own comments, etc.)
The specific thing I’d suggest is to get rid of ‘authors can agree/disagree vote on their own comments’ (LW already has a ‘disendorse’ feature), and to replace the current UI with a tiny bar graph showing the rough relative number of strong agree, weak agree, strong disagree, and weak disagree votes (at least on hover).
I feel zero motivation to use it. I feel zero value gained from it, in its current form. I actually find it a deterrent, e.g. looking at the information coming in on my comment above gave me a noticeable “ok just never comment on LW again” feeling.
(I now fear social punishment for admitting this fact, like people will decide that me having detected such an impulse means I’m some kind of petty or lame or bad or whatever, but eh, it’s true and relevant. I don’t find downvotes motivationally deterring in the same fashion, at all.)
EDIT: this has been true in other instances of looking at these numbers on my other comments in the past; not an isolated incident.
More detail on the underlying emotion:
“Okay, so it’s … it’s plus eight, on some karma meaning … something, but negative nine on agreement? What the heck does this even mean, do people think it’s good but wrong, are some people upvoting but others downvoting in a different place—I hate this. I hate everything about this. Just give up and go somewhere where the information is clear and parse-able.”
Like, maybe it would feel better if I could see something that at least confirmed to me how many people voted in both places? So I’m not left with absolutely no idea how to compare the +8 to the −9?
But overall it just hurts/confuses and I’m having to actively fight my own you’d-be-happier-not-being-here feelings, which are very strong in a way that they aren’t in the one-vote system, and wouldn’t be in either my compass rose system or Rob’s heart/X system.
The parent comment serves as a counterexample to this interpretation: It seems natural to agreement-downvote your comment to indicate that I don’t share this feeling/salient-impression, without meaning to communicate that I believe your feeling-report to be false (about your own impression). And to karma-upvote it to indicate that I care for existence of this feeling to become a known issue and to incentivise corroboration from others (with visibility given by karma-upvoting) who feel similarly (which might in part be communicated with agreement-upvoting).
I think you’re confusing “this should make sense to you, Duncan” with “therefore this makes sense to you, Duncan”
(or more broadly, “this should make sense to people” with “therefore, it will/will be good.”)
I agree that there is some effortful, System-2 processing that I could do, to draw out the meaning that you have spelled out above.
The important distinction is about existence of System-1 distillation that enables ease, which develops with a bit of exposure, and of the character of that distillation. (Is it ugly/ruinous/not-forming, despite the training data being fine?) Whether a new thing is immediately familiar is much less strategically relevant.
This function has been available, and I’ve encountered it off and on, for months. This isn’t a case of “c’mon, give it a few tries before you judge it.” I’ve had more than a bit of exposure.
If being highly upvoted yet highly disagreed with make you feel deterred and never want to comment again, wouldn’t that also be the case if you see a lot of light orange beside your comments?
Since it seems unlikely you’ll forget your own proposal nor what the colours correspond to.
In fact it may hasten your departure since bright colours are a lot more difficult to ignore than a grey number.
I do not have a model/explanation for why, but no, apparently not. I’ve got pretty decent introspection and very good predicting-future-Duncan’s-responses skill and the light orange does not produce the same demoralization as negative numbers.
Though the negative numbers also produce less demoralization if the prompt is changed in accordance with some suggestions to something like “I could truthfully say this or something close to it from my own beliefs and experience.”
Their role is different: it’s about quality/incentives, so the appropriate way of deciding visibility (comment ordering) and aggregating into user’s overall footprint/contribution. Agreement clarifies attitude to individual comments without compromising the quality vote, in particular making it straightforward/convenient to express approval/incentivization of disagreed-with comments. In this way agreement vote improves fidelity of the more strategic quality/incentives vote, while communicating an additional tactical fact about each particular comment.
After using the new system for a couple of days, I now believe that a single-click[1] system, like the one Duncan describes, would probably be preferable for interaction efficiency / satisfaction reasons. (Having to click on two different UI widgets in two different screen locations—i.e., mouse move, click, another mouse move, click—is an annoyance.)
One downside of Duncan’s proposed widget design would be that it cannot accommodate the full range of currently permissible input values. The current two-widget system has 25 possible states (karma and agreement can each independently take on any of five values: ++, +, 0, −, −−), while the proposed “blue and orange compass rose” single-widget system has only 9 possible states (the neutral state, plus two strengths of vote × four directions).
It is not immediately obvious to me what an ideal solution would look like. The obvious solution (in terms of interaction design) would be to construct a mapping from the 25 states to the 9, wherein some of the 25 currently available input states should be impermissible, and some sets of the remainder of the 25 should each be collapsed into one of the 9 input states of the proposed widget. (I haven’t thought about the problem enough to know if a satisfactory such mapping exists.)
Or, to be more precise, a “single-click / double-click / click-and-hold, depending on implementation details and desired outcome, but in all cases a pointer interaction with only one UI widget in one location on the screen” system.
Hypothetically, you could represent all of the states by using the diamond, but adding a second ‘diamond’ or ‘shell’ around it, and making all of the vertexes and regions clickable. To express a +/+ you click in the upper right region; to express ++/++, the uppermost right region; to express 0/++, you click on the right-most tip; to express ++/0, you click on the bottom tip; and so on. The regions can be colored. (And for users who don’t get strong votes, it degrades nicely: you should omit the outer shell corresponding to the strong votes.) I’m sure I’ve seen this before in video games or something, but I’m not sure where or what it may be called (various search queries for ‘diamond’ don’t pull up anything relevant). It’s a bit like a radar chart, but discretized.
This would be easy to use (as long as the vertexes have big hit boxes) since you make only 1 click (rather than needing to click 4 times or hold long twice for a ++/++) and use the mouse to choose what the pair is (which is a good use of mice), and could be implemented even as far back as Web 1.0 with imagemaps, but somewhat hard to explain—however, that’s what tooltips are for, and this is for power-users in the first place, so some learning curve is tolerable.
I find all of the four-way graphical depictions in this subthread to be horribly confusing at an immediate glance; indeed I had to fight against reversing the axes on the one you showed. I already know what a karma system is, as imperfect as that is, and I already know what agreement and disagreement are—and being able to choose to only use one of the axes at any given moment is an engagement win for me, because (for instance) if I start out wanting to react “yes, I agree” but then have to think about “but also was the post good in a different sense” before I can record my answer, or vice versa, that means I have to perform the entire other thought-train with the first part in my working-memory stack. And my incentive and inclination to vote at all doesn’t start out very high. It’s like replacing two smaller stairs with one large stair of combined height.
A more specific failure mode there is lack of representation for null results on either axis, especially “I think this was a good contribution, but it makes no specific claims to agree or disagree with and instead advances the conversation some other way” and “I think this was a good contribution, but I will have to think for another week to figure out whether I agree or disagree with it on the object level, and vaguely holding onto the voting-intention in the back of my mind for a week gives a horrible feeling of cruftifying my already-strained attention systems”.
To try to expand the separate-axes system in this exact case, I have upvoted your comment here on the quality axis and also marked my disagreement (which I don’t like the term “downvote” for, as described elsewhere), because I think it’s a good thing that you went to the effort of thinking about this and posting about it, and I think the explanation is coherent and reasonable, but I also think the suggestion itself would be more complicated and difficult and overall worse than what’s been implemented, largely because of differing impressions of the actual world. I think this is much better than having the choice of downvoting the comment and thus indicating that I wish you hadn’t posted this, which is false, or upvoting it and risking a perception that I wanted the proposal to be implemented, which is also false.
I have meta-difficulty grasping why you find the at-a-glance compounding of info difficult under the separate-axes system. Do you feel inclined to try to explain it differently? In particular, I do not understand why “a lot more people” and “strong/weak votes” play so heavily into your reported thought process. I process the numbers as pre-aggregated information with a stock “congealed” (one-mental-step access) approximate correction-and-blurring model of the imperfections of the aggregation and selection most of the time. Trying to dig further is rare and mainly happens if I’m quite surprised by what I see initially.
I think many of the UI ideas here are potentially interesting, but one major issue is the amount of space we have to work with. The design here is particularly cool because it’s a compass rose which matches the LW logo, but… I don’t see how we could fit a version into every comment that actually worked. (Maybe if it was little but got big when you hovered over it?)
(to be clear these seem potentially fixable, just noting that that’s where my attention goes next in a problem-solving-y way)
FYI In my mind there’s still some radically different solutions that might be worth trying for agree/disagree, I’m still pretty uncertain about the whole thing.
Yeah, the UI issues seem real and substantial.
In my mind, the thing is roughly as tall as the entire box holding the current vote buttons.
hmm, I could see that working. Click-target seems small-ish but maybe fixable with some-UI magic (and maybe it’s actually just fine?)
I found the current UI intuitive. I find the four-pointed star you suggested confusing (though mayyyybe I’d like it if I got used to it?). I tend to mix up my left and my right, and I don’t associate left/right with false/true at all, nor do I associate blue with “truth”. (if anything, I associate blue more with goodness, so I might have guessed dark-blue was ‘good and true’ and light-blue was ‘good and false’)
A version of this I’m confident would be easier for me to track is, e.g.:
It’s less pretty, but:
The shapes give me an indication of what each direction means. ✔ and ✖ I think are very useful and clear in that respect: to me, they’re obviously about true/false rather than good/bad.
Green vs. red still isn’t super clear. But it’s at least clearer than blue vs. red, to me; and if I forget what the colors mean, I have clear indicators via ‘ah, there’s a green X, but no green checkmark, because the heart is the special “good on all dimensions” symbol, and because green means “good” (so it would be redundant to have a green heart and a green checkmark)’.
The left and right options are smaller and more faded. Some consequences:
(a) This makes the image as a whole feel less overwhelming, because there’s a clear hierarchy that encourages me to first pay attention to one thing, then only consider the other thing as an afterthought. In this case, I first notice the heart and X, which give me an anchor for what green, red, and X mean. Then I notice the smaller symbols, which I can then use my anchors to help interpret. This is easier than trying to parse four symbols at the exact same moment, especially when those symbols have complicated interactions rather than being primitives.
I think this points at the core reason Duncan’s proposal is harder for me to fit in my head than the status quo: my working memory can barely handle four things at once, and the four options here are really ordered pairs. At least, my brain thinks of them as ordered pairs rather than as primitives: I don’t have four distinct qualia or images or concepts for (true, good), (true, bad), (false, good), and (false,bad), I just have the dichotomies “true v. false” and “good v. bad”. Trying to compare all four options at once overloads my brain, whereas trying to compare two things (good v. bad) and then two other things (true v. false) is a lot easier for me.
Having a “heart” symbol is a step in the right direction in that respect, because it’s closer to “a unitary concept” in my mind rather than “an ordered pair”. If I had four very clearly distinct symbols for the four options, and they all made sense to me and were hard to confuse for each other, then that might more-or-less solve the problem for me.
(b) This makes it easier for me to chunk the two faded options as a separate category, and to think my way to ‘what does these mean?’ hierarchically: first I notice that these are the two ‘mixed’ options (because they’re small and faded and off to the sides), then I notice which one is ‘true mixed’ versus ‘false mixed’ (because true mixed will have a check, while false mixed has an X).
Here’s a version that’s probably closer to what would actually work for me:
Now all four are closer to being conceptual primitives for me. 💚 is ‘good on all the dimensions’; ❌ is ‘bad on all the dimensions’.
The facepalm emoji is meant to evoke a specific emotional reaction: that exasperated feeling I get when I see someone saying a thing that’s technically true but is totally irrelevant, or counter-productive. (Colored purple because purple is an ‘ambiguous but bad-leaning’ color, e.g., in Hollywood movies, and is associated with villainy and trolling.)
The shaking-head icon is meant to evoke another emotional reaction: the feeling of being a teacher who’s happy with their student’s performance, but is condescendingly shaking their head to say “No, you got the wrong answer”. (Colored blue because blue is ‘ambiguous but good-leaning’ and is associated with innocence and youthful naïveté.)
Neither of these emotional reactions capture the range of situations where I’d want to vote (true,bad) or (false,good). But my goal is to give me a vivid, salient handle at all on what the symbols might mean, at a glance; I think the hard part for me is rapidly distinguishing the symbols at all when there are so many options, not so much ‘figuring out the True Meaning of the symbol once I’ve distinguished it from the other three’.
I don’t like my own proposals, so do the disagree-votes mean that you agree with me that these are bad proposals, or do they mean you disagree with me and think they’re good? :P
(I should have phrased this as a bald assertion rather than a question, so people could (dis)agree with it to efficiently reply. :P)
For me it meant “I think this is a bad proposal”.
For what it’s worth, the head icon doesn’t read to me at all like a condescending head-shake. My brain parses it as “contented face plus halo”.
With two axes, each on a scale -strong/-weak/null/weak/strong, there are 24 non-trivial possibilities. Why have you chosen these four, excluding such things as “this is an important contribution that I completely agree with”, or “this is balderdash on both axes”?
Subjective sense of what would make LessWrong both a) more a place I’m excited to be, and b) (not unrelatedly) more of a place that helps me be better according to my own goals and values.
I’m also struggling to interpret cases where karma & agreement diverge, and would also prefer a system that lets me understand how individuals have voted. E.g. Duncan’s comment above currently has positive karma but negative agreement, with different numbers of upvotes and agreement votes. There are many potential voting patterns that can have such a result, so it’s unclear how to interpret it.
Whereas in Duncan’s suggestion, a) all votes contain two bits of information and hence take a stand on something like agreement (so there’s never a divergence between numbers of votes on different axes), and b) you can tell if e.g. your score is the result of lots of voters with “begrudging upvotes”, or “conflicted downvotes” or something.
I didn’t notice that! I don’t want to have to decide on whether to reward or punish someone every time I figure out whether they said a true or false thing. Seems like it would also severely enhance the problem of “people who say things that most people believe get lots of karma”.
The alternative solutions you are gesturing at do communicate the problems of the current solution, but I think they are worse than the current solution, and I’m not sure there is a feasible UI change that’s significantly better than the current solution (among methods for collecting the data with the same meaning, quality/agreement score). Being convenient to use and not using up too much space are harsh constraints.