I’m not sure to what extent these comments can be modeled as expressing a “positive” or a “negative” reaction, the nonsensical one-line explanations made them mostly “insane” reactions (in my perception), which might overshadow the intended interpretation. It might have been a cleaner test if there were no explanations, or if you made an effort to carefully rationalize the random judgments (although that would be a more significant interference).
It’s a “damned if you do, damned if you don’t” sort of dilemma.
I know from watching them plummet into oblivion that comments which are just “Upvoted” or “Downvoted” are not a good idea for any anchoring question—they’ll quickly be hidden, so any effect size will be a lot smaller than usual, and it’s possible that hidden comments themselves anchor (my guess: negatively, by making people think “why is this attracting stupid comments?’).
While if you go with more carefully rationalized comments, that’s sort of like http://xkcd.com/810/ and starts to draw on the experimenter’s own strengths & weaknesses (I’m sure I could make both quality criticisms and praises of psychology-related articles, but not so much technical decision theory articles).
I hoped my strategy would be a golden mean of not too trivial to be downvoted into oblivion, but not so high-quality and individualized that comparability was lost. I think I came close, since the positive comments saw only a small negative net downvote, indicating LWers may not have regarded it as good enough to upvote but also not so obviously bad as to merit a downvote.
(Of course, I didn’t expect the positive and negative comments to be treated differently—they’re pretty much the same thing, with a negation. I’m not sure how I would have designed it differently if I had known about the double-standard in advance.)
Of course, I didn’t expect the positive and negative comments to be treated differently
(Positive and somewhat stupid comments tend to be upvoted back to 0 even after they get downvoted at some point, so it’s not just absence of response. I consider it a dangerous vulnerability of LW to poorly thinking but socially conforming participants, whose active participation should be discouraged, but who are instead mildly rewarded.)
I consider it a dangerous vulnerability of LW to poorly thinking but socially conforming participants, whose active participation should be discouraged, but who are instead mildly rewarded.
It’s a huge problem that I have observed eroding quality of thought and discussion over time. I’m relieved to see others acknowledge it.
A respected member saying “I know, right?” as you just did is valuable evidence, whereas the same from a no-name poster is noise. The naive reaction risks forming cliques with mutual back-scratching from big names.
Full disclosure: That kind of fluff is how I got most of my karma.
I’m not sure to what extent these comments can be modeled as expressing a “positive” or a “negative” reaction, the nonsensical one-line explanations made them mostly “insane” reactions (in my perception), which might overshadow the intended interpretation. It might have been a cleaner test if there were no explanations, or if you made an effort to carefully rationalize the random judgments (although that would be a more significant interference).
It’s a “damned if you do, damned if you don’t” sort of dilemma.
I know from watching them plummet into oblivion that comments which are just “Upvoted” or “Downvoted” are not a good idea for any anchoring question—they’ll quickly be hidden, so any effect size will be a lot smaller than usual, and it’s possible that hidden comments themselves anchor (my guess: negatively, by making people think “why is this attracting stupid comments?’).
While if you go with more carefully rationalized comments, that’s sort of like http://xkcd.com/810/ and starts to draw on the experimenter’s own strengths & weaknesses (I’m sure I could make both quality criticisms and praises of psychology-related articles, but not so much technical decision theory articles).
I hoped my strategy would be a golden mean of not too trivial to be downvoted into oblivion, but not so high-quality and individualized that comparability was lost. I think I came close, since the positive comments saw only a small negative net downvote, indicating LWers may not have regarded it as good enough to upvote but also not so obviously bad as to merit a downvote.
(Of course, I didn’t expect the positive and negative comments to be treated differently—they’re pretty much the same thing, with a negation. I’m not sure how I would have designed it differently if I had known about the double-standard in advance.)
(Positive and somewhat stupid comments tend to be upvoted back to 0 even after they get downvoted at some point, so it’s not just absence of response. I consider it a dangerous vulnerability of LW to poorly thinking but socially conforming participants, whose active participation should be discouraged, but who are instead mildly rewarded.)
It’s a huge problem that I have observed eroding quality of thought and discussion over time. I’m relieved to see others acknowledge it.
A respected member saying “I know, right?” as you just did is valuable evidence, whereas the same from a no-name poster is noise. The naive reaction risks forming cliques with mutual back-scratching from big names.
Full disclosure: That kind of fluff is how I got most of my karma.