mwengler comments on LW anchoring experiment: maybe

[mwengler]

I would really appreciate a very brief statement of your conclusions. My apologies, but I don’t feel like shoveling through your analysis just to find out whether there is an effect, a weak effect, a backwards effect, or whatever.

[[deleted]]

Just skip the intro, R-code and graphs (too heavy on math).

Question 1:

Is there a difference in karma between posts that received a negative initial comment and those that received a positive initial comment? (Any difference suggests that one or both is having an effect.)

Conclusion 1:

The difference in means has shrunk but not gone away; it’s large enough that 10% of the possible effect sizes (of “a negative initial comment rather than positive”) may be zero or actually be positive (increase karma) instead. This is a little concerning, but I don’t take this too seriously:

• this is not a lot of data

• as we’ve seen there are extreme outliers suggesting that the assumptions of normality may be badly wrong

• even at face value, 10 karma points doesn’t seem like it’s large enough to have any important real-world consequences (like make people leave LW who should’ve stayed)

Question 2:

Is there a difference in karma between the two kinds of initial comments, as I began to suspect during the experiment?

Conclusion 2:

As one would hope, neither group of comments ends up with net positive mean score, but they’re clearly being treated very differently: the negative comments get downvoted far more than the positive comments. I take this as perhaps implying that LW’s reputation for being negative & hostile is a bit overblown: we’re negative and hostile to poorly thought out criticisms and arguments, not fluffy praise.

tl;dr: maybe

[jsalvatier]

I was interested in the details of this, but yes, even I would have appreciated a tl:dr.