Recently ran across this post that recommends using a Beta-Binomial distribution to more correctly represent the uncertainty that any given post adds to your overall karma. I thought it was a cool idea and would love to see what my Karma is when represented that way, rather than just adding everything together:
It gives a reasonably rigorous way of predicting how many upvotes and downvotes a post will get, given the history of the user who wrote it. Specifically, it defines a probabilistic model: for each user, we can specify a Beta distribution with various unknown parameters, and then learn those parameters from the user’s post history. The details of that learning are rather charming if you’re a statistician, or aspire to be one, but don’t translate very well.
mr-hire would like to know what his particular Beta distribution looks like. To find out, we have to adapt Moulton’s method to the LW karma system. This turns out to be a little difficult, and requires some additional modeling choices:
Moulton models votes on individual posts with a Binomial distribution, which is used for sequences of binary outcomes. In this case each voter either upvotes the post (with probability p) or downvotes it (with probability 1-p) -- we ignore non-voters since it’s hard to know how many of them there are. But a LessWrong voter has four choices: they can vote Up or Down, and they can vote Normal or Strong, so the Binomial distribution is no longer appropriate.
This is fixable with a different choice of distributions, but then you run into another problem. In LW, even normal votes vary in value: an upvote from a high-karma user is worth more than one from a low-karma user. Do we wish to model this effect, and if so how?
If you were willing to treat all user votes equally I think you could get away with using the Dirichlet-multinomial. If not, I think you have to give up on modeling individual votes and try to model karma directly, without breaking it down into its component upvotes and downvotes.
For me, just understanding the weighting for the votes based on existing karma of the voter would be sufficient to say my idea was “implemented”. Even without knowing that it was helpful to learn about that feature of the votes.
Recently ran across this post that recommends using a Beta-Binomial distribution to more correctly represent the uncertainty that any given post adds to your overall karma. I thought it was a cool idea and would love to see what my Karma is when represented that way, rather than just adding everything together:
https://moultano.wordpress.com/2013/08/21/how-karma-should-work-betabinomial/
I’d be interested in a mathy person translating that into less mathy language that roughly communicates why it’s interesting.
It gives a reasonably rigorous way of predicting how many upvotes and downvotes a post will get, given the history of the user who wrote it. Specifically, it defines a probabilistic model: for each user, we can specify a Beta distribution with various unknown parameters, and then learn those parameters from the user’s post history. The details of that learning are rather charming if you’re a statistician, or aspire to be one, but don’t translate very well.
mr-hire would like to know what his particular Beta distribution looks like. To find out, we have to adapt Moulton’s method to the LW karma system. This turns out to be a little difficult, and requires some additional modeling choices:
Moulton models votes on individual posts with a Binomial distribution, which is used for sequences of binary outcomes. In this case each voter either upvotes the post (with probability p) or downvotes it (with probability 1-p) -- we ignore non-voters since it’s hard to know how many of them there are. But a LessWrong voter has four choices: they can vote Up or Down, and they can vote Normal or Strong, so the Binomial distribution is no longer appropriate.
This is fixable with a different choice of distributions, but then you run into another problem. In LW, even normal votes vary in value: an upvote from a high-karma user is worth more than one from a low-karma user. Do we wish to model this effect, and if so how?
If you were willing to treat all user votes equally I think you could get away with using the Dirichlet-multinomial. If not, I think you have to give up on modeling individual votes and try to model karma directly, without breaking it down into its component upvotes and downvotes.
For me, just understanding the weighting for the votes based on existing karma of the voter would be sufficient to say my idea was “implemented”. Even without knowing that it was helpful to learn about that feature of the votes.