The fact that the leaderboard has someone with a billion points, because they have been participating for years, is kind-of irrelevant, and misleading.
There are many leaderboards, including ones that only consider questions that opened recently. Or tournaments with a distinct start and end date.
(And this would do a far better job aligning incentives on questions than the current leaderboard system, since for a leaderboard system, proper scoring rules for points are not actually incentive compatible.)
This is true, but you can create leaderboards that minimize the incentive to use variance-increasing strategies (or variance-decreasing ones if you’re in the lead). (Basically just include a lot of questions so that variance-increasing strategies will most likely backfire, and then have gradually increasing payouts for better rankings.)
I agree that what you describe sounds ideal, and maybe it makes sense for Metaculists to think of the points in that way. For making it a reality, I worry that it would cost a lot. (And you’d need a solution against the problem that everyone who wants a few extra dollars could create an account to predict the community median on every question to get some fraction of the total prize pool for just that.)
There are many leaderboards, including ones that only consider questions that opened recently. Or tournaments with a distinct start and end date.
This is true, but you can create leaderboards that minimize the incentive to use variance-increasing strategies (or variance-decreasing ones if you’re in the lead). (Basically just include a lot of questions so that variance-increasing strategies will most likely backfire, and then have gradually increasing payouts for better rankings.)
I agree that what you describe sounds ideal, and maybe it makes sense for Metaculists to think of the points in that way. For making it a reality, I worry that it would cost a lot. (And you’d need a solution against the problem that everyone who wants a few extra dollars could create an account to predict the community median on every question to get some fraction of the total prize pool for just that.)