Creating lists
Suppose you are trying to create a list. It may be of the “best” popular science books, or most controversial movies of the last twenty years, tips for getting over a breakup or the most interesting cat gifs posted in the last few days.
There are many reasons for wanting to create one of these lists, but only a few main simple methods:
Voting model—This is the simplest model, but popularity doesn’t always equal quality. It is also particularly problematic for regularly updated lists (like Reddit), where a constantly changing audience can result in large amounts of duplicate content and where easily consumable content has an advantage.
Curator model—A single expect can often do an admirable job of collecting high-quality content, but this is subject to their own personal biases. It is also effort intensive to evaluate different curators to see if they have done a good job.
Voting model with (content) rules—This can cut out the irrelevant or sugary content that is often upvoted, but creating good rules is hard. Often there is no objective line between high and low-quality content. These rules can often result in conflict.
Voting model with sections—This is a solution to some of the limitations of 1 and 3. Instead of declaring some things off-topic outright, they can be thrown into their own section. This is the optimal solution, but is usually neglected.
Voting model with selection—This covers any model where only certain people are allowed to vote. Sometimes selection is extraordinarily rigorous, however, it can still be very effective when it isn’t. As an example, Metafilter charges a $5 one-time only fee and that is sufficient to keep the quality high.
The main thing is to publish the list online in such a way that people have to click and go to a new page to see each element in the list.
It took me a moment to realize this isn’t about lists, but about ranking of subjective values for a group. Start with Arrow’s Theorem, and that’s even without the problem of mapping multidimensional values onto a single ordinal ranking.
number 3 is really just punting the problem to “how do you pick rules that your selectorate agrees on”. If you had this agreement, then #1 would have worked. Or perhaps #3 is just #2 with a veneer of voting—the rulemaker is really deciding.
Also, you left off at least one actual working method for real-world ranking: bidding and payment. This translates nebulous personal values into a common measurement (the bidding currency, which may not be money—it could be time or some other resource). To some extent, this is voting model with unequal weighting.
In prioritizing stories at work, we use negotiation and consensus rather than strict voting, with a fallback to authority when necessary (rarely).
There’s a ton of work done on the problem of governance (which this is, just disguised as the problem of list making).
Liquid Democracy, Futarchy, and Quadratic Voting are all examples of cool answers to the question: How can we get groups to make good decisions?
Do you have reason to think that readers here would generally select model 1 automatically? That wouldn’t have been my expectation.
(Model 1 and its variants 3 and 4 have a big advantage: they are easy to implement.)
http://www.google.com/search?client=safari&rls=en&q=OCD+lists&ie=UTF-8&oe=UTF-8