The astounding notion is that human beings are unbiased estimators of beans in a jar, having no significant directional error on the problem, yet with large variance. It implies that we tend to get the answer wrong but there’s no systematic reason why. It requires that there be lots of errors that vary from individual to individual—and this is reliably true, enough so to keep most individuals from guessing the jar correctly. And yet there are no directional errors that everyone makes, or if there are, they cancel out very precisely in the average case, despite the large individual variations. Which is just plain odd. I find myself somewhat suspicious of the claim, and wonder whether other experiments that found less amazing accuracy were not as popularly reported.
This is precisely what I find disquieting about wisdom-of-crowds arguments—they require that our errors are nondirectional and normally distributed, but we know they aren’t. We have cognitive biases!
It’s a similar argument to my proposal of Rational Airways, an airline that asks you to sign a release when buying a ticket to the effect that you realise how tiny the risk of a terrorist attack is, and therefore are willing to travel with Rational, who do not apply any annoying security procedures.