It seems to me that the apparent failure of Intrade to function as a predictor/knowledge aggregator before the US presidential elections was an important natural experiment.
BTW, if you want hard numbers on this—Brier scores and RMSEs, I’ve calculated them for a number of predictors including Intrade. In general, Intrade was a mediocre predictor, outperformed by most predictors. In the case of the Senate races it came close to Nate Silver, but I have reason to believe that Silver scored poorly on the Senate races since Wang & Holbrook apparently called all Senate races correctly instead of missing 2 like Intrade and Silver.
BTW, if you want hard numbers on this—Brier scores and RMSEs, I’ve calculated them for a number of predictors including Intrade. In general, Intrade was a mediocre predictor, outperformed by most predictors. In the case of the Senate races it came close to Nate Silver, but I have reason to believe that Silver scored poorly on the Senate races since Wang & Holbrook apparently called all Senate races correctly instead of missing 2 like Intrade and Silver.
http://appliedrationality.org/2012/11/09/was-nate-silver-the-most-accurate-2012-election-pundit/
http://www.gwern.net/2012%20election%20predictions