Looking at the comments on the quiz link post, the average was 34.8/42. Using the same scoring system the prediction market would have got 36⁄42 (3 didn’t replicate which the market gave >50% credence to). If competent laypeople (some of whom by their own admission didn’t spend long on the task) can get 34.8, 36 doesn’t seem unreasonable.
I think the paper looks especially impressive due to the ability to give probability estimates but having done the quiz the ones I got wrong were amongst those which I felt least confident about.
Looking at the comments on the quiz link post, the average was 34.8/42. Using the same scoring system the prediction market would have got 36⁄42 (3 didn’t replicate which the market gave >50% credence to). If competent laypeople (some of whom by their own admission didn’t spend long on the task) can get 34.8, 36 doesn’t seem unreasonable.
I think the paper looks especially impressive due to the ability to give probability estimates but having done the quiz the ones I got wrong were amongst those which I felt least confident about.