That still has the problem that it doesn’t test for lack of bias, but for having bias that matches that of the people who wrote the stories. I suggest instead using real cases—and not taken from the media, because that means selection bias, but taking all the cases from the files of a particular police department during a particular span of time.
What true stories can we use besides police cases? (Also, note that in this case you’re only testing for being as smart as the police or making the same judgments as the jury—even using cases with a confession may get you false confessions.)
Point. Still, we’ve been recording lots of different kinds of events for a long time. Off the top of my head, other kinds of historical data that could be useful here:
Medical cases,
minor scientific controversies,
engineering projects,
battles,
the stock market,
markets in general,
expeditions.
That still has the problem that it doesn’t test for lack of bias, but for having bias that matches that of the people who wrote the stories. I suggest instead using real cases—and not taken from the media, because that means selection bias, but taking all the cases from the files of a particular police department during a particular span of time.
What true stories can we use besides police cases? (Also, note that in this case you’re only testing for being as smart as the police or making the same judgments as the jury—even using cases with a confession may get you false confessions.)
You can take cases with enough evidence to overdetermine the result, and then subtract pieces.
Point. Still, we’ve been recording lots of different kinds of events for a long time. Off the top of my head, other kinds of historical data that could be useful here:
Medical cases, minor scientific controversies, engineering projects, battles, the stock market, markets in general, expeditions.