Researchers planning any sort of correlational study should post their predictions in advance on something like a prediction market setting, and correlational studies should only be publishable if they can document that the predictions were made publicly in advance. I’m talking here about correlational studies that make any sort of causal claim. Anybody who has done correlational studies knows that if you correlate 20 things with each other, you can always make up a story that links the significant correlations with each other in a plausible way, and people love stories. The evolutionary psychologists have good material to make up stories, because they can talk about ‘our primitive ancestors,’ which is a per se interesting story. The existence of a significant correlation is the beginning of trying to establish a causal claim, not the end.
Bruce_K._Britton
Karma: 5
Yes, making data and calculations available would help to check results.
Data-mining and story-telling are only misleading when the results are presented as evidence for causal links, which has been so badly misleading in so many cases in the history of science that it would be very helpful to regulate strongly the practice of drawing conclusions about causal links from correlational data. Registering predictions before studies are done would be very helpful in evaluating claims about causal links derived from correlational data.
If no predictions are registered in advance, everybody would know the results were from an exploratory study, which is fine. What we want to avoid is allowing people to do exploratory studies, but present them as if they were hypothesis testing studies.