Parapsychology: the control group for science

Para­psy­chol­o­gists are con­stantly protest­ing that they are play­ing by all the stan­dard sci­en­tific rules, and yet their re­sults are be­ing ig­nored—that they are un­fairly be­ing held to higher stan­dards than ev­ery­one else. I’m will­ing to be­lieve that. It just means that the stan­dard statis­ti­cal meth­ods of sci­ence are so weak and flawed as to per­mit a field of study to sus­tain it­self in the com­plete ab­sence of any sub­ject mat­ter.

— Eliezer Yud­kowsky, Fre­quen­tist Statis­tics are Fre­quently Subjective

Imag­ine if, way back at the start of the sci­en­tific en­ter­prise, some­one had said, “What we re­ally need is a con­trol group for sci­ence—peo­ple who will be­have ex­actly like sci­en­tists, do­ing ex­per­i­ments, pub­lish­ing jour­nals, and so on, but whose field of study is com­pletely empty: one in which the null hy­poth­e­sis is always true.

“That way, we’ll be able to gauge the effect of pub­li­ca­tion bias, ex­per­i­men­tal er­ror, mi­suse of statis­tics, data fraud, and so on, which will help us un­der­stand how se­ri­ous such prob­lems are in the real sci­en­tific liter­a­ture.”

Isn’t that a great idea?

By an ac­ci­dent of his­tor­i­cal chance, we ac­tu­ally have ex­actly such a con­trol group, namely para­psy­chol­o­gists: peo­ple who study ex­tra-sen­sory per­cep­tion, telepa­thy, pre­cog­ni­tion, and so on.

There’s no par­tic­u­lar rea­son to think para­psy­chol­o­gists are do­ing any­thing other than what sci­en­tists would do; their ex­per­i­ments are similar to those of sci­en­tists, they use statis­tics in similar ways, and there’s no rea­son to think they falsify data any more than any other group. Yet de­spite the fact that their null hy­pothe­ses are always true, para­psy­chol­o­gists get pos­i­tive re­sults.

This is dis­turb­ing, and must lead us to won­der how many pos­i­tive re­sults in real sci­ence are ac­tu­ally wrong.

The point of all this is not to mock para­psy­chol­ogy for the sake of it, but rather to em­pha­sise that para­psy­chol­ogy is use­ful as a con­trol group for sci­ence. Scien­tists should aim to im­prove their pro­ce­dures to the point where, if the con­trol group used these same pro­ce­dures, they would get an ac­cept­ably low level of pos­i­tive re­sults. That this is not yet the case in­di­cates the need for more stringent sci­en­tific pro­ce­dures.


The idea for this mini-es­say and many of its ac­tual points were sug­gested by (or stolen from) Eliezer Yud­kowsky’s Fre­quen­tist Statis­tics are Fre­quently Sub­jec­tive, though the idea might have origi­nated with Michael Vas­sar.

This was origi­nally pub­lished at a differ­ent lo­ca­tion on the web, but was moved here for band­width rea­sons at Eliezer’s sug­ges­tion.

Com­ments /​ criticisms

A dis­cus­sion on Hacker News con­tained one very as­tute crit­i­cism: that some things which may once have been con­sid­ered part of para­psy­chol­ogy ac­tu­ally turned out to be real, though with perfectly sen­si­ble, phys­i­cal causes. Still, I think this is un­likely for the more ex­otic sub­jects like telepa­thy, pre­cog­ni­tion, et cetera.