This comment on HN, if true, seems pretty damning (emphasis added):
I went to Cornell and I’m one of the many students that participated in this guy’s experiments (although not this particular one with the erotic pictures. I got regular pictures.)
I can tell you that every semester that I was there he was running a version of the “Are you psychic?” experiment. I’m sure he’s been doing it every semester for a very long time. Undoubtedly there have been loads of experiments where it didn’t pan out. (If you’re curious about my results, I got 54% and a cheerful grad student greeted me after the fact by saying “congrats! you’re psychic!”)
The fact is, if you run an experiment like this enough times you are going to get a significant result eventually. That’s why you have alpha values. If it’s at .05, that means that 5% of the time you’re going to get a false positive. I think that’s what this is
Oh, wow. I read the article and the bit where he said “I waited for eight years so I’d have enough data to be sure it wasn’t a fluke” sounded to me like it took him eight years to find a fluke big enough that it fell within the publishable p-value range—if this comment is true then he either doesn’t understand statistics (bad), or is manipulating the statistics (very bad). One possibility is that he’s doing this as a proof of concept that the p-value criteria is flawed: cognitive dissonance in academics trying to disbelieve a sound study showing psychic phenomenon would be a powerful force indeed to enact change.
This comment on HN, if true, seems pretty damning (emphasis added):
Oh, wow. I read the article and the bit where he said “I waited for eight years so I’d have enough data to be sure it wasn’t a fluke” sounded to me like it took him eight years to find a fluke big enough that it fell within the publishable p-value range—if this comment is true then he either doesn’t understand statistics (bad), or is manipulating the statistics (very bad). One possibility is that he’s doing this as a proof of concept that the p-value criteria is flawed: cognitive dissonance in academics trying to disbelieve a sound study showing psychic phenomenon would be a powerful force indeed to enact change.