Some Claims Are Just Too Extraordinary

“I would sooner be­lieve that two Yan­kee pro­fes­sors would lie, than that stones would fall from heaven.”

-- Thomas Jeffer­son, on meteors

“How would I ex­plain the event of my left arm be­ing re­placed by a blue ten­ta­cle? The an­swer is that I wouldn’t. It isn’t go­ing to hap­pen.”

-- Eliezer Yud­kowsky, “A Tech­ni­cal Ex­pla­na­tion of Tech­ni­cal Ex­pla­na­tion”

“If a ship landed in my yard and LGMs stepped out, I’d push past their liter­a­ture and try to find the ca­ble that dropped the saucer on my roses. Lack of a ca­ble or any sig­nifi­cant burn­ing to the flow­ers, I’d then grab a ham­mer and start knock­ing about in the ship till I was con­vinced that noth­ing said “In­tel In­side.” Then when I dis­cov­ered a “Flux Ca­pac­i­tor” type thing I would fi­nally stop and say, “Hey, cool gad­get!” As­sum­ing the uni­ver­sal benev­olence of the LGMs, I’d yank it out and de­mand from the near­est “Grey” (they are the tall nice ones), “where the hell did this come from?” Greys don’t talk, they com­mu­ni­cate via telepa­thy, so I’d ig­nore the voice in­side my head. Then step­ping out­side the saucer and sit­ting in a lawn chair, I’d throw peb­bles at the aliens till I was sure they were solid. Then I’d look down at the “Flux Ca­pac­i­tor” and make sure it hadn’t mor­phed into my bird feeder. Fi­nally, with proof in my hand and aliens sit­ting on my deck (they’d be offered beers, though I’ve heard that they ab­sorb en­ergy like a plant) I’d grab my cell phone and tell my doc­tor that I’m hav­ing a se­ri­ous manic epi­sode with full-blown vi­sual hal­lu­ci­na­tions.”

-- Peter K. Ber­tine, on the Ex­tropian mailing list

We un­der­es­ti­mate the power of sci­ence, and over­es­ti­mate the power of per­sonal ob­ser­va­tion. A peer-re­viewed, jour­nal-pub­lished, repli­cated re­port is worth far more than what you see with your own eyes. Our own eyes can de­ceive us. Peo­ple can fool them­selves, hal­lu­ci­nate, and even go in­sane. The con­trols on pub­li­ca­tion in ma­jor jour­nals are more trust­wor­thy than the very fabric of your brain. If you see with your own eyes that the sky is blue, and Science says it is green, then sir, I ad­vise that you trust in Science.

This is not what most sci­en­tists will tell you, of course; but I think it is prag­mat­i­cally true. Be­cause in real life, what hap­pens is that your eyes have a lit­tle malfunc­tion and de­cide that the sky is green, and sci­ence will tell you that the sky is blue.

A repli­cated sci­en­tific re­port is a spe­cial kind of ex­traor­di­nary claim, de­signed by the sur­round­ing pro­cess to be more ex­traor­di­nary ev­i­dence than sim­ple ver­bal claims. It is more ex­traor­di­nary ev­i­dence be­cause the sur­round­ing pro­cess—and I would place a far greater pre­mium on the repli­ca­tion than on the peer re­view, by the way—is con­structed to deny en­trance to claims that are in fact false. In this way, the repli­cated sci­en­tific re­port be­comes ca­pa­ble of over­com­ing greater bur­dens of prior im­prob­a­bil­ity.

There are some bur­dens of prior im­prob­a­bil­ity so great that sim­ple ver­bal claims can­not over­come them. I would not be­lieve some­one who claimed that their coffee was di­s­obey­ing con­ser­va­tion of an­gu­lar mo­men­tum—but I might be­lieve the same re­port pub­lished in Physics To­day, with at least three repli­ca­tions. Who would be­lieve in quan­tum me­chan­ics if a stranger walked up to us on the street and whispered it to us?

Are there some bur­dens of prior im­prob­a­bil­ity so great that sci­ence it­self can­not over­come them?

What about the claim that 2 + 2 = 5?

What about jour­nals that claim to pub­lish repli­cated re­ports of ESP?

Some­times, even claims de­liber­ately con­structed to be ex­traor­di­nary ev­i­dence end up just not be­ing ex­traor­di­nary enough.