Scientific Evidence, Legal Evidence, Rational Evidence

Sup­pose that your good friend, the po­lice com­mis­sioner, tells you in strictest con­fi­dence that the crime king­pin of your city is Wulky Wilk­insen. As a ra­tio­nal­ist, are you li­censed to be­lieve this state­ment? Put it this way: if you go ahead and mess around with Wulky’s teenage daugh­ter, I’d call you foolhardy. Since it is pru­dent to act as if Wulky has a sub­stan­tially higher-than-de­fault prob­a­bil­ity of be­ing a crime boss, the po­lice com­mis­sioner’s state­ment must have been strong Bayesian ev­i­dence.

Our le­gal sys­tem will not im­prison Wulky on the ba­sis of the po­lice com­mis­sioner’s state­ment. It is not ad­mis­si­ble as le­gal ev­i­dence. Maybe if you locked up ev­ery per­son ac­cused of be­ing a crime boss by a po­lice com­mis­sioner, you’d ini­tially catch a lot of crime bosses, plus some peo­ple that a po­lice com­mis­sioner didn’t like. Power tends to cor­rupt: over time, you’d catch fewer and fewer real crime bosses (who would go to greater lengths to en­sure anonymity) and more and more in­no­cent vic­tims (un­re­strained power at­tracts cor­rup­tion like honey at­tracts flies).

This does not mean that the po­lice com­mis­sioner’s state­ment is not ra­tio­nal ev­i­dence. It still has a lop­sided like­li­hood ra­tio, and you’d still be a fool to mess with Wulky’s teenager daugh­ter. But on a so­cial level, in pur­suit of a so­cial goal, we de­liber­ately define “le­gal ev­i­dence” to in­clude only par­tic­u­lar kinds of ev­i­dence, such as the po­lice com­mis­sioner’s own ob­ser­va­tions on the night of April 4th. All le­gal ev­i­dence should ideally be ra­tio­nal ev­i­dence, but not the other way around. We im­pose spe­cial, strong, ad­di­tional stan­dards be­fore we anoint ra­tio­nal ev­i­dence as “le­gal ev­i­dence”.

As I write this sen­tence at 8:33pm, Pa­cific time, on Au­gust 18th 2007, I am wear­ing white socks. As a ra­tio­nal­ist, are you li­censed to be­lieve the pre­vi­ous state­ment? Yes. Could I tes­tify to it in court? Yes. Is it a sci­en­tific state­ment? No, be­cause there is no ex­per­i­ment you can perform your­self to ver­ify it. Science is made up of gen­er­al­iza­tions which ap­ply to many par­tic­u­lar in­stances, so that you can run new real-world ex­per­i­ments which test the gen­er­al­iza­tion, and thereby ver­ify for your­self that the gen­er­al­iza­tion is true, with­out hav­ing to trust any­one’s au­thor­ity. Science is the pub­li­cly re­pro­ducible knowl­edge of hu­mankind.

Like a court sys­tem, sci­ence as a so­cial pro­cess is made up of fal­lible hu­mans. We want a pro­tected pool of be­liefs that are es­pe­cially re­li­able. And we want so­cial rules that en­courage the gen­er­a­tion of such knowl­edge. So we im­pose spe­cial, strong, ad­di­tional stan­dards be­fore we can­on­ize ra­tio­nal knowl­edge as “sci­en­tific knowl­edge”, adding it to the pro­tected be­lief pool.

Is a ra­tio­nal­ist li­censed to be­lieve in the his­tor­i­cal ex­is­tence of Alexan­der the Great? Yes. We have a rough pic­ture of an­cient Greece, un­trust­wor­thy but bet­ter than max­i­mum en­tropy. But we are de­pen­dent on au­thor­i­ties such as Plutarch; we can­not dis­card Plutarch and ver­ify ev­ery­thing for our­selves. His­tor­i­cal knowl­edge is not sci­en­tific knowl­edge.

Is a ra­tio­nal­ist li­censed to be­lieve that the Sun will rise on Septem­ber 18th, 2007? Yes—not with ab­solute cer­tainty, but that’s the way to bet. (Pedants: in­ter­pret this as the Earth’s ro­ta­tion and or­bit re­main­ing roughly con­stant rel­a­tive to the Sun.) Is this state­ment, as I write this es­say on Au­gust 18th 2007, a sci­en­tific be­lief?

It may seem per­verse to deny the ad­jec­tive “sci­en­tific” to state­ments like “The Sun will rise on Septem­ber 18th, 2007.” If Science could not make pre­dic­tions about fu­ture events—events which have not yet hap­pened—then it would be use­less; it could make no pre­dic­tion in ad­vance of ex­per­i­ment. The pre­dic­tion that the Sun will rise is, definitely, an ex­trap­o­la­tion from sci­en­tific gen­er­al­iza­tions. It is based upon mod­els of the So­lar Sys­tem which you could test for your­self by ex­per­i­ment.

But imag­ine that you’re con­struct­ing an ex­per­i­ment to ver­ify pre­dic­tion #27, in a new con­text, of an ac­cepted the­ory Q. You may not have any con­crete rea­son to sus­pect the be­lief is wrong; you just want to test it in a new con­text. It seems dan­ger­ous to say, be­fore run­ning the ex­per­i­ment, that there is a “sci­en­tific be­lief” about the re­sult. There is a “con­ven­tional pre­dic­tion” or “the­ory Q’s pre­dic­tion”. But if you already know the “sci­en­tific be­lief” about the re­sult, why bother to run the ex­per­i­ment?

You be­gin to see, I hope, why I iden­tify Science with gen­er­al­iza­tions, rather than the his­tory of any one ex­per­i­ment. A his­tor­i­cal event hap­pens once; gen­er­al­iza­tions ap­ply over many events. His­tory is not re­pro­ducible; sci­en­tific gen­er­al­iza­tions are.

Is my defi­ni­tion of “sci­en­tific knowl­edge” true? That is not a well-formed ques­tion. The spe­cial stan­dards we im­pose upon sci­ence are prag­matic choices. Nowhere upon the stars or the moun­tains is it writ­ten that p<0.05 shall be the stan­dard for sci­en­tific pub­li­ca­tion. Many now ar­gue that 0.05 is too weak, and that it would be use­ful to lower it to 0.01 or 0.001.

Per­haps fu­ture gen­er­a­tions, act­ing on the the­ory that sci­ence is the pub­lic, re­pro­ducible knowl­edge of hu­mankind, will only la­bel as “sci­en­tific” pa­pers pub­lished in an open-ac­cess jour­nal. If you charge for ac­cess to the knowl­edge, is it part of the knowl­edge of hu­mankind? Can we trust a re­sult if peo­ple must pay to crit­i­cize it? Is it re­ally sci­ence?

The ques­tion “Is it re­ally sci­ence?” is ill-formed. Is a $20,000/​year closed-ac­cess jour­nal re­ally Bayesian ev­i­dence? As with the po­lice com­mis­sioner’s pri­vate as­surance that Wulky is the king­pin, I think we must an­swer “Yes.” But should the closed-ac­cess jour­nal be fur­ther can­on­ized as “sci­ence”? Should we al­low it into the spe­cial, pro­tected be­lief pool? For my­self, I think sci­ence would be bet­ter served by the dic­tum that only open knowl­edge counts as the pub­lic, re­pro­ducible knowl­edge pool of hu­mankind.