Building Intuitions On Non-Empirical Arguments In Science

Link post


Aeon: Post-Em­piri­cal Science Is An Oxy­moron And It is Danger­ous:

There is no agreed crite­rion to dis­t­in­guish sci­ence from pseu­do­science, or just plain or­di­nary bul­lshit, open­ing the door to all man­ner of meta­physics mas­querad­ing as sci­ence. This is ‘post-em­piri­cal’ sci­ence, where truth no longer mat­ters, and it is po­ten­tially very dan­ger­ous.

It’s not difficult to find re­cent ex­am­ples. On 8 June 2019, the front cover of New Scien­tist mag­a­z­ine boldly de­clared that we’re ‘In­side the Mir­ror­verse’. Its ed­i­tors bid us ‘Wel­come to the par­allel re­al­ity that’s hid­ing in plain sight’. […]

[Some physi­cists] claim that neu­trons [are] flit­ting be­tween par­allel uni­verses. They ad­mit that the chances of prov­ing this are ‘low’, or even ‘zero’, but it doesn’t re­ally mat­ter. When it comes to grab­bing at­ten­tion, invit­ing that all-im­por­tant click, or pur­chase, spec­u­la­tive meta­physics wins hands down.

Th­ese the­o­ries are based on the no­tion that our Uni­verse is not unique, that there ex­ists a large num­ber of other uni­verses that some­how sit alongside or par­allel to our own. For ex­am­ple, in the so-called Many-Wor­lds in­ter­pre­ta­tion of quan­tum me­chan­ics, there are uni­verses con­tain­ing our par­allel selves, iden­ti­cal to us but for their differ­ent ex­pe­riences of quan­tum physics. Th­ese the­o­ries are at­trac­tive to some few the­o­ret­i­cal physi­cists and philoso­phers, but there is ab­solutely no em­piri­cal ev­i­dence for them. And, as it seems we can’t ever ex­pe­rience these other uni­verses, there will never be any ev­i­dence for them. As Brous­sard ex­plained, these the­o­ries are suffi­ciently slip­pery to duck any kind of challenge that ex­per­i­men­tal­ists might try to throw at them, and there’s always some­one happy to keep the idea al­ive.

Is this re­ally sci­ence? The an­swer de­pends on what you think so­ciety needs from sci­ence. In our post-truth age of ca­sual lies, fake news and al­ter­na­tive facts, so­ciety is un­der ex­traor­di­nary pres­sure from those push­ing po­ten­tially dan­ger­ous an­ti­scien­tific pro­pa­ganda – rang­ing from cli­mate-change de­nial to the anti-vaxxer move­ment to home­o­pathic medicines. I, for one, pre­fer a sci­ence that is ra­tio­nal and based on ev­i­dence, a sci­ence that is con­cerned with the­o­ries and em­piri­cal facts, a sci­ence that pro­motes the search for truth, no mat­ter how tran­sient or con­tin­gent. I pre­fer a sci­ence that does not read­ily ad­mit the­o­ries so vague and slip­pery that em­piri­cal tests are ei­ther im­pos­si­ble or they mean ab­solutely noth­ing at all.

As always, a sin­gle quote doesn’t do the ar­gu­ment jus­tice, so go read the ar­ti­cle. But I think this cap­tures the ba­sic ar­gu­ment: mul­ti­verse the­o­ries are bad, be­cause they’re untestable, and untestable sci­ence is pseu­do­science.

Many great peo­ple, both philoso­phers of sci­ence and prac­tic­ing sci­en­tists, have already dis­cussed the prob­lems with this point of view. But none of them lay out their ar­gu­ment in quite the way that makes the most sense to me. I want to do that here, with­out claiming any origi­nal­ity or spe­cial ex­per­tise in the sub­ject, to see if it helps con­vince any­one else.


Con­sider a clas­sic ex­am­ple: mod­ern pa­le­on­tol­ogy does a good job at pre­dict­ing dinosaur fos­sils. But the cre­ation­ist ex­pla­na­tion – Satan buried fake dinosaur fos­sils to mis­lead us – also pre­dicts the same fos­sils (we as­sume Satan is good at dis­guis­ing his ex­is­tence, so that the lack of other strong ev­i­dence for Satan doesn’t con­tra­dict the the­ory). What prin­ci­ples help us re­al­ize that the Satan hy­poth­e­sis is ob­vi­ously stupid and the usual pa­le­on­tolog­i­cal one more plau­si­ble?

One bad re­sponse: pa­le­on­tol­ogy can bet­ter pre­dict char­ac­ter­is­tics of dinosaur fos­sils, us­ing ar­gu­ments like “since ple­siosaurs are aquatic, they will be found in ar­eas that were un­der­wa­ter dur­ing the Me­so­zoic, but since tyran­nosaurs are ter­res­trial, they will be found in ar­eas that were on land”, and this makes it bet­ter than the Satan hy­poth­e­sis, which can only retro­d­ict these char­ac­ter­is­tics. But this isn’t quite true: since Satan is try­ing to fool us into be­liev­ing the mod­ern pa­le­on­tol­ogy paradigm, he’ll hide the fos­sils in ways that con­form to its pre­dic­tions, so we will pre­dict ple­siosaur fos­sils will only be found at sea – oth­er­wise the gig would be up!

A sec­ond bad re­sponse: “The hy­poth­e­sis that all our find­ings were planted to de­ceive us bleeds into con­spir­acy the­o­ries and touches on the prob­lem of skep­ti­cism. Th­ese things are in­her­ently out­side the realm of sci­ence.” But ar­chae­olog­i­cal find­ings are very of­ten de­liber­ate hoaxes planted to de­ceive ar­chae­ol­o­gists, and in prac­tice ar­chae­ol­o­gists con­sider and test that hy­poth­e­sis the same way they con­sider and test ev­ery other hy­poth­e­sis. Rule this out by fiat and we have to ac­cept Pilt­down Man, or at least claim that the peo­ple ar­gu­ing against the ve­rac­ity of Pilt­down Man were do­ing some­thing other than Science.

A third bad re­sponse: “Satan is su­per­nat­u­ral and sci­ence is not al­lowed to con­sider su­per­nat­u­ral ex­pla­na­tions.” Fine then, re­place Satan with an alien. I think this is a stupid dis­tinc­tion – if demons re­ally did in­terfere in earthly af­fairs, then we could in­ves­ti­gate their ac­tions us­ing the same meth­ods we use to in­ves­ti­gate ev­ery other pro­cess. But this would take a long time to ar­gue well, so for now let’s just stick with the alien.

A fourth bad re­sponse: “There is no em­piri­cal test that dis­t­in­guishes the Satan hy­poth­e­sis from the pa­le­on­tol­ogy hy­poth­e­sis, there­fore the Satan hy­poth­e­sis is in­her­ently un­falsifi­able and there­fore pseu­do­scien­tific.” But this can’t be right. After all, there’s no em­piri­cal test that dis­t­in­guishes the pa­le­on­tol­ogy hy­poth­e­sis from the Satan hy­poth­e­sis! If we call one of them pseu­do­science based on their in­sep­a­ra­bil­ity, we have to call the other one pseu­do­science too!

A naive Pop­pe­rian (which maybe no­body re­ally is) would have to stop here, and say that we pre­dict dinosaur fos­sils will have such-and-such char­ac­ter­is­tics, but that ques­tions like that pro­cess that drives this pat­tern – a long-dead ecosys­tem of ac­tual dinosaurs, or the Devil plant­ing dinosaur bones to de­ceive us – is a mys­ti­cal ques­tion be­yond the abil­ity of Science to even con­ceiv­ably solve.

I think the cor­rect re­sponse is to say that both the­o­ries ex­plain the data, and one can­not em­piri­cally test which the­ory is true, but the pa­le­on­tol­ogy the­ory is more el­e­gant (I am tempted to say “sim­pler”, but that might im­ply I have a rigor­ous math­e­mat­i­cal defi­ni­tion of the form of sim­plic­ity in­volved, which I don’t). It re­quires fewer other weird things to be true. It in­volves fewer other hid­den vari­ables. It trans­forms our wor­ld­view less. It gets a cleaner shave with Oc­cam’s Ra­zor. This el­e­gance is so im­por­tant to us that it ex­plains our vast prefer­ence for the first the­ory over the sec­ond.

A long tra­di­tion of philoso­phers of sci­ence have already writ­ten elo­quently about this, summed up by Sean Car­roll here:

What makes an ex­pla­na­tion “the best.” Thomas Kuhn ,af­ter his in­fluen­tial book The Struc­ture of Scien­tific Revolu­tions led many peo­ple to think of him as a rel­a­tivist when it came to sci­en­tific claims, at­tempted to cor­rect this mis­im­pres­sion by offer­ing a list of crite­ria that sci­en­tists use in prac­tice to judge one the­ory bet­ter than an­other one: ac­cu­racy, con­sis­tency, broad scope, sim­plic­ity, and fruit­ful­ness. “Ac­cu­racy” (fit­ting the data) is one of these crite­ria, but by no means the sole one. Any work­ing sci­en­tist can think of cases where each of these con­cepts has been in­voked in fa­vor of one the­ory or an­other. But there is no un­am­bigu­ous al­gorithm ac­cord­ing to which we can feed in these crite­ria, a list of the­o­ries, and a set of data, and ex­pect the best the­ory to pop out. The way in which we judge sci­en­tific the­o­ries is in­escapably re­flec­tive, messy, and hu­man. That’s the re­al­ity of how sci­ence is ac­tu­ally done; it’s a mat­ter of judg­ment, not of draw­ing bright lines be­tween truth and falsity or sci­ence and non-sci­ence. For­tu­nately, in typ­i­cal cases the ac­cu­mu­la­tion of ev­i­dence even­tu­ally leaves only one vi­able the­ory in the eyes of most rea­son­able ob­servers.

The dinosaur hy­poth­e­sis and the Satan hy­poth­e­sis both fit the data, but the dinosaur hy­poth­e­sis wins hands-down on sim­plic­ity. As Car­roll pre­dicts, most rea­son­able ob­servers are able to con­verge on the same solu­tion here, de­spite the philo­soph­i­cal com­plex­ity.


I’m start­ing with this ex­treme case be­cause its very ex­trem­ity makes it eas­ier to see the mechanism in ac­tion. But I think the same pro­cess ap­plies to other cases that peo­ple re­ally worry about.

Con­sider the rid­dle of the Sphinx. There’s pretty good ar­chae­olog­i­cal ev­i­dence sup­port­ing the con­sen­sus po­si­tion that it was built by Pharaoh Khafre. But there are a few holes in that story, and a few scat­tered ar­ti­facts sug­gest it was ac­tu­ally built by Pharaoh Khufu; a re­spectable minor­ity of ar­chae­ol­o­gists be­lieve this. And there are a few anoma­lies which, if taken wildly out of con­text, you can use to tell a story that it was built long be­fore Egypt ex­isted at all, maybe by At­lantis or aliens.

So there are three com­pet­ing hy­pothe­ses. All of them are con­sis­tent with cur­rent ev­i­dence (even the At­lantis one, which was writ­ten af­ter the cur­rent ev­i­dence was found and care­fully adds enough epicy­cles not to blatantly con­tra­dict it). Per­haps one day ev­i­dence will come to light that sup­ports one above the oth­ers; maybe in some un­ex­ca­vated tomb, a hi­ero­glyphic tablet says “I cre­ated the Sphinx, sincerely yours, Pharaoh Khufu”. But maybe this won’t hap­pen. Maybe we already have all the Sphinx-re­lated ev­i­dence we’re go­ing to get. Maybe the in­for­ma­tion nec­es­sary to dis­t­in­guish among these hy­pothe­ses has been ut­terly lost be­yond any con­ceiv­able abil­ity to re­con­struct.

I don’t want to say “No hy­poth­e­sis can be tested any fur­ther, so Science is use­less to us here”, be­cause then we’re forced to con­clude stupid things like “Science has no opinion on whether the Sphinx was built by Khafre or At­lanteans,” whereas I think most sci­en­tists would ac­tu­ally have very strong opinions on that.

But what about the ques­tion of whether the Sphinx was built by Khafre or Khufu? This is a real open ques­tion with re­spectable ar­chae­ol­o­gists on both sides; what can we do about it?

I think the an­swer would have to be: the same thing we did with the Satan vs. pa­le­on­tol­ogy ques­tion, only now it’s a lot harder. We try to figure out which the­ory re­quires fewer other weird things to be true, fewer hid­den vari­ables, less trans­for­ma­tion of our wor­ld­view – which the­ory works bet­ter with Oc­cam’s Ra­zor. This is rel­a­tively easy in the At­lantis case, and hard but po­ten­tially pos­si­ble in the Khafre vs. Khufu case.

(Bayesi­ans can rephrase this to: given that we have a cer­tain amount of ev­i­dence for each, can we quan­tify ex­actly how much ev­i­dence, and what our pri­ors for each should be. It would end not with a de­ci­sive vic­tory of one or the other, but with a prob­a­bil­ity dis­tri­bu­tion, maybe 80% chance it was Khafre, 20% chance it was Khufu)

I think this is a to­tally le­gi­t­i­mate thing for Egyp­tol­o­gists to do, even if it never re­sults in a par­tic­u­lar testable claim that gets tested. If you don’t think it’s a le­gi­t­i­mate thing for Egyp­tol­o­gists to do, I have trou­ble figur­ing out how you can jus­tify Egyp­tol­o­gists re­ject­ing the At­lantis the­ory.

(Again, Bayesi­ans would start with a very low prior for At­lantis, and as­sess the ev­i­dence as very low, and end up with a prob­a­bil­ity dis­tri­bu­tion some­thing like Khafre 80%, Khufu 19.999999%, At­lantis 0.000001%)


How does this re­late to things like mul­ti­verse the­ory? Be­fore we get there, one more hokey ex­am­ple:

Sup­pose sci­en­tists mea­sure the mass of one par­ti­cle at 32.604 units, the mass of an­other re­lated par­ti­cle at 204.897 units, and the mass of a third re­lated par­ti­cle at 4452.767 units. For a while, this is just how things are – it seems to be an ir­re­ducible brute fact about the uni­verse. Then some the­o­rist no­tices that if you set the mass of the first par­ti­cle as x, then the sec­ond is 2πx and the third is 43 πx^2. They the­o­rize that per­haps the quan­tum field forms some sort of ex­tradi­men­sional sphere, the first par­ti­cle rep­re­sents a di­ame­ter of a great cir­cle of the sphere, the sec­ond the cir­cum­fer­ence of the great cir­cle, and the third the vol­ume of the sphere.

(please ex­cuse the stu­pidity of my ex­am­ple, I don’t know enough about physics to come up with some­thing that isn’t stupid, but I hope it will illus­trate my point)

In fact, imag­ine that there are a hun­dred differ­ent par­ti­cles, all with differ­ent masses, and all one hun­dred have masses that perfectly cor­re­spond to var­i­ous math­e­mat­i­cal prop­er­ties of spheres.

Is the per­son who made this dis­cov­ery do­ing Science? And should we con­sider their the­ory a use­ful con­tri­bu­tion to physics?

I think the an­swer is clearly yes. But con­sider what this com­mits us to. Sup­pose the sci­en­tist came up with their Ex­tradi­men­sional Sphere hy­poth­e­sis af­ter learn­ing the masses of the rele­vant par­ti­cles, and so it has not pre­dicted any­thing. Sup­pose the ex­tradi­men­sional sphere is out­side nor­mal space, curled up into some di­men­sion we can’t pos­si­bly ac­cess or test with­out a par­ti­cle ac­cel­er­a­tor the size of the moon. Sup­pose there are no undis­cov­ered par­ti­cles in this set that can be tested to see if they also re­flect sphere-re­lated pa­ram­e­ters. This the­ory is ex­actly the kind of postem­piri­cal, meta­phys­i­cal con­struct that the Aeon ar­ti­cle sav­ages.

But it’s re­ally com­pel­ling. We have a hun­dred differ­ent par­ti­cles, and this the­ory retro­d­icts the prop­er­ties of each of them perfectly. And it’s so sim­ple – just say the word “sphere” and the rest falls out nat­u­rally! You would have to be crazy not to think it was at least pretty plau­si­ble, or that the sci­en­tist who de­vel­oped it had done some good work.

Nor do I think it seems right to say “The dis­cov­ery that all of our un­ex­plained vari­ables perfectly match the pa­ram­e­ters of a sphere is good, but the hy­poth­e­sis that there re­ally is a sphere is out­side the bounds of Science.” That sounds too much like say­ing “It’s fine to say dinosaur bones have such-and-such char­ac­ter­is­tics, but we must never spec­u­late about what kind of pro­cess pro­duced them, or whether it in­volved ac­tual dinosaurs”.


My un­der­stand­ing of the mul­ti­verse de­bate is that it works the same way. Scien­tists ob­serve the be­hav­ior of par­ti­cles, and find that a mul­ti­verse ex­plains that be­hav­ior more sim­ply and el­e­gantly than not-a-mul­ti­verse.

One (doubtless ex­ag­ger­ated) way I’ve heard mul­ti­verse pro­po­nents ex­plain their po­si­tion is like this: in cer­tain situ­a­tions the math de­clares two con­tra­dic­tory an­swers – in the clas­sic ex­am­ple, Schrod­inger’s cat will be both al­ive and dead. But when we open the box, we see only a dead cat or an al­ive cat, not both. Mul­ti­verse op­po­nents say “Some un­known force steps in at the last sec­ond and de­stroys one of the pos­si­bil­ity branches”. Mul­ti­verse pro­po­nents say “No it doesn’t, both pos­si­bil­ity branches hap­pen ex­actly the way the math says, and we end up in one of them.”

Tak­ing this ex­ag­ger­ated dumbed-down ac­count as ex­actly right, this sounds about as hard as the dinosaurs-vs-Satan ex­am­ple, in terms of figur­ing out which is more Oc­cam’s Ra­zor com­pli­ant. I’m sure the re­al­ity is more nu­anced, but I think it can be judged by the same pro­cess. Per­haps this is the kind of rea­son­ing that only gets us to a 90% prob­a­bil­ity there is a mul­ti­verse, rather than a 99.999999% one. But I think de­ter­min­ing that the­o­ries have 90% prob­a­bil­ity is a rea­son­able sci­en­tific thing to do.


At times, the Aeon ar­ti­cle seems to flirt with ad­mit­ting that some­thing like this is nec­es­sary:

Such prob­lems were judged by philoso­phers of sci­ence to be in­sur­mountable, and Pop­per’s falsifi­a­bil­ity crite­rion was aban­doned (though, cu­ri­ously, it still lives on in the minds of many prac­tis­ing sci­en­tists). But rather than seek an al­ter­na­tive, in 1983 the philoso­pher Larry Lau­dan de­clared that the de­mar­ca­tion prob­lem is ac­tu­ally in­tractable, and must there­fore be a pseudo-prob­lem. He ar­gued that the real dis­tinc­tion is be­tween knowl­edge that is re­li­able or un­re­li­able, ir­re­spec­tive of its prove­nance, and claimed that terms such as ‘pseu­do­science’ and ‘un­scien­tific’ have no real mean­ing.

But it always jumps back from the precipice:

So, if we can’t make use of falsifi­a­bil­ity, what do we use in­stead? I don’t think we have any real al­ter­na­tive but to adopt what I might call the em­piri­cal crite­rion. De­mar­ca­tion is not some kind of bi­nary yes-or-no, right-or-wrong, black-or-white judg­ment. We have to ad­mit shades of grey. Pop­per him­self was ready to ac­cept this, [say­ing]:

“The crite­rion of de­mar­ca­tion can­not be an ab­solutely sharp one but will it­self have de­grees. There will be well-testable the­o­ries, hardly testable the­o­ries, and non-testable the­o­ries. Those which are non-testable are of no in­ter­est to em­piri­cal sci­en­tists. They may be de­scribed as meta­phys­i­cal.”

Here, ‘testa­bil­ity’ im­plies only that a the­ory ei­ther makes con­tact, or holds some promise of mak­ing con­tact, with em­piri­cal ev­i­dence. It makes no pre­sump­tions about what we might do in light of the ev­i­dence. If the ev­i­dence ver­ifies the the­ory, that’s great – we cel­e­brate and start look­ing for an­other test. If the ev­i­dence fails to sup­port the the­ory, then we might pon­der for a while or tin­ker with the aux­iliary as­sump­tions. Either way, there’s a ten­sion be­tween the meta­phys­i­cal con­tent of the the­ory and the em­piri­cal data – a ten­sion be­tween the ideas and the facts – which pre­vents the meta­physics from get­ting com­pletely out of hand. In this way, the meta­physics is tamed or ‘nat­u­ral­ised’, and we have some­thing to work with. This is sci­ence.

But as we’ve seen, many things we re­ally want to in­clude as sci­ence are not testable: our cre­dence for real dinosaurs over Satan plant­ing fos­sils, our cre­dence for Khafre build­ing the Sphinx over Khufu or At­lanteans, or el­e­gant pat­terns that ex­plain the fea­tures of the uni­verse like the Ex­tradi­men­sional-Sphere The­ory.

The Aeon ar­ti­cle is aware of Car­roll’s work – which, along with the para­graph quoted in Sec­tion II above, in­cludes a lot of de­tailed Bayesian rea­son­ing en­com­pass­ing ev­ery­thing I’ve dis­cussed. But the ar­ti­cle dis­misses it in a few sen­tences:

Sean Car­roll, a vo­cal ad­vo­cate for the Many-Wor­lds in­ter­pre­ta­tion, prefers ab­duc­tion, or what he calls ‘in­fer­ence to the best ex­pla­na­tion’, which leaves us with the­o­ries that are merely ‘par­si­mo­nious’, a mat­ter of judg­ment, and ‘still might rea­son­ably be true’. But whose judg­ment? In the ab­sence of facts, what con­sti­tutes ‘the best ex­pla­na­tion’?

Car­roll seeks to dress his no­tion of in­fer­ence in the cloth of re­spectabil­ity pro­vided by some­thing called Bayesian prob­a­bil­ity the­ory, hap­pily over­look­ing its en­tirely sub­jec­tive na­ture. It’s a short step from here to the the­o­rist-turned-philoso­pher Richard Dawid’s efforts to jus­tify the string the­ory pro­gramme in terms of ‘the­o­ret­i­cally con­firmed the­ory’ and ‘non-em­piri­cal the­ory as­sess­ment’. The ‘best ex­pla­na­tion’ is then based on a choice be­tween purely meta­phys­i­cal con­structs, with­out refer­ence to em­piri­cal ev­i­dence, based on the ap­pli­ca­tion of a prob­a­bil­ity the­ory that can be read­ily en­g­ineered to suit per­sonal prej­u­dices.

“A choice be­tween purely meta­phys­i­cal con­structs, with­out refer­ence to em­piri­cal ev­i­dence” sounds pretty bad, un­til you re­al­ize he’s talk­ing about the same rea­son­ing we use to de­ter­mine that real dinosaurs are more likely than Satan plant­ing fos­sils.

I don’t want to go over the ex­act ways in which Bayesian meth­ods are sub­jec­tive (which I think are over­es­ti­mated) vs. ob­jec­tive. I think it’s more fruit­ful to point out that your brain is already us­ing Bayesian meth­ods to in­ter­pret the pho­tons strik­ing your eyes into this sen­tence, to make snap de­ci­sions about what sense the words are used in, and to in­te­grate them into your model of the world. If Bayesian meth­ods are good enough to give you ev­ery sin­gle piece of ev­i­dence about the na­ture of the ex­ter­nal world that you have ever en­coun­tered in your en­tire life, I say they’re good enough for sci­ence.

Or if you don’t like that, you can use the ex­pla­na­tion above, which barely uses the word “Bayes” at all and just de­scribes ev­ery­thing in terms like “Oc­cam’s Ra­zor” and “you wouldn’t want to con­clude some­thing like that, would you?”

I know there are sep­a­rate de­bates about whether this kind of rea­son­ing-from-sim­plic­ity is ac­tu­ally good enough, when used by or­di­nary peo­ple, to con­sis­tently ar­rive at truth. Or whether it’s a pro­duc­tive way to con­duct sci­ence that will give us good new the­o­ries, or a waste of ev­ery­body’s time. I sym­pa­thize with some these con­cerns, though I am nowhere near sci­en­tifi­cally ed­u­cated enough to have an ac­tual opinion on the ques­tions at play.

But I think it’s im­por­tant to ar­gue that even be­fore you de­scribe the ad­van­tages and dis­ad­van­tages of the com­pli­cated Bayesian math that lets you do this, some­thing like this has to be done. The untestable is a fun­da­men­tal part of sci­ence, im­pos­si­ble to re­move. We can de­bate how to ex­plain it. But deny­ing it isn’t an op­tion.