How Common Are Science Failures?

After a brief spurt of de­bate over the claim that “97% of rele­vant pub­lished pa­pers sup­port an­thro­pogenic cli­mate change”, I think the pic­ture has mostly set­tled to an agree­ment that – al­though we can con­test the method­ol­ogy of that par­tic­u­lar study – there are mul­ti­ple lines of ev­i­dence that the num­ber is some­where in the nineties.

So if any doubt at all is to re­main about cli­mate change, it has to come from the worry that some­times en­tire sci­en­tific fields can get things near-unan­i­mously wrong, es­pe­cially for poli­ti­cal or con­for­mity-re­lated rea­sons.

In fact, I’d go so far as to say that if we are not cli­ma­tol­o­gists our­selves, our prior on cli­mate change should be based upon how fre­quently en­tire sci­en­tific fields get things ter­ribly wrong for poli­ti­cal or con­for­mity-re­lated rea­sons.

Skep­tics mock the claim that sci­ence was wrong be­fore, but skep­tics mock ev­ery­thing. A bet­ter plan might be to try to quan­tify the fre­quency of sci­en­tific failures so we can see how good (or bad) the chances are for any given field.

Be­fore we in­ves­ti­gate, we should define our refer­ence class prop­erly. I think a sci­en­tific mis­take only counts as a rea­son for doubt­ing cli­mate change (or any other com­monly-ac­cepted sci­en­tific paradigm) if:

1. It was made some­time in the re­cent past. Aris­to­tle was wrong about all sorts of things, and so were those doc­tors who thought ev­ery­thing had to do with black bile, but the sci­en­tific com­mu­nity back then was a lot less rigor­ous than our own. Let’s say it counts if it’s af­ter 1900.

2. It was part of a re­ally im­por­tant the­ory, one of the fun­da­men­tal paradigms of an en­tire field. I’m sure some tiny group of biol­o­gists have been wrong about how many chro­mo­somes a shrew has, but that’s prob­a­bly an eas­ier mis­take to wan­der into than all of cli­ma­tol­ogy screw­ing up si­mul­ta­neously.

3. It was a stub­born re­sis­tance to the truth, rather than just a failure to have come up with the cor­rect the­ory im­me­di­ately. Peo­ple were geo­cen­trists be­fore they were he­lio­cen­trists, but this wasn’t be­cause the field of as­tron­omy be­came overly poli­ti­cized and self-as­sured, it was be­cause (aside from one an­cient Greek guy no­body re­ally read) he­lio­cen­trism wasn’t in­vented un­til the 1500s, and af­ter that it took peo­ple a cou­ple of gen­er­a­tions to catch on. In the same way, New­ton’s the­ory of grav­ity wasn’t quite as good as Ein­stein’s, but this would not shame physi­cists in the same way cli­mate change be­ing wrong would shame cli­ma­tol­o­gists. Let’s say that in or­der to count, the cor­rect the­ory has to be very well known (the cor­rect the­ory is al­lowed to be “this phe­nomenon doesn’t ex­ist at all and you are wast­ing your time”) and there is a large group of peo­ple mostly out­side the main­stream sci­en­tific es­tab­lish­ment push­ing it (for ap­prox­i­mately cor­rect rea­sons) whom sci­en­tists just re­fuse to listen to.

4. We now know that the past sci­en­tific es­tab­lish­ment was definitely, definitely wrong and ev­ery­one agrees about this and it is not se­ri­ously in doubt. This crite­rion isn’t to be fair to the cli­ma­tol­o­gists, this is to be fair to me when I have to read the com­ments to this post and get a bunch of “Nutri­tion­ists have yet to sign on to my pet the­ory of diet, that proves some sci­en­tific fields are hope­lessly cor­rupt!”

Do any such sci­en­tific failures ex­ist?

If we want to play this game on Easy Mode, our first tar­get will be Ly­senko­ism, the com­pletely bonkers the­ory of agri­cul­ture and ge­net­ics adopted by the Soviet Union. A low-level agri­cul­tural biol­o­gist, Ly­senko, came up with ques­tion­able ways of in­creas­ing agri­cul­tural out­put through some­thing kind of like La­mar­ck­ian evolu­tion. The Soviet gov­ern­ment wanted to in­spire peo­ple in the mid­dle of a famine, didn’t re­ally like real sci­en­tists be­cause they seemed kind of bour­geois, and wanted to dis­credit ge­net­ics be­cause her­i­ta­bil­ity seemed con­trary to the idea of New Soviet Man. So they pro­moted Ly­senko enough times that ev­ery­one got the mes­sage that Ly­senko­ism was the road to get­ting good po­si­tions. All the ca­reerists switched over to the new paradigm, and the hold­outs who con­tinued to be­lieve in ge­net­ics were de­nounced as fas­cists. Ac­cord­ing to Wikipe­dia, “in 1948, ge­net­ics was offi­cially de­clared “a bour­geois pseu­do­science”; all ge­net­i­cists were fired from their jobs (some were also ar­rested), and all ge­netic re­search was dis­con­tinued.”

About twenty years later the Soviets quietly came to their senses and cov­ered up the whole thing.

I would ar­gue that Stal­inist Rus­sia, where the gov­ern­ment was very clearly in­ter­ven­ing in sci­ence and kil­ling the peo­ple it didn’t like, isn’t a fair test case for a the­ory to­day. But cli­mate change op­po­nents would prob­a­bly re­spond that the liberal world or­der is un­fairly pro­mot­ing sci­en­tists who sup­port cli­mate change and per­se­cut­ing those who op­pose it. And Ly­senko­ism at least proves that is the sort of thing which can in the­ory some­times hap­pen. So let’s grum­ble a lit­tle but give it to them.

Now we turn the dial up to Hard Mode. Are there any cases of failure on a similar level within a sci­en­tific com­mu­nity in a coun­try not ac­tively be­ing ruled by Stalin?

I can think of two: Freudian psy­cho­anal­y­sis and be­hav­iorist psy­chol­ogy.

Freudian psy­cho­anal­y­sis needs no in­tro­duc­tion. It dom­i­nated psy­chi­a­try – not at all a small field – from about 1930 to 1980. As far as any­one can tell, the en­tire gi­gan­tic ed­ifice has no re­deem­ing qual­ities. I mean, it cor­rectly de­scribes the ex­is­tence of a sub­con­scious, and it may have some in­sight­ful things to say on child­hood trauma, but as far as a de­cent model of the brain or of psy­cholog­i­cal treat­ment goes, it was a gi­ant mis­take.

I got a lit­tle bet­ter idea just how big a mis­take do­ing some re­search for the Anti-Re­ac­tionary FAQ. I wanted to see how ho­mo­sex­u­als were viewed back in the 1950s and ran across two New York Times ar­ti­cles about them (1, 2). It’s re­ally creepy to see them ex­plain­ing how in­stead of hold­ing on to folk be­liefs about how ho­mo­sex­u­als are nor­mal peo­ple just like you or me, peo­ple need to start listen­ing to the psy­cho­an­a­lytic ex­perts, who know the real story be­hind why some peo­ple are ho­mo­sex­ual. The in­ter­views with the ex­perts in the ar­ti­cle are a lit­tle sur­real.

Psy­cho­anal­y­sis wasn’t an hon­est mis­take. The field already had a perfectly good al­ter­na­tive – de­nounc­ing the whole thing as bunk – and sen­si­ble non-psy­cho­an­a­lysts seemed to do ex­actly that. On the other hand, the more you got “ed­u­cated” about psy­chi­a­try in psy­cho­an­a­lytic in­sti­tu­tions, and the more you wanted to be­come a psy­chi­a­trist your­self, the more you got bi­ased into think psy­cho­anal­y­sis was ob­vi­ously cor­rect and dis­miss­ing the doubters as sci­ence de­nal­ists or what­ever it was they said back then.

So this seems like a gen­uine ex­am­ple of a sci­en­tific field failing.

Be­hav­iorism in psy­chol­ogy was…well, this part will be con­tro­ver­sial. A weak ver­sion is “psy­chol­o­gists should not study thoughts or emo­tions be­cause these are un­know­able by sci­en­tific meth­ods; in­stead they should limit them­selves to be­hav­iors”. A strong ver­sion is “thoughts and emo­tions don’t ex­ist; they are post hoc ex­pla­na­tions in­vented by peo­ple to ra­tio­nal­ize their be­hav­iors”. Peo­ple are go­ing to tell me that real psy­chol­o­gists only be­lieved the weak ver­sion, but hav­ing read more than a lit­tle 1950s psy­chol­ogy, I’m go­ing to tell them they’re wrong. I think a lot of peo­ple be­lieved the strong ver­sion and that in fact it was the dom­i­nant paradigm in the field.

And of course com­mon peo­ple said this was stupid, of course we have thoughts and emo­tions, and the ex­perts just said that kind of drivel was ex­actly what com­mon peo­ple would think. Then came the cog­ni­tive rev­olu­tion and peo­ple re­al­ized thoughts and emo­tions were ac­tu­ally kind of easy to study. And then we got MRI ma­chines and are now a good chunk of the way to see­ing them.

So this too I will count as a sci­en­tific failure.

But – and this seems im­por­tant – I can’t think of any oth­ers.

Sup­pose there are about fifty sci­en­tific fields ap­prox­i­mately as im­por­tant as ge­net­ics or psy­chi­a­try or psy­chol­ogy. And sup­pose within the past cen­tury, each of them had room for about five paradigms as im­por­tant as psy­cho­anal­y­sis or be­hav­iorism or Ly­senko­ism.

That would mean there are about 250 pos­si­bil­ities for sci­ence failure, of which three were ac­tu­ally sci­ence failures – for a failure rate of 1.2%.

This doesn’t seem much more en­courag­ing for the anti-global-warm­ing cause than the 3% of pa­pers that sup­port them.

I think I’m be­ing pretty fair here – af­ter all, Ly­senko­ism was limited to one ex­tremely-screwed-up coun­try, and peo­ple are go­ing to yell that be­hav­iorism wasn’t as bad as I made it sound. And two of the three failures are in psy­chol­ogy, a so­cial sci­ence much fuzzier than cli­ma­tol­ogy where we can ex­pect far more er­rors. A cynic might say if we in­clude psy­chol­ogy we might as well go all the way and in­clude eco­nomics, so­ciol­ogy, and an­thro­pol­ogy, rais­ing our er­ror count to over nine thou­sand.

But if we want to be even fairer, we can ad­mit that there are prob­a­bly some sci­ence failures that haven’t been de­tected yet. I can think of three that I very strongly sus­pect are in that cat­e­gory, al­though I won’t tell you what they are so as to not dis­tract from the meta-level de­bate. That brings us to 2.4%. Ad­mit that maybe I’ve only caught half of the im­pend­ing sci­ence failures out there, and we get to 3.6%. Still not much of an im­prove­ment for the anti-AGW crowd over hav­ing 3% of the liter­a­ture.

Un­less of course I am miss­ing a whole load of well-known sci­ence failures which you will re­mind me about in the com­ments.

[Edit: Wow, peo­ple are re­ally bad at fol­low­ing crite­ria 3 and 4, even go­ing so far as to post the ex­act ex­am­ples I said not to. Don’t let that be you.]