Learning To Love Scientific Consensus

[Re­lated to: Con­trar­i­ans, Crack­pots, and Con­sen­sus, How Com­mon Are Science Failures?. Epistemic sta­tus is “sub­tle and likely to be mis­in­ter­preted”.]

I.

There’s a list of sci­en­tific mav­er­icks who were ridiculed by hide­bound re­ac­tionar­ies but later vin­di­cated that’s been go­ing viral. I ex­am­ined the first ten mav­er­icks on the list to see if its claims held up. Over­all I wasn’t too im­pressed. Let me go over them in more de­tail.

SVANTE ARRHENIUS:

His idea that elec­trolytes are full of charged atoms was con­sid­ered crazy. The atomic the­ory was new at the time, and ev­ery­one “knew” that atoms were in­di­visi­ble (and hence they could not lose or gain any elec­tric charge.) Be­cause of his hereti­cal idea, he only re­ceived his uni­ver­sity de­gree by a very nar­row mar­gin.

Sure, the pro­fes­sors who were judg­ing his PhD the­sis weren’t too con­vinced. So Ar­rhe­nius sent his pro­posal to the world’s top chemists at the time, and they were su­per-in­ter­ested and started fight­ing among them­selves to work with Ar­rhe­nius on it. Top chemist Wilhelm Ost­wald re­ceived the pa­per the same day his daugh­ter was born, and sug­gested that the pa­per was the more ex­cit­ing of the two events. He jour­neyed to Ar­rhe­nius’ home­town of Upp­sala, Swe­den to try to con­vince Ar­rhe­nius to work with him; Ar­rhe­nius re­fused for per­sonal rea­sons but later got a schol­ar­ship and worked with the top physi­cists in Europe. Ar­rhe­nius be­came a pro­fes­sor in a pres­ti­gious uni­ver­sity about ten years af­ter pre­sent­ing his “ridiculed” pa­per, and won the No­bel Prize ten years af­ter that.

HANS ALFVEN:

Astronomers thought that grav­ity alone is im­por­tant in so­lar sys­tems, in galax­ies, etc. Alfven’s idea that plasma physics is of equal or greater im­por­tance to grav­ity was de­rided for decades.

This isn’t a great de­scrip­tion of Alfven’s con­flict with the es­tab­lish­ment, but the list seems ba­si­cally right in­so­far as Alfven’s ideas were ig­nored for thirty years be­fore be­ing proven mostly cor­rect. I will give them this one.

JOHN BAIRD:

When the first tele­vi­sion sys­tem was demon­strated to the Royal So­ciety (Bri­tish sci­en­tists,) they scoffed and ridiculed, call­ing Baird a swindler.

I can’t find any refer­ence to this in var­i­ous Baird ar­ti­cles and bi­ogra­phies. The clos­est I can come is this ar­ti­cle by some­one who was there at the demon­stra­tion, who said “They didn’t be­lieve it…the pic­tures were a bit of a blur but it was amaz­ing, they were all ab­solutely flab­ber­gasted by it.” It looks like he is us­ing “they didn’t be­lieve it” in the col­lo­quial way of “they thought it was amaz­ing”. A TIME mag­a­z­ine ar­ti­cle from the time de­scribed the same sci­en­tists as “deeply im­pressed”, though the word­ing is kind of un­clear and they might have been refer­ring to a differ­ent demon­stra­tion a year later.

In any case, it seems very clear that within a year ev­ery­one agreed he was le­gi­t­i­mate and over­came their ini­tial shock.

ROBERT BAKKER:

Every­one knows that dinosaurs are like Gila mon­sters or big tor­toises: large, slow, and in­tol­er­ant of the cold. And they’re all col­ored olive drab too! 🙂

Bakker did help pro­duce the paradigm shift in pa­le­on­tol­ogy from cold-blooded dinosaurs to warm-blooded dinosaurs. But he was not a lone mav­er­ick be­ing ridiculed by ev­ery­one else. He learned that dinosaurs were warm-blooded from his pro­fes­sor at Yale, who was also part of the minor­ity-but-to­tally-ex­ist­ing fac­tion that be­lieved dinosaurs were warm-blooded. He him­self got a PhD at Har­vard from pro­fes­sors who were ap­par­ently sym­pa­thetic to the same the­ory. And within seven years of his first pa­per be­ing pub­lished, Scien­tific Amer­i­can was call­ing his ideas “the dinosaur re­nais­sance”, which doesn’t leave a lot of time for him to be ridiculed and ig­nored in.

BARDEEN & BRATTAIN:

Not ridiculed, but their boss W. Shock­ley nixed their idea for a non-FET “crys­tal tri­ode” de­vice. When they started in­ves­ti­gat­ing it, he made them stop. They were sup­posed to be work­ing on FETs in­stead.

ARG, I GOT THIS WRONG, THIS PART BELOW IS A BELL LABS STORY REGARDING ZONE REFINING OF SILICON, NOT THE BJT TRANSISTOR PROJECT: So, they as­sem­bled their ZONE REFINING ex­per­i­ment on a wheeled cart and con­tinued. When­ever the boss was sched­uled to check up on them, they could shove it into an ad­ja­cent un­used lab.

Okay, it looks like the guy com­piling the list ad­mits he was wrong on this one. Mov­ing on…

BRETZ:

En­dured decades of scorn as the laugh­ing­stock of the ge­ol­ogy world. His crime was to in­sist that enor­mous amounts of ev­i­dence showed that, in Eastern Wash­ing­ton state, the “sca­bland” desert land­scape had en­dured an an­cient catas­tro­phy: a flood of stag­ger­ing pro­por­tions. This was out­right heresy, since the ge­ol­ogy com­mu­nity of the time had dog­matic be­lief in a “unifor­mi­tar­ian” po­si­tion, where all changes must take place slowly and in­cre­men­tally over vast time scales. Bretz’ ideas were en­tirely vin­di­cated by the 1950s. Quote: “All my en­e­mies are dead, so I have no one to gloat over.”

This one is ba­si­cally right and I’ll give it to them.

CHANDRASEKHAR:

Chan­dra origi­nated Black Hole the­ory and pub­lished sev­eral pa­pers. He was at­tacked vi­ciously by his close col­league Sir Arthur Ed­ding­ton, and his the­ory was dis­cred­ited in the eyes of the re­search com­mu­nity. They were wrong, and Ed­ding­ton ap­par­ently took such strong ac­tion based on an in­cor­rect pet the­ory of his own. In the end Chan­dra could not even pur­sue a ca­reer in England, and he moved his re­search to the U. of Chicago in 1937, la­bor­ing in rel­a­tive ob­scu­rity for decades.

Sort of true, but he was hardly shunned by the sci­en­tific com­mu­nity. He made his dis­cov­er­ies about black holes in the early 1930s, was well-re­ceived by many peo­ple, and won a Bronze Medal in some physics com­pe­ti­tion. In 1935, Ed­ding­ton at­tacked his the­ory, pos­si­bly be­cause Ed­ding­ton was racist and didn’t like In­dian peo­ple. But many other sci­en­tists, in­clud­ing Niels Bohr and Wolf­gang Pauli, con­tinued to sup­port him (quietly, so as not to offend Ed­ding­ton, which will be a re­cur­ring theme in these kinds of situ­a­tions). Chan­drasekhar was made a Fel­low of the Royal So­ciety in 1944, won the Royal Astro­nom­i­cal So­ciety Gold Medal in 1953, and gen­er­ally led a long and pres­ti­gious life. His the­o­ries were re­s­ur­rected once peo­ple had bet­ter ev­i­dence that black holes ex­isted. I’ll give this one half a point.

CHLADNI:

The sci­en­tific com­mu­nity re­garded Me­te­orites in the same way that mod­ern sci­en­tists re­gard UFO ab­duc­tions and psy­chic phe­nomenon: quaint su­per­sti­tions only be­lieved by peas­ant folk. All the eye­wit­ness re­ports were dis­be­lieved. At one point the ridicule be­came so in­tense that many mu­se­ums with me­te­orites in their ge­ol­ogy col­lec­tions de­cided to trash those valuable sam­ples. (Some­times hos­tile skep­ti­cism con­trols re­al­ity, and the strongest ev­i­dence is ed­ited to con­form to con­cen­sus dis­be­liefs.) Fi­nally in the early 1800’s Ernst Ch­ladni ac­tu­ally sat down and in­spected the ev­i­dence pro­fes­sion­ally, and found that claimed me­te­orites were en­tirely un­like known earth rocks. His study changed some minds. At the same time some large me­teor falls were wit­nessed by sci­en­tists, and the ma­jor­ity who in­sisted that only ig­no­rant peas­ants ever saw such things were shamed into silence.

As the quote points out, this is a kind of weird one as me­te­orite work was ridiculed for a long time, but Ch­ladni was taken se­ri­ously and helped change minds. Look­ing at Wikipe­dia, a lucky me­te­orite fall two years af­ter Ch­ladni first pub­lished his the­ory helped turn the tide in his fa­vor, and by ten years af­ter pub­li­ca­tion Ch­ladni’s me­te­orite the­o­ries were pretty well-re­garded. Even when peo­ple dis­agreed with him about me­te­orites, Ch­ladni re­mained widely re­spected for some of his other work in acous­tics.

There is a story here, but it’s prob­a­bly not right to cen­ter it around Ch­ladni, and his work was only scorned for a few years be­fore ev­ery­one agreed it was true. I’ll give this an­other half a point.

CRICK & WATSON

Not ridiculed. But they were in­structed to drop their re­search. They con­tinued it as “bootleg” re­search.

The list ad­mits they were “not ridiculed”. They were told to stop their re­search be­cause there was all sorts of aca­demic poli­tics around who was go­ing to be the first to dis­cover DNA, and the guy in charge of their uni­ver­sity was root­ing for an­other team.

DOPPLER

Pro­posed a the­ory of the op­ti­cal Dop­pler Effect in 1842, but was bit­terly op­posed for two decades be­cause it did not fit with the ac­cepted physics of the time (it con­tra­dicted the Lu­minifer­ous Aether the­ory.) Dop­pler was fi­nally proven right in 1868 when W. Hug­gins ob­served red shifts and blue shifts in stel­lar spec­tra. Un­for­tu­nately this was fif­teen years af­ter Dop­pler had died.

I haven’t been able to find any­thing about this in var­i­ous short on­line bi­ogra­phies of Dop­pler (1, 2). Dop­pler tested the effect him­self by hav­ing some­one play a trum­pet on a train (re­ally), some­one else suc­cess­fully tested it in 1845, and it was in­de­pen­dently re­dis­cov­ered in 1848. Dop­pler him­self was made the head of the In­sti­tute For Ex­per­i­men­tal Physics in Vienna and died about as pres­ti­gious and be­loved as a physi­cist can get.

So my im­pres­sion is that only a third of these peo­ple re­ally fit the pat­tern. Most of them were doubted for very short pe­ri­ods, con­tinued to be re­spected in their fields for their other ac­com­plish­ments even dur­ing those pe­ri­ods, or were part of medium-sized move­ments rather than be­ing lone ge­niuses. After a few years – maybe an av­er­age of ten, very rarely as long as thirty – their con­tri­bu­tions were rec­og­nized and they as­sumed their right­ful place in the pan­theon. Science isn’t perfect. But it is darned good.

[EDIT: Bill Beatty, au­thor of the origi­nal list, re­sponds here. My re­sponse to the re­sponse here.]

II.

I bring this up in the con­text of my last post on progress in the ra­tio­nal­ist move­ment. There used to be a stereo­type that ra­tio­nal­ists were too quick to challenge sci­en­tific con­sen­sus. I think that was ex­ag­ger­ated, but based on a core of truth. Given that we’re in­ter­ested in the ways that bias can pre­vent peo­ple from ac­cept­ing truth, it’s un­sur­pris­ing that we would fo­cus on cases like these.

But I per­son­ally have changed my think­ing on this a lot. Not in any way that I can ex­plain ex­plic­itly – I’ve always thought some­thing like:

Scien­tific con­sen­sus is the best tool we have for seek­ing truth. It’s not perfect, and it’s fre­quently over­turned by later sci­en­tists, but this is usu­ally – albeit not liter­ally always – the work of well-cre­den­tialed in­sid­ers, op­er­at­ing pretty quickly af­ter the ev­i­dence that should over­turn it be­comes available. Any in­di­vi­d­ual should be very doubt­ful of their abil­ity to beat it, while not be­ing so doubt­ful that no­body ever im­proves it and sci­ence can never progress.

– and I still think that. But I’ve shifted from be­ing the sort of per­son who shares viral lists of ma­ligned ge­niuses, to the sort of per­son who de­bunks those lists. I’ve started em­pha­siz­ing the “best tool we have” part of the sen­tence, and whisper­ing the “isn’t perfect” part, rather than vice versa.

I’ve changed my mind on this be­cause of per­sonal ex­pe­rience. Rather than try­ing to de­scribe it, it might be more helpful to give the most salient ex­am­ples.

1. The Repli­ca­tion Cri­sis: I pre­vi­ously thought the sci­en­tific con­sen­sus was flawed be­cause it failed to take the repli­ca­tion crisis se­ri­ously enough. I later learned that ev­ery­one else took the repli­ac­tion crisis ex­actly as se­ri­ously as I did. A poll in Na­ture shows that 90% of sci­en­tists be­lieve re­pro­ducibil­ity is­sues con­sti­tute a “crisis”, com­pared to only 3% (!) who don’t. For ev­ery per­son com­plain­ing about “method­olog­i­cal ter­ror­ists”, there are a dozen who are very con­cerned and try­ing to change the way they prac­tice re­search.

This is es­pe­cially im­pres­sive be­cause as far as I can tell the whole shift hap­pened in about ten years. I would date the be­gin­ning of the crisis from Ioan­ni­dis’ origi­nal 2005 pa­per, al­though it was only aimed at medicine. It got into high gear in psy­chol­ogy some­time around 2011 with Si­mon­sohn’s False Pos­i­tive Psy­chol­ogy. A Google Trends anal­y­sis sug­gests peo­ple only started search­ing the rele­vant key­words around 2013.

I started think­ing about this sort of thing in 2009 af­ter read­ing this LW post. At the time I thought this was some sort of ex­cit­ing failure of mod­ern sci­ence that I alone had figured out. But this was well af­ter sharp peo­ple like Ioan­ni­dis were talk­ing about it, and only a few years be­fore ev­ery­one was talk­ing about it. Fram­ing this as “I was right and sci­en­tific con­sen­sus was wrong” seems grandiose. Bet­ter might be “I started bet­ting on a win­ning horse about a quar­ter of the way be­tween the be­gin­ning of the race and when its vic­tory be­came blatantly ob­vi­ous to ev­ery­one”.

2. Nutri­tion: The Bad Old Paradigm of nu­tri­tion says that obese peo­ple just have poor im­pulse con­trol, that weight is a sim­ple mat­ter of calories in vs. calories out, and that all calories are equally good ex­cept fat, which for some in­ex­pli­ca­ble rea­son is the Devil. Any­body who’s read a few good books about nu­tri­tion sci­ence knows that the Bad Old Paradigm is woe­fully in­ad­e­quate. I read a few of those books and be­came con­vinced that I was right and sci­en­tific con­sen­sus was wrong.

Un­for­tu­nately, this whole is­sue ex­ploded when Gary Taubes pub­lished Good Calories, Bad Calories, which as best I can tell com­bined the first pub­li­cly available good cri­tique of the Bad Old Paradigm with a flawed and ba­si­cally false at­tempt at a new paradigm. There were lots of con­fused at­tacks against Taubes’ bad in­for­ma­tion which did col­lat­eral dam­age to his good in­for­ma­tion, and lots of con­fused defenses of his good in­for­ma­tion which in­ad­ver­tently shielded his bad in­for­ma­tion from crit­i­cism. I pre­vi­ously fo­cused on defend the good parts, but re­cently shifted more to­wards crit­i­ciz­ing the bad parts.

After read­ing some more good books here (one of which I hope to re­view soon), my im­pres­sion is that most nu­tri­tion sci­en­tists don’t be­lieve in the Bad Old Paradigm and haven’t for a while. At the very least, most of them seem to be­lieve in the li­po­stat and think it’s im­por­tant, which is my proxy for “ba­si­cally has their heart in the right place”. In­so­far as the Bad Old Paradigm con­tinues to be pop­u­lar wis­dom, it’s be­cause of the diet in­dus­try, the gov­ern­ment, so­cial in­er­tia, and no­body re­ally hav­ing a good new paradigm to re­place it with. I’m grad­u­ally see­ing pop­u­lar wis­dom shift, and nu­tri­tion sci­en­tists them­selves seem to be helping this pro­cess rather than hurt­ing it.

Maybe some­body in this area has dis­cov­ered the new paradigm and is a mav­er­ick be­ing per­se­cuted by hide­bound re­ac­tionar­ies. But it isn’t Gary Taubes. And it cer­tainly isn’t me.

3. So­cial-Jus­tice-Re­lated Is­sues: Another nar­ra­tive I used to be­lieve was that a lot of sketchy ideas were be­ing flat­tered be­cause they spoke to left-lead­ing aca­demics’ bi­ases in fa­vor of so­cial jus­tice. Im­plicit as­so­ci­a­tion tests, stereo­type threat, the idea of zero mean­ingful psy­cholog­i­cal differ­ences be­tween men and women, et cetera.

When I started wor­ry­ing about im­plicit as­so­ci­a­tion tests, I thought I was defy­ing some kind of broad sci­en­tific con­sen­sus. But the meta-analy­ses show­ing the Im­plicit As­so­ci­a­tion Test didn’t do what peo­ple thought had been around since 2009 and have only got­ten more nu­mer­ous since then, with broad me­dia cov­er­age. Prob­lems with stereo­type threat re­search are get­ting main­stream cov­er­age and even air­time on NPR.

The prob­lem here is that there was no equiv­a­lent of the Na­ture poll on the repli­ca­tion crisis, so I didn’t re­al­ize any of this was hap­pen­ing un­til just re­cently. For ex­am­ple, in 2016 this Voxs­plainer made it sound like there was a mono­lithic con­sen­sus in fa­vor of Im­plicit As­so­ci­a­tion Tests that no sane per­son had ever dis­agreed with, even though by that point there were already sev­eral big meta-analy­ses find­ing they weren’t prac­ti­cally use­ful. The cor­rect con­clu­sion isn’t that this is re­ally what sci­en­tific con­sen­sus thinks. The cor­rect con­clu­sion is that Vox shouldn’t be trusted about any sci­ence more com­pli­cated than the wedge vs. in­clined plane. Once I re­al­ized that there was all this in­tel­li­gent anal­y­sis go­ing on that I’d never heard about, my claim to be boldly defy­ing the sci­en­tific con­sen­sus evap­o­rated.

Yes, Cordelia Fine is still around and is still writ­ing books ar­gu­ing against gen­der differ­ences. But she’s start­ing to sound re­ally defen­sive, ba­si­cally the liter­ary equiv­a­lent of “I know I’m go­ing to be down­voted to hell for this, but…”. Mean­while, other sci­en­tists are do­ing a good job point­ing out the flaws in her books and con­duct­ing stud­ies like this biggest-ever look at male vs. fe­male brain differ­ences, this mag­is­te­rial look at per­son­al­ity differ­ences, et cetera – not to men­tion great and widely-ac­cepted work on how in­ter­sex peo­ple take on more char­ac­ter­is­tics of their hor­monal than their so­cial gen­der (hon­estly, we should prob­a­bly thank trans­gen­der peo­ple for mak­ing this field so­cially ac­cept­able again). Peo­ple talk a lot about how Larry Sum­mers was fired from Har­vard for talk­ing about male vs. fe­male differ­ences, but Steven Pinker did a whole de­bate on this and re­mains a Har­vard pro­fes­sor.

Even things about ge­netic psy­cholog­i­cal differ­ences be­tween pop­u­la­tion groups are less bold and mav­er­ick-y than their pro­po­nents like to think. The rele­vant sur­veys I know try­ing to elicit sci­en­tific con­sen­sus (1, 2, 3) all find that, when asked anony­mously, most sci­en­tists think these differ­ences ex­plain about 25% – 50% of var­i­ance.

I hate to bring that up, be­cause it’ll prob­a­bly start a flame war in the com­ments, but I think it’s im­por­tant as a sign of ex­actly how hard it is to poli­ti­cize sci­ence. Global warm­ing skep­tics talk about how maybe the sci­en­tific con­sen­sus on global warm­ing is false be­cause cli­ma­tol­o­gists face poli­ti­cal pres­sure to bias their re­sults in fa­vor of the the­ory. But sci­en­tists study­ing these ar­eas face much more poli­ti­cal pres­sure, and as long as you give the sur­veys anony­mously they’re happy to ex­press hor­ren­dously taboo opinions. This is about the strongest ev­i­dence in fa­vor of the con­sen­sus on global warm­ing – and sci­en­tific con­sen­sus in gen­eral – that I could imag­ine.

4. Nu­ture As­sump­tion and Blank Slatism: The pro­logue of the first edi­tion of The Nur­ture As­sump­tion is Ju­dith Rich Har­ris tel­ling her “mav­er­ick ge­nius kept down by hide­bound re­ac­tionar­ies” story. But the pro­logue of the sec­ond edi­tion is her be­ing much more hope­ful:

To some ex­tent at least, times have changed…there is now more ac­cep­tance of the idea that be­hav­ior is in­fluenced by genes and that in­di­vi­d­ual differ­ences in be­hav­ior are due in part to differnces in genes. Peo­ple are more will­ing to ad­mit that chil­dren can in­herit be­hav­ioral quirks and per­son­al­ity char­ac­ter­is­tics…was it this cul­tural shift that led to greater ac­cep­tance of my the­ory? Or was it the fact that new find­ings, con­sis­tent with the the­ory, kept turn­ing up? Over time, the early, an­gry re­sponse to The Nur­ture As­sump­tion has soft­ened no­tice­ably, both within and out­side of academia. To­day, the book is widely cited in text­books and jour­nal ar­ti­cles. It’s as­signed and dis­cussed in courses in many col­leges and uni­ver­si­ties; it shows up in ex­ams…in his fore­ward to the first edic­tion of The Nur­ture As­sump­tion, Steven Pinker made a rash pre­dic­tion about the book: “I pre­dict it will come to be seen as a turn­ing point in the his­tory of psy­chol­ogy”. Per­haps it is too soon to judge whether psy­chol­ogy has rounded a bend; per­haps it will take the per­spec­tive of twenty or thirty years. Even at this point, though, there are signs of a slight shift in di­rec­tion. Within de­vel­op­men­tal psy­chol­ogy, I’ve no­ticed that de­scrip­tions of pro­ce­dures and re­sults are be­gin­ning to sound a bit defen­sive. Greater progress has been made in other ar­eas of psy­chol­ogy. And the email I re­ceive from stu­dents gives me high hopes for the younger gen­er­a­tion com­ing up.

There were ten years be­tween the first and sec­ond edi­tions of The Nur­ture As­sump­tion. In the al­most ten years since the pub­li­ca­tion of the sec­ond edi­tion, my im­pres­sion is that its ideas have be­come even more widely-ac­cepted. This month’s edi­tion of the Amer­i­can Jour­nal of Psy­chi­a­try, onbe of the top jour­nals in the field, has a great study show­ing that child abuse does not cause cog­ni­tive dis­abil­ity, in con­trast to sev­eral pre­vi­ous stud­ies in the area. It cites Deary, Plomin, and Ioan­ni­dis, hits all of the talk­ing points about ge­netic con­found­ing of de­vel­op­men­tal out­comes, and re­ceives glow­ing en­dorse­ment in the jour­nal’s ed­i­to­rial sec­tion, which says that “if our causal ex­pla­na­tions are wrong, we may be wast­ing our effort or even do­ing dam­age”. Every sin­gle psy­chi­a­trist in the coun­try is get­ting ex­posed to this way of think­ing.

And this has real re­sults. I got to pre­sent a sum­mary of be­hav­ioral ge­net­ics to a meet­ing of psy­chi­a­trists, in­clud­ing a lot of psy­cho­an­a­lysts, and I was shocked that most of them were at least a lit­tle re­cep­tive. I think they mi­s­un­der­stood it. I think they care­fully raised caveats in ex­actly the right places to en­sure they didn’t have to change any­thing they were do­ing. But the over­all re­sponse was “Oh, yeah, we’ve heard stuff like that, it seems plau­si­ble, good thing that for var­i­ous hard-to-ex­plain rea­sons none of it ap­plies to us.” This is what the first stage of progress looks like.

5. In­tel­li­gence Ex­plo­sion And AI Risk: This was an­other place where I and many of my friends thought we were right and the con­sen­sus was wrong. It was an­other place where a lot of self-ap­pointed defen­ders of the con­sen­sus told us we were crack­pots and needed to listen to what real sci­en­tists thought. And again, when I looked into it, there was no con­sen­sus against the idea and lots of promi­nent re­searchers were in fa­vor. Go­ing to the Asilo­mar Con­fer­ence and see­ing a bunch of peo­ple from MIT and Har­vard talk about how con­cerned they were re­ally opened my eyes on this. Google now has an AI Ethics Board, Berkeley, Oxford, and MIT have foun­da­tions work­ing on it, and peo­ple like Elon Musk and Bill Gates are in­volved. Bostrom’s sur­vey of AI re­searchers and some more re­cent and rigor­ous not-yet-pub­lished sur­veys I’ve heard about con­firm the im­pres­sion. No­body would ever say there’s a sci­en­tific con­sen­sus in fa­vor of Bostrom’s the­o­ries. But at this point I think it’s also in­defen­si­ble to say there’s a con­sen­sus against.

Bostrom first started writ­ing about these sorts of things ex­ten­sively in the early 2000s, so there was re­ally only a ten-year gap be­tween en­ter­ing the in­tel­lec­tual en­vi­ron­ment and it be­com­ing a (mostly) ac­cepted part of the es­tab­lished field. Those ten years felt pretty long while we were in them, but the abil­ity of a field to ac­cept an on-the-face-of-it com­pletely-in­sane-sound­ing the­ory within ten years seems to me a very strong ar­gu­ment against the hide­bound-re­ac­tionar­ies the­ory and a very strong ar­gu­ment for con­sid­er­ing sci­en­tific con­senses to be un­rea­son­ably effec­tive.

6. IQ: Another case where I wor­ried about ap­par­ent failure of sci­en­tific con­sen­sus due to poli­ti­cally bias. I cer­tainly en­coun­tered a lot of false­hoods around this when I was younger. My high school psy­chol­ogy text­book in­cluded a sec­tion claiming that all IQ tests were bi­ased to­wards rich white peo­ple be­cause they were based en­tirely on ques­tions like “how many shots be­low par is a bo­gey?” Then it pre­sented an “al­ter­nate IQ test” which “proved” that poor minori­ties had higher IQs than rich whites by ask­ing some other ques­tions with the op­po­site bias (I think they were about slang for drugs – cer­tainly an in­ter­est­ing way to fight stereo­types). This kind of thing nat­u­rally made me as­sume that no­body had any idea what was ac­tu­ally in IQ tests and sci­en­tists were idiots.

But more re­cently I’ve been read­ing ac­tual sur­veys, which find that about 97% of ex­pert psy­chol­o­gists and 85% of ap­plied psy­chol­o­gists agree that IQ tests mea­sure cog­ni­tive abil­ity “rea­son­ably well”. And 77% of ex­pert psy­chol­o­gists and 63% of ap­plied psy­chol­o­gists agree IQ tests are cul­ture-fair (with slightly differ­ent num­bers de­pend­ing on how you ask the ques­tion, but always about 50% of both groups).

This seems like less of a prob­lem with ex­pert con­sen­sus, and more of a prob­lem of no­body else (in­clud­ing text­book writ­ers!) listen­ing to ex­perts who are con­tinu­ally try­ing to beat re­al­ity into peo­ple’s heads. But I have a vague mem­ory of hav­ing re­cently seen a sur­vey (which I can’t find) that even ex­perts in softer fields like so­ciol­ogy are gen­er­ally in fa­vor of IQ and ad­mit that it has its uses. And even some left/​liberal sources like Vox and Fred­die deBoer are aware of the con­sen­sus and will­ing to re­spect it.

At the same time, I’ve en­coun­tered some peo­ple like Bors­boom and Nostalge­braist who have rel­a­tively so­phis­ti­cated (and limited) cri­tiques of IQ, and who have al­lowed me to round off other peo­ple’s less-well-framed cri­tiques to some­thing more like what they are say­ing and less like the stupid things my high school text­book said.

So it seems to me that gen­er­ally ex­perts agree with rea­son­able state­ments about IQ, and where they seem to dis­agree they may hold rea­son­able dis­agree­ments rather than un­rea­son­able ones. Again, where this fails is not in the ex­perts but in the abil­ity of peo­ple who don’t listen to the ex­perts to get dis­pro­por­tionate so­cial power and hide the ex­is­tence of the ex­pert con­sen­sus.

III.

Last week I wrote about uni­ver­sally-known crit­i­cisms of economists, like “they’re silly for as­sum­ing ev­ery­one be­haves perfectly ra­tio­nally”:

My im­pres­sion is that economists not only know about these crit­i­cisms, but in­vented them. Dur­ing the last few paradigm shifts in eco­nomics, the new guard lev­ied these com­plaints against the old guard, mostly won, and their ar­gu­ments per­co­lated down into the cul­ture as The Cor­rect Ar­gu­ments To Use Against Eco­nomics. Now the new guard is do­ing their own thing – be­hav­ioral eco­nomics, ex­per­i­men­tal eco­nomics, eco­nomics of effec­tive gov­ern­ment in­ter­ven­tion. The new paradigm prob­a­bly has a lot of prob­lems too, but it’s a pretty good bet that ran­dom peo­ple you stop on the street aren’t go­ing to know about them.

The same pat­tern ex­plains a lot of my con­cerns above. I knew some crit­i­cisms of a sci­en­tific paradigm. They seemed right. I con­cluded that sci­en­tists weren’t very smart and maybe I was smarter. I should have con­cluded that some cut­ting-edge sci­en­tists were mak­ing good crit­i­cisms of an old paradigm. I can still flat­ter my­self by say­ing that it’s no small achieve­ment to rec­og­nize a new paradigm early and bet on the win­ning horse. But the pat­tern I was see­ing was part of the pro­cess of sci­ence, not a con­dem­na­tion of it.

Most peo­ple un­der­stand this in­tu­itively about past paradigm shifts. When a cre­ation­ist says that we can’t trust sci­ence be­cause it used to be­lieve in phlo­gis­ton and now it be­lieves in com­bus­tion, we cor­rectly re­spond that this is ex­actly why we can trust sci­ence. But this les­son doesn’t always gen­er­al­ize when you’re in the mid­dle of a paradigm shift right now and hav­ing trou­ble see­ing the other side.

I re­al­ize I’m (iron­i­cally) risk­ing mak­ing my nar­ra­tive of sci­en­tific suc­cess un­falsifi­able. Sup­pose some­one wants to ar­gue that sci­en­tific con­sen­sus is wrong. If they point to some­thing it used to be wrong about, I can re­spond “Yes, but it self-cor­rected and it’s cor­rect now, so that’s fine.” If they point to some­thing where cut­ting-edge sci­en­tists say it’s wrong but no­body else agrees, I can re­spond “Yes, this is what the be­gin­ning of a paradigm shift looks like, so that’s fine”. And if they point to some­thing where no­body in the field thinks it’s wrong, I can say “You’re a crack­pot for go­ing against all rep­utable sci­en­tists; the prob­lem is with you.” And if later they turn out to be right, and ev­ery­one ac­knowl­edges it, I can say “Yes, but it self-cor­rected and it’s cor­rect now, so that’s fine.”

(and I’m mak­ing it even eas­ier for my­self in that I say “sci­en­tific con­sen­sus for” when I prob­a­bly mean “no sci­en­tific con­sen­sus against”. I don’t claim that 90%+ of sci­en­tists always be­lieve true things, only that there are very few cases where 90%+ of sci­en­tists be­lieve things which smarter peo­ple know to be false.)

Against this I can only offer a per­sonal nar­ra­tive: the only light I have by which to judge sci­en­tific con­sen­sus is my own In­side View as­sess­ment of what seems cor­rect. Again and again I have tried to defy sci­en­tific con­sen­sus. And ev­ery time, I ei­ther find that I am wrong, find that I am a few years ahead of a trend that most sci­en­tists even­tu­ally agree with, or find that what I thought was “sci­en­tific con­sen­sus” was ac­tu­ally a fic­tion ped­dled by bi­ased in­dus­try or me­dia sources slan­der­ing a sci­en­tific com­mu­nity which ac­tu­ally had a much more so­phis­ti­cated pic­ture. My his­tory of try­ing to fight sci­en­tific con­sen­sus has been a Man Who Was Thurs­day-es­que se­ries of em­barass­ments as I find again and again that my sup­posed en­emy agrees with me and is even bet­ter at what I am try­ing to do than I am.

Scien­tific con­sen­sus hasn’t just been ac­cu­rate, it’s been un­rea­son­ably ac­cu­rate. Hu­mans are fal­lible be­ings. They are not known for their abil­ity the change their mind, to will­ingly ac­cept new in­for­ma­tion, or to put truth-seek­ing above poli­ti­cal squab­bles. And our mod­ern so­ciety is not ex­actly known for be­ing an apoli­ti­cal philoso­pher-king­dom with strong truth-seek­ing in­sti­tu­tions com­pletely im­mune from par­ti­san pres­sure. I feel a deep temp­ta­tion to sym­pa­thize with global warm­ing de­nial­ists who worry that the cli­ma­tolog­i­cal con­sen­sus is bi­ased poli­ti­cized crap, be­cause that is ex­actly the sort of thing which I would ex­pect to come out of our bi­ased poli­ti­cized crappy so­ciety. Yet again and again I have seen ex­am­ples of sci­en­tific fields that have main­tained strong com­mit­ments to the truth in the face of pres­sure that would shat­ter any lesser in­sti­tu­tion. I’ve seen fields where peo­ple be­lieve in­cred­ibly-bizarre sound­ing things that will get them mocked at cock­tail par­ties just be­cause those things seem to be backed by the ma­jor­ity of the ev­i­dence. I’ve even seen peo­ple change their minds, in spite of all the in­cen­tives to the con­trary. I can’t ex­plain this. The idea that sci­en­tific con­sen­sus is al­most always an ac­cu­rate re­flec­tion of the best knowl­edge we have at the time seems even more flab­ber­gast­ing than any par­tic­u­lar idea that sci­en­tists might or might not be­lieve. But it seems to be true.

(note that I’m talk­ing about “sci­en­tific con­sen­sus” to mean a very high-level pat­tern, con­sist­ing of hun­dreds of sci­en­tists over the space of decades eval­u­at­ing a broad body of work. Any in­di­vi­d­ual study is still prob­a­bly to­tal garbage.)

Given how weird all of this is, I re­al­ize there’s an­other pos­si­ble bias here that should be taken very se­ri­ously – which is that I’m wrong about one or both sides of this. Which is more likely: that Science always agrees with Truth? Or that one guy’s per­cep­tion of Science always agrees with that same guy’s per­cep­tion of Truth? The lat­ter gives me two de­grees of free­dom: I can ei­ther cherry-pick ex­perts who agree with me and de­clare them to be Con­sen­sus, or I can con­form my opinions to con­sen­sus so slav­ishly that I end up dis­cov­er­ing only that Con­sen­sus agrees with it­self. I don’t feel like I’m mak­ing this kind of mis­take. But then again, no­body ever feels like they’re be­ing bi­ased.

But if I’m mak­ing this mis­take, I think it’s at least a bet­ter mis­take than the one where peo­ple dream up sto­ries about be­ing mav­er­icks per­se­cuted by hide­bound re­ac­tionar­ies. This mis­take at least sets the terms of de­bate as “let’s try to as­cer­tain what the sci­en­tific com­mu­nity thinks” and for­bids me from be­liev­ing com­pletely crack­pot­tish things. And it en­courages trust in one of our more trust­wor­thy pub­lic in­sti­tu­tions, always a proso­cial sort of thing to do. I would rather have a world of peo­ple de­bat­ing who agrees with sci­en­tific con­sen­sus or not, than a world of peo­ple de­bat­ing whether sci­en­tific con­sen­sus is even valuable.

There are two caveats to the above. First, I think it’s dan­ger­ous to pro­mote a norm of agree­ing with sci­en­tific con­sen­sus, in­so­far as that helps en­courage ex­actly the mis­takes about the na­ture of con­sen­sus that I dis­cussed above. When poorly-in­formed diet in­dus­try gu­rus sup­port the Bad Old Paradigm, their ral­ly­ing cry is usu­ally “You’re a stupid crack­pot, bow to the sci­en­tific con­sen­sus which agrees with me”. I gave three ex­am­ples above of cases where I would have got­ten the sci­en­tific con­sen­sus 100% wrong if I didn’t have ac­cess to a for­mal sur­vey of sci­en­tific ex­perts. In a world where these sur­veys had never been done – or some ex­ist­ing field with­out these sur­veys – or some field where these sur­veys have been done in­ac­cu­rately or in a bi­ased man­ner – peo­ple will of­ten be­lieve the con­sen­sus to be the op­po­site of what it re­ally is. In those cases, de­mands that peo­ple re­spect con­sen­sus can be used to shut down peo­ple who are ac­tu­ally right – the field-wide equiv­a­lent of call­ing true facts you don’t like de­bunked and well-re­futed. I see this hap­pen­ing all the time and I worry that wax­ing too po­et­i­cally about the un­rea­son­able effec­tive­ness of sci­en­tific con­sen­sus will only serve to em­power these peo­ple. Good­hart’s Law says that a mea­sure which be­comes a tar­get ceases to be a use­ful mea­sure, so we should be re­luc­tant to tar­get sci­en­tific con­sen­sus too strongly.

And sec­ond, I think that even when the Out­side View tells you that the con­sen­sus is cor­rect, you should con­tinue pur­su­ing your In­side View hunch that it isn’t. This avoids awk­ward situ­a­tions like ev­ery in­di­vi­d­ual sci­en­tist doubt­ing the con­sen­sus, but sup­press­ing their doubts be­cause the “sci­en­tific con­sen­sus” has to be right.

So maybe the things I’m say­ing about sci­en­tific con­sen­sus aren’t very ac­tion­able. But re­spect­ing sci­en­tific con­sen­sus in a non-ac­tion­able way is a lot less ex­haust­ing than be­liev­ing your­self to be against it, and talk­ing about how you’re against it, and tak­ing flak for be­ing against it. And in the same way it’s helpful to be­lieve that God is good, even if He never re­ally gets around to do­ing much about it, so it’s re­as­sur­ing to be able to have faith in our in­sti­tu­tions ev­ery so of­ten.

No comments.