Noisy Poll Results And Reptilian Muslim Climatologists from Mars

Be­ware of Phan­tom Lizardmen

I have only done a lit­tle bit of so­cial sci­ence re­search, but it was enough to make me hate peo­ple. One study I helped with an­a­lyzed whether peo­ple from differ­ent coun­tries had differ­ent an­swers on a cer­tain psy­cholog­i­cal test. So we put up a web­site where peo­ple an­swered some ques­tions about them­selves (like “what coun­try are you from?”) and then took the psy­cholog­i­cal test.

And so of course peo­ple screwed it up in ev­ery con­ceiv­able way. There were the merely dumb, like the guy who put “male” as his na­tion­al­ity and “Amer­i­can” as his gen­der. But there were also the ac­tively mal­i­cious or at least an­noy­ing, like the peo­ple (yes, more than one) who wrote in “Mar­tian”.

I think we all prob­a­bly know some­one like this, maybe a cou­ple peo­ple like this.

I also think most of us don’t know some­one who be­lieves rep­tilian aliens in hu­man form con­trol all the ma­jor na­tions of Earth.

Public Policy Pol­ling’s re­cent poll on con­spir­acy the­o­ries mostly showed up on my Face­book feed as “Four per­cent of Amer­i­cans be­lieve lizard­men are run­ning the Earth”.

(of note, an ad­di­tional 7% of Amer­i­cans are “not sure” whether lizard­men are run­ning the Earth or not.)

Imag­ine the situ­a­tion. You’re at home, eat­ing din­ner. You get a call from some­one who says “Hello, this is Public Policy Pol­ling. Would you mind an­swer­ing some ques­tions for us?” You say “Sure”. An ex­tremely dig­nified sound­ing voice says – and this is the ex­act word­ing of the ques­tion – “Do you be­lieve that shape-shift­ing rep­tilian peo­ple con­trol our world by tak­ing on hu­man form and gain­ing poli­ti­cal power to ma­nipu­late our so­ciety, or not?” Then it urges you to press 1 if yes, press 2 if no, press 3 if not sure.

So first we get the peo­ple who think “Wait, was 1 the one for if I did be­lieve in lizard­men, or if I didn’t? I’ll just press 1 and move on to the next ques­tion.”

Then we get the peo­ple who are like “I never heard it be­fore, but if this nice pol­lster thinks it’s true, I might as well go along with them.”

Then we get the peo­ple who are all “F#&k you, pol­ling com­pany, I don’t want peo­ple call­ing me when I’m at din­ner. You screw with me, I tell you what I’m go­ing to do. I’m go­ing to tell you I be­lieve lizard peo­ple are run­ning the planet.”

And then we get the peo­ple who put “Mar­tian” as their na­tion­al­ity in psy­chol­ogy ex­per­i­ments. Be­cause some men just want to watch the world burn.

Do these three groups to­tal 4% of the US pop­u­la­tion? Seems plau­si­ble.

I re­ally wish polls like these would in­clude a con­trol ques­tion, some­thing ut­terly im­plau­si­ble even by lizard-peo­ple stan­dards, some­thing like “Do you be­lieve Barack Obama is a hip­popota­mus?” What­ever per­cent of peo­ple an­swer yes to the hippo ques­tion get sub­tracted out from the other ques­tions.

Poll An­swers As At­tire

Alas, not all weird poll an­swers can be ex­plained that eas­ily. On the same poll, 13% of Amer­i­cans claimed to be­lieve Barack Obama was the Anti-Christ. Sub­tract­ing our Lizard­man’s Con­stant of 4%, that leaves 9% of Amer­i­cans who ap­par­ently gave this an­swer with some­thing ap­proach­ing sincer­ity.

(a friend on Face­book pointed out that 5% of Obama vot­ers claimed to be­lieve that Obama was the Anti-Christ, which seems to be an­other piece of ev­i­dence in fa­vor of a Lizard­man’s Con­stant of 4-5%. On the other hand, I do en­joy pic­tur­ing some­one stand­ing in a vot­ing booth, think­ing to them­selves “Well, on the one hand, Obama is the Anti-Christ. On the other, do I re­ally want four years of Rom­ney?”)

Some pol­lsters are start­ing to con­sider these sorts of things symp­tomatic of what they term sym­bolic be­lief, which seems to be kind of what the Less Wrong se­quences call Pro­fess­ing and Cheer­ing or Belief As At­tire. Ba­si­cally, peo­ple are be­ing emo­tivists rather than re­al­ists about be­lief. “Obama is the Anti-Christ” is an­other way of just say­ing “Boo Obama!”, rather than ex­press­ing some sort of propo­si­tion about the world.

And the same is true of “Obama is a Mus­lim” or “Obama was not born in Amer­ica”.

Never At­tribute To Stu­pidity What Can Be Ad­e­quately Ex­plained By Malice

But some­times it’s not some ab­struse sub­tle bias. Some­times it’s not a good-na­tured joke. Some­times peo­ple might just be ac­tively work­ing to cor­rupt your data.

Another link I’ve seen on my Face­book wall a few times is this one: Are Cli­mate Change Scep­tics More Likely To Be Con­spir­acy The­o­rists? It’s based on a pa­per by Stephen Le­wandowsky et al called NASA Faked The Moon Land­ing, There­fore Cli­mate Science Is A Hoax – An Anal­y­sis Of The Mo­ti­vated Re­jec­tion Of Science.

The pa­per’s the­sis was that cli­mate change skep­tics are mo­ti­vated by con­spir­acy ideation – a be­lief that there are large groups of sinister peo­ple out to de­ceive them. This seems sort of rea­son­able on the face of it – be­ing a cli­mate change skep­tic re­quires go­ing against the be­lief of the en­tire sci­en­tific es­tab­lish­ment. My guess is that there prob­a­bly is a sig­nifi­cant link here wait­ing to be dis­cov­ered.

Un­for­tu­nately, it’s…pos­si­ble Stephan Le­wandowsky wasn’t the best per­son to in­ves­ti­gate this? Aside from be­ing a pro­fes­sor of cog­ni­tive sci­ence, he also runs Shap­ing To­mor­row’s World, a group that pro­motes “re-ex­am­in­ing some of the as­sump­tions we make about our tech­nolog­i­cal, so­cial and eco­nomic sys­tems” and which seems to be largely about pro­mot­ing global warm­ing ac­tivism. While I think it’s ad­mirable that he is in­volved in that, it raises con­flict of in­ter­est ques­tions. And the way his pa­per is writ­ten – start­ing with the over-the-top ti­tle – doesn’t do him any fa­vors.

(if the con­flict of in­ter­est an­gle doesn’t make im­me­di­ate and ob­vi­ous sense to you, imag­ine how sketchy it would be if a pro­fes­sional global warm­ing de­nier was in­volved in re­search­ing the mo­ti­va­tions of global warm­ing sup­port­ers)

But enough of my per­sonal opinions. What’s the pa­per look like?

The method­ol­ogy goes like this: they send re­quests to sev­eral pop­u­lar cli­mate blogs, both be­liever and skep­tic, ask­ing them to link their read­ers to an on­line sur­vey. The sur­vey asks peo­ple their be­liefs on global warm­ing and on lots of con­spir­acy the­o­ries and fringe be­liefs.

On first glance, the re­sults are ex­tremely damn­ing. Peo­ple who re­jected cli­mate sci­ence were wildly more likely to re­ject pretty much ev­ery other form of sci­ence as well, in­clud­ing the “the­ory” that HIV causes AIDS and the “the­ory” that cigarettes cause can­cer. They were more will­ing to be­lieve aliens landed at Roswell, that 9-11 was an in­side job, and, yes, that NASA faked the moon land­ing. The con­clu­sion: cli­mate skep­tics are just re­ally stupid peo­ple.

But a bunch of global warm­ing skep­tics started re-an­a­lyz­ing the data and com­ing up with their own in­ter­pre­ta­tions. They found that many large pro-global-warm­ing blogs posted the link to the sur­vey, but very few anti-global-warm­ing blogs did. This then de­volved into liter­ally the worst flame war I have ever seen on the In­ter­net, cen­ter­ing around ac­cu­sa­tions about whether the study au­thors de­liber­ately ex­cluded large anti-global warm­ing blogs, or whether the au­thors asked the writ­ers of anti-global-warm­ing blogs and these writ­ers just ig­nored the re­quest (my im­pres­sion is that most peo­ple now agree it was the lat­ter). In ei­ther case, it ended up with most peo­ple tak­ing the sur­vey be­ing from the pro-global-warm­ing blogs, and only a few skep­tics.

More in­ter­est­ingly, they found that pretty much all of the link be­tween global warm­ing skep­ti­cism and stu­pidity was a cou­ple of peo­ple (there were so few skep­tics, and so few con­spir­acy be­liev­ers, that these cou­ple of peo­ple made up a pretty big pro­por­tion of them, and way more than enough to get a “sig­nifi­cant” differ­ence with the global warm­ing be­liev­ers). Fur­ther, most of these cou­ple of peo­ple had given the max­i­mally skep­ti­cal an­swer to ev­ery sin­gle ques­tion about global warm­ing, and the max­i­mally cre­d­u­lous an­swer to ev­ery sin­gle ques­tion about con­spir­a­cies.

The dan­ger here now seems ob­vi­ous. Global warm­ing be­liever blogs pub­lish a link to this study, say­ing glee­fully that it’s go­ing to prove that global warm­ing skep­tics are idiots who also think NASA faked the moon land­ing and the world is run by lizard­men or what­ever. Some global warm­ing be­liev­ers de­cide to help this pro­cess along by pre­tend­ing to be su­per-strong global warm­ing skep­tics and filling in the stupi­dest an­swers they can to ev­ery ques­tion. The few real global warm­ing skep­tics who take the sur­vey aren’t enough sig­nal to com­pletely drown out this noise. There­fore, they do the statis­tics and triumphantly an­nounce that global warm­ing skep­ti­cism is linked to stupid be­liefs.

The global warm­ing skep­tic blo­go­sphere has in my opinion done more than enough work to pre­sent a very very strong case that this is what hap­pened (some­body else do an in­de­pen­dent look at the con­tro­versy and dou­ble-check this for me?) And Pro­fes­sor Le­wandowsky’s an­swer was…

…to pub­lish a sec­ond pa­per, say­ing his re­sults had been con­firmed be­cause cli­mate skep­tics were so ob­sessed with con­spir­acy the­o­ries that they had ac­cused his data prov­ing they were ob­sessed with con­spir­a­cies of be­ing part of a con­spir­acy. The name of the pa­per? Re­cur­sive Fury. I have to hand it to him, this is pos­si­bly the most chutz­pah I have ever seen a sin­gle hu­man be­ing dis­play.

(the pa­per is now par­tially offline as the jour­nal in­ves­ti­gates it for eth­i­cal some­thing some­thing)

The les­son from all three of the cases in this post seems clear. When we’re talk­ing about very un­pop­u­lar be­liefs, polls can only give a weak sig­nal. Any pos­si­ble source of noise – jokesters, cog­ni­tive bi­ases, or de­liber­ate mis­be­hav­ior – can eas­ily over­whelm the sig­nal. There­fore, polls that rely on de­tect­ing very weak sig­nals should be taken with a grain of salt.