ClearerThinking’s Fact-Checking 2.0

Cross-posted from Huffing­ton Post. See also The End of Bul­lshit at the Hands of Crit­i­cal Ra­tion­al­ism.

De­bat­ing sea­son is in full swing
, and as per usual the pres­i­den­tial can­di­dates are play­ing fast and loose with the truth. Fact-check­ing sites such as Poli­tiFact and Fac­ have had plenty of easy tar­gets in the de­bates so far. For in­stance, in the CNN Repub­li­can de­bate on Septem­ber 16, Fio­rina made sev­eral du­bi­ous claims about the Planned Par­ent­hood video, as did Cruz about the Iran agree­ment. Similarly, in the CNN Demo­cratic de­bate on Oc­to­ber 13, San­ders falsely claimed that the U.S. has “more wealth and in­come in­equal­ity than any other coun­try”, whereas Chafee fudged the data on his Rhode Is­land record. No doubt we are go­ing to see more of that in the rest of the pres­i­den­tial cam­paign. The fact-check­ers won’t need to worry about find­ing easy tar­gets.

Re­search shows that fact-check­ing ac­tu­ally does make a differ­ence. In­cred­ible as it may seem, the can­di­dates would prob­a­bly have been even more care­less with the truth if it weren’t for the fact-check­ers. To some ex­tent, fact-check­ers are a de­ter­rent to poli­ti­ci­ans in­clined to stretch the truth.

At the same time, the fact that false­hoods and mis­rep­re­sen­ta­tions of the truth are still so com­mon shows that this de­ter­rence effect is not par­tic­u­larly strong. This raises the ques­tion how we can make it stronger. Is there a way to im­prove on Poli­tiFact’s and Fac­’s model—Fact-Check­ing 2.0, if you will?

Spencer Green­berg of Clear­erThink­ing and I have de­vel­oped a tool which we hope could play that role. Green­berg has cre­ated an ap­pli­ca­tion to em­bed videos of recorded de­bates and then add sub­ti­tles to them. In these sub­ti­tles, I point out false­hoods and mis­rep­re­sen­ta­tions of the truth at the mo­ment when the can­di­dates make them. For in­stance, when Fio­rina says about the Planned Par­ent­hood video that there is “a fully formed fe­tus on the table, its heart beat­ing, its legs kick­ing, while some­one says we have to keep it al­ive to har­vest its brain”, I write in the sub­ti­tles:


We think that read­ing that a can­di­date’s state­ment is false just as it is made could have quite a strik­ing effect. It could trig­ger more visceral feel­ings among the view­ers than stan­dard fact-check­ing, which is pub­lished in sep­a­rate ar­ti­cles. To over and over again read in the sub­ti­tles that what you’re be­ing told sim­ply isn’t true should out­rage any­one who finds truth-tel­ling an im­por­tant qual­ity.

Another salient fea­ture of our sub­ti­tles is that we go be­yond stan­dard fact-check­ing. There are many other ways of mis­lead­ing the au­di­ence be­sides play­ing fast and loose with the truth, such as eva­sions, ad hominem-at­tacks and other log­i­cal fal­la­cies. Many of these are hard to spot for the view­ers. We must there­fore go be­yond fact-check­ing and also do ar­gu­ment-check­ing, as we call it. If fact-check­ing grew more effec­tive, and mis­rep­re­sent­ing the truth less vi­able a strat­egy, poli­ti­ci­ans pre­sum­ably would more fre­quently re­sort to Plan B: evad­ing ques­tions where they don’t want the read­ers to know the truth. To stop that, we need care­ful ar­gu­ment-check­ing in ad­di­tion to fact-check­ing.

So far, I’ve an­no­tated the en­tire CNN Repub­li­can De­bate, a 12 minute video from the CNN Demo­cratic De­bate (more an­no­ta­tions of this de­bate will come) and nine short clips (1-3 min­utes) from the Fox News Repub­li­can De­bate (Au­gust 6). My aim is to be as com­plete as pos­si­ble, and I think that I’ve cap­tured an over­whelming ma­jor­ity of the fac­tual er­rors, eva­sions, and fal­la­cies in the clips. The videos can be found on Clear­erThink­ing as well as be­low.


The CNN Repub­li­can de­bate, sub­ti­tled in full.


The first 12 min­utes of the CNN Demo­cratic de­bate.


Nine short clips from the Fox News De­bate: Christie and Paul, Bush, Car­son, Cruz, Huck­abee, Ka­sich, Ru­bio, Trump, Walker.

What is per­haps most strik­ing is the sheer num­ber of false­hoods, eva­sions and fal­la­cies the can­di­dates make. The 2hr 55 min long CNN Repub­li­can de­bate con­tains 273 fact-check­ing and ar­gu­ment-check­ing com­ments (many of which re­fer to var­i­ous fact-check­ing sites). In to­tal, 27 % of the video is sub­ti­tled. Similar num­bers hold for the other videos.

Con­ven­tional wis­dom has it that poli­ti­ci­ans lie and de­ceive on a mas­sive scale. My analy­ses prove con­ven­tional wis­dom right. The can­di­dates use all sorts of trick­ery to put them­selves in a bet­ter light and smear their op­po­nents.

All of this trick­ery is severely prob­le­matic from sev­eral per­spec­tives. Firstly, it is likely to un­der­mine the vot­ers’ con­fi­dence in the poli­ti­cal sys­tem. This is es­pe­cially true for vot­ers on the los­ing side. Why be loyal to a gov­ern­ment which has gained power by mis­lead­ing the elec­torate? No doubt many vot­ers do think in those terms, more or less ex­plic­itly.

It is also likely to dam­age the image of democ­racy. The Amer­i­can pres­i­den­tial elec­tion is fol­lowed all over the world by mil­lions if not billions of peo­ple. Many of them live in coun­tries where democ­racy ac­tivists are strug­gling to amass sup­port against au­thor­i­tar­ian regimes. It hardly helps them that the elec­tion de­bates in the U.S. and other demo­cratic coun­tries look like this.

All of these de­cep­tive ar­gu­ments and claims also make it harder for vot­ers to make in­formed de­ci­sions. Tele­vised de­bates are sup­posed to help vot­ers to get a bet­ter view of the can­di­dates’ poli­cies and track-records, but how could they, if they can’t trust what is be­ing said? This is per­haps the most se­ri­ous con­se­quence of poor de­bates, since it is likely to lead to poorer de­ci­sions on the part of the vot­ers, which in turn will lead to poorer poli­ti­cal lead­er­ship and poorer poli­cies.

Be­sides func­tion­ing as a more effec­tive lie de­ter­rent to the can­di­dates, im­proved fact-check­ing could also nudge the net­works to ad­just the set-up of the de­bates. The way the net­works lead the de­bates to­day hardly en­courages se­ri­ous and ra­tio­nal ar­gu­men­ta­tion. To the con­trary, they of­ten pos­i­tively goad the can­di­dates against each other. Im­proved fact-check­ing could make it more salient to the view­ers how poor the de­bates are, and in­duce them to de­mand a bet­ter de­bate set-up. The net­works need to come up with a for­mat which in­cen­tivizes the can­di­dates to ar­gue fairly and truth­fully, and which makes it clear who has not. For in­stance, they could broad­cast the de­bate again the next day, with fact-check­ing and ar­gu­ment-check­ing sub­ti­tles.

Another means to im­prove the de­bates is fur­ther tech­nolog­i­cal in­no­va­tion. For ex­am­ple, there should be a video an­no­ta­tion equiv­a­lent to Ge­, the web ap­pli­ca­tion which al­lows you to an­no­tate text on any web­page in a con­ve­nient way. That would be very use­ful for fact-check­ing and ar­gu­ment-check­ing pur­poses.

Fact-check­ing could even be­come au­to­matic, as Google CEO Eric Sch­midt pre­dicted it would be within five years in 2006. Though Sch­midt was over-op­ti­mistic, Google al­gorithms are able to fact-check web­sites with a high de­gree of ac­cu­racy to­day, whilst Wash­ing­ton Post already has built a rudi­men­tary au­to­matic fact-checker.

But be­sides new soft­ware ap­pli­ca­tions and bet­ter de­bat­ing for­mats, we also need some­thing else, namely a raised aware­ness among the pub­lic what a great prob­lem poli­ti­ci­ans’ care­less at­ti­tude to the truth is. They should ask them­selves: are peo­ple in­clined to mis­lead the vot­ers re­ally suited to shape the fu­ture of the world?

Poli­ti­ci­ans are nor­mally held to high moral stan­dards. Vot­ers tend to take very strict views on other forms of dishon­est be­hav­ior, such as cheat­ing and tax eva­sion. Why, then, is it that they don’t take a stric­ter view on in­tel­lec­tual dishon­esty? Be­sides be­ing morally ob­jec­tion­able, in­tel­lec­tual dishon­esty is likely to lead to poor de­ci­sions. Vot­ers would there­fore be wise to let in­tel­lec­tual hon­esty be an im­por­tant crite­rion when they cast their vote. If they started do­ing that on a grand scale, that would do more to im­prove the level of poli­ti­cal de­bate than any­thing else I can think of.

Thanks to Ais­linn Pluta, Doug Moore, Janko Prester, Philip Thone­mann, Stella Val­lgårda and Staffan Holm­berg for their con­tri­bu­tions to the an­no­ta­tions.