Speculations on improving debating

Link post

I was re­cently dis­cussing with a friend whether or not com­pet­i­tive de­bat­ing makes you bet­ter at figur­ing out what is ac­tu­ally true. This is an in­ter­est­ing ques­tion, be­cause de­bat­ing in­fluences peo­ple in a va­ri­ety of differ­ent ways. The most ba­sic is that par­ti­ci­pat­ing in de­bates, or prepar­ing for them, teaches you a lot of facts about the world; I would know much less about in­ter­na­tional poli­tics in par­tic­u­lar were it not for de­bat­ing. It also markedly im­proves your abil­ity to no­tice flaws and fal­la­cies in ar­gu­ments, which ob­vi­ously helps you avoid fal­ling for them. This skill can ex­tend too far, though: de­baters can be very good at re­but­ting even very sen­si­ble ar­gu­ments. In the Amer­i­can Par­li­a­men­tary style of de­bate, the propos­ing team is al­lowed to choose and pre­pare in ad­vance any mo­tion which is not so blatantly, ob­vi­ously true that it would be im­pos­si­ble to de­bate (e.g. “mur­der is bad”). A lot of propo­si­tion teams skirt quite close to that line—but even so, op­pos­ing teams win a ma­jor­ity of the time. De­baters can poke holes in es­sen­tially any ar­gu­ment.

To be fair, de­baters are aware that the ex­is­tence of con­vinc­ing re­but­tals doesn’t nec­es­sar­ily in­val­i­date a claim. But mo­ti­vated rea­son­ing can be a pow­er­ful force, es­pe­cially amongst in­tel­li­gent peo­ple. This is par­tic­u­larly true when you’ve been trained to re­spond op­po­si­tion­ally to any claim that doesn’t sup­port your cur­rent po­si­tion. I can think of sev­eral cases where I’ve been able to ig­nore a nag­ging voice of doubt by quickly find­ing a plau­si­ble re­sponse that turned out to be in­cor­rect. The so­cial me­dia filter bub­bles that we all cur­rently in­habit can am­plify this effect by caching in your mind hun­dreds of ex­am­ples en­g­ineered to be mem­o­rable and stoke out­rage; to counter it, it’s more im­por­tant than ever to gen­uinely con­sider op­pos­ing ar­gu­ments rather than go­ing to the au­to­matic re­but­tal mode that de­bat­ing in­grains.

The other thing that de­bat­ing (ar­guably) does is cul­ti­vate a non-em­piri­cal mind­set. At the end of the de­bate, the ques­tion isn’t set­tled. There may be more clar­ity over ex­actly which fac­tors might swing a con­clu­sion one way or the other, but al­most no­body ever both­ers to go out and do that fur­ther re­search, or even fact-check claims made dur­ing the de­bate—de­spite the fact that it’s now eas­ier to ac­cess such data than ever be­fore. Of course if you asked de­baters ex­plic­itly they’d be clear that em­piri­cal re­sults are usu­ally re­quired to ac­tu­ally draw firm con­clu­sions. But if you spend long enough think­ing and ar­gu­ing in a cer­tain way, it’s difficult to imag­ine that it doesn’t carry over to your rea­son­ing in gen­eral.

At this point I want to be a lit­tle spec­u­la­tive. I’m not sure whether the is­sues I dis­cussed above can be ad­dressed within any­thing re­sem­bling a tra­di­tional de­bate (I’d be in­ter­ested to hear your com­ments on this). But let’s say that we want to de­sign an en­tirely new form of de­bat­ing which in­stil­led the best pos­si­ble habits of thought. What could it look like? It would need to be driven by em­piri­cism, while still hav­ing room for con­cep­tual anal­y­sis. It would still be fun and com­pet­i­tive, but de­bates would also build up the sort of knowl­edge which could ac­tu­ally help drive de­ci­sions. Do­main-spe­cific knowl­edge would be helpful but not es­sen­tial.

This was in the back of my mind when I read about two very in­ter­est­ing stud­ies. The first was by re­searchers at Uber: they calcu­lated the will­ing­ness of con­sumers to pay for Uber rides by com­par­ing times when the calcu­lated surge in­dex was very similar, but the ac­tual surge price was differ­ent, e.g. 2.24 vs 2.25, rounded to 2x and 2.5x price in­creases re­spec­tively. The study was heav­ily crit­i­cised for equat­ing will­ing­ness to pay in­creased surge prices with over­all “con­sumer sur­plus” cre­ated by Uber, but the ini­tial method­ol­ogy was still quite clever. The sec­ond stud­ied Italy, which is ap­par­ently the bank rob­bery cap­i­tal of Europe. Re­searchers analysed the re­la­tion­ship be­tween du­ra­tion of rob­beries, amount stolen, prob­a­bil­ity of be­ing caught and prison sen­tence. Their hope was that there would be some con­sis­tent trade­off be­tween the ex­pected gain and ex­pected pun­ish­ment; it turned out that more ca­pa­ble rob­bers be­haved as if they as­signed higher di­su­til­ity to prison time than less ca­pa­ble ones. This sort of cre­ative ap­proach to an­swer­ing im­por­tant ques­tions is some­thing we sorely need more of; let’s con­sider a new form of de­bat­ing, neode­bat­ing, which is aimed at en­courag­ing it.

The core ideas un­der­ly­ing neode­bat­ing would be that, in­stead of de­bat­ing about what is true, we could de­bate about how to find out what is true; and in­stead of de­bat­ing what the effects of a cer­tain policy would be, we could de­bate which policy would best cre­ate cer­tain effects. In some de­bates this would look like both sides de­sign­ing ex­per­i­men­tal method­olo­gies like the ones I out­lined above, then cri­tiquing and defend­ing them. In oth­ers, it would re­quire teams to dream up spe­cific in­ter­ven­tions, e.g. “What’s the best way to build stronger com­mu­nity spirit in de­prived ar­eas?” De­bates could fo­cus on per­sonal de­ci­sions: “Your best friend is go­ing through a midlife crisis and feels like their life is mean­ingless. What do you do about it?” But they could also be about some of the biggest mod­ern is­sues: “Redesign the ed­u­ca­tion sys­tem to make it as effec­tive as pos­si­ble in con­vey­ing skills and knowl­edge.” Th­ese de­bates would be judged on the cur­rent stan­dards of per­sua­sive­ness and elo­quence, but also on cre­ativity and bold­ness. The most fun de­bates I’ve ever done are the ones which in­tro­duced me to to­tally new ideas: in an ideal im­ple­men­ta­tion of neode­bat­ing, ev­ery de­bate would be like that. And I think the mind­set re­quired is ex­actly what we should want from poli­ti­cal lead­ers—an in­no­va­tive, ex­per­i­men­tal ap­proach to find­ing the best poli­cies, plus knowl­edge of how to test and eval­u­ate them. It would even pro­mote the statis­ti­cal liter­acy re­quired to iden­tify and ar­gue about cor­re­la­tion, cau­sa­tion, con­founders and con­trols. Wouldn’t that be great? If I were be­ing re­ally ideal­is­tic, I’d even build in a mechanism for teams in a de­bate to make bets about rele­vant fu­ture events, with points awarded retroac­tively to whichever team turned out to be cor­rect.

Any­way, enough day­dream­ing. For now, I think my main point is that ev­ery­one—but de­baters in par­tic­u­lar—should spend some time an­swer­ing a few ques­tions. Which ar­gu­ments do you cri­tique most rigor­ously? Do you ever try to gen­er­ate novel and cre­ative solu­tions to big prob­lems? Do you seek out enough em­piri­cal data? And when and how do you ac­tu­ally change your mind?