Spec­u­la­tions on im­prov­ing debating

Link post

I was re­cently dis­cuss­ing with a friend whether or not com­pet­it­ive de­bat­ing makes you bet­ter at fig­ur­ing out what is ac­tu­ally true. This is an in­ter­est­ing ques­tion, be­cause de­bat­ing in­flu­ences people in a vari­ety of dif­fer­ent ways. The most ba­sic is that par­ti­cip­at­ing in de­bates, or pre­par­ing for them, teaches you a lot of facts about the world; I would know much less about in­ter­na­tional polit­ics in par­tic­u­lar were it not for de­bat­ing. It also markedly im­proves your abil­ity to no­tice flaws and fal­la­cies in ar­gu­ments, which ob­vi­ously helps you avoid fall­ing for them. This skill can ex­tend too far, though: de­baters can be very good at re­but­ting even very sens­ible ar­gu­ments. In the Amer­ican Parlia­ment­ary style of de­bate, the pro­pos­ing team is al­lowed to choose and pre­pare in ad­vance any mo­tion which is not so blatantly, ob­vi­ously true that it would be im­possible to de­bate (e.g. “murder is bad”). A lot of pro­pos­i­tion teams skirt quite close to that line—but even so, op­pos­ing teams win a ma­jor­ity of the time. De­baters can poke holes in es­sen­tially any ar­gu­ment.

To be fair, de­baters are aware that the ex­ist­ence of con­vin­cing re­but­tals doesn’t ne­ces­sar­ily in­val­id­ate a claim. But mo­tiv­ated reas­on­ing can be a power­ful force, es­pe­cially amongst in­tel­li­gent people. This is par­tic­u­larly true when you’ve been trained to re­spond op­pos­i­tion­ally to any claim that doesn’t sup­port your cur­rent po­s­i­tion. I can think of sev­eral cases where I’ve been able to ig­nore a nag­ging voice of doubt by quickly find­ing a plaus­ible re­sponse that turned out to be in­cor­rect. The so­cial me­dia fil­ter bubbles that we all cur­rently in­habit can amp­lify this ef­fect by cach­ing in your mind hun­dreds of ex­amples en­gin­eered to be mem­or­able and stoke out­rage; to counter it, it’s more im­port­ant than ever to genu­inely con­sider op­pos­ing ar­gu­ments rather than go­ing to the auto­matic re­but­tal mode that de­bat­ing in­grains.

The other thing that de­bat­ing (ar­gu­ably) does is cul­tiv­ate a non-em­pir­ical mind­set. At the end of the de­bate, the ques­tion isn’t settled. There may be more clar­ity over ex­actly which factors might swing a con­clu­sion one way or the other, but al­most nobody ever both­ers to go out and do that fur­ther re­search, or even fact-check claims made dur­ing the de­bate—des­pite the fact that it’s now easier to ac­cess such data than ever be­fore. Of course if you asked de­baters ex­pli­citly they’d be clear that em­pir­ical res­ults are usu­ally re­quired to ac­tu­ally draw firm con­clu­sions. But if you spend long enough think­ing and ar­guing in a cer­tain way, it’s dif­fi­cult to ima­gine that it doesn’t carry over to your reas­on­ing in gen­eral.

At this point I want to be a little spec­u­lat­ive. I’m not sure whether the is­sues I dis­cussed above can be ad­dressed within any­thing re­sem­bling a tra­di­tional de­bate (I’d be in­ter­ested to hear your com­ments on this). But let’s say that we want to design an en­tirely new form of de­bat­ing which in­stilled the best pos­sible habits of thought. What could it look like? It would need to be driven by em­pir­i­cism, while still hav­ing room for con­cep­tual ana­lysis. It would still be fun and com­pet­it­ive, but de­bates would also build up the sort of know­ledge which could ac­tu­ally help drive de­cisions. Do­main-spe­cific know­ledge would be help­ful but not es­sen­tial.

This was in the back of my mind when I read about two very in­ter­est­ing stud­ies. The first was by re­search­ers at Uber: they cal­cu­lated the will­ing­ness of con­sumers to pay for Uber rides by com­par­ing times when the cal­cu­lated surge in­dex was very sim­ilar, but the ac­tual surge price was dif­fer­ent, e.g. 2.24 vs 2.25, roun­ded to 2x and 2.5x price in­creases re­spect­ively. The study was heav­ily cri­ti­cised for equat­ing will­ing­ness to pay in­creased surge prices with over­all “con­sumer sur­plus” cre­ated by Uber, but the ini­tial meth­od­o­logy was still quite clever. The second stud­ied Italy, which is ap­par­ently the bank rob­bery cap­ital of Europe. Re­search­ers ana­lysed the re­la­tion­ship between dur­a­tion of rob­ber­ies, amount stolen, prob­ab­il­ity of be­ing caught and prison sen­tence. Their hope was that there would be some con­sist­ent tradeoff between the ex­pec­ted gain and ex­pec­ted pun­ish­ment; it turned out that more cap­able rob­bers be­haved as if they as­signed higher dis­util­ity to prison time than less cap­able ones. This sort of cre­at­ive ap­proach to an­swer­ing im­port­ant ques­tions is some­thing we sorely need more of; let’s con­sider a new form of de­bat­ing, neode­bat­ing, which is aimed at en­cour­aging it.

The core ideas un­der­ly­ing neode­bat­ing would be that, in­stead of de­bat­ing about what is true, we could de­bate about how to find out what is true; and in­stead of de­bat­ing what the ef­fects of a cer­tain policy would be, we could de­bate which policy would best cre­ate cer­tain ef­fects. In some de­bates this would look like both sides design­ing ex­per­i­mental meth­od­o­lo­gies like the ones I out­lined above, then cri­tiquing and de­fend­ing them. In oth­ers, it would re­quire teams to dream up spe­cific in­ter­ven­tions, e.g. “What’s the best way to build stronger com­munity spirit in de­prived areas?” De­bates could fo­cus on per­sonal de­cisions: “Your best friend is go­ing through a mid­life crisis and feels like their life is mean­ing­less. What do you do about it?” But they could also be about some of the biggest mod­ern is­sues: “Redesign the edu­ca­tion sys­tem to make it as ef­fect­ive as pos­sible in con­vey­ing skills and know­ledge.” These de­bates would be judged on the cur­rent stand­ards of per­suas­ive­ness and elo­quence, but also on cre­ativ­ity and bold­ness. The most fun de­bates I’ve ever done are the ones which in­tro­duced me to totally new ideas: in an ideal im­ple­ment­a­tion of neode­bat­ing, every de­bate would be like that. And I think the mind­set re­quired is ex­actly what we should want from polit­ical lead­ers—an in­nov­at­ive, ex­per­i­mental ap­proach to find­ing the best policies, plus know­ledge of how to test and eval­u­ate them. It would even pro­mote the stat­ist­ical lit­er­acy re­quired to identify and ar­gue about cor­rel­a­tion, caus­a­tion, con­founders and con­trols. Wouldn’t that be great? If I were be­ing really ideal­istic, I’d even build in a mech­an­ism for teams in a de­bate to make bets about rel­ev­ant fu­ture events, with points awar­ded ret­ro­act­ively to whichever team turned out to be cor­rect.

Any­way, enough day­dream­ing. For now, I think my main point is that every­one—but de­baters in par­tic­u­lar—should spend some time an­swer­ing a few ques­tions. Which ar­gu­ments do you cri­tique most rig­or­ously? Do you ever try to gen­er­ate novel and cre­at­ive solu­tions to big prob­lems? Do you seek out enough em­pir­ical data? And when and how do you ac­tu­ally change your mind?