Better debates
[Crossposted from https://tsvibt.blogspot.com/2023/05/better-debates.html.]
When two people disagree about a proposition even though they’ve thought about it alot, the disagreement is often hard to resolve. There’s a gulf of data, concepts, intuitions, experiences, inferences. Some of this gulf has to be resolved by the two people individually trying to collate and present their own positions more clearly and legibly, so that they can build up concepts and propositions in whoever is receiving the model. Also, most new understanding comes from people working on their own or with others who are already synced up—for the most part they already agree on what and how to investigate, they have shared context of past experience and data, they agree on background assumptions, they have a shared language, they trust each other.
But still, a lot of value comes from debate. The debaters are forced to make their evidence and logic legible. Ideas are tested against other ideas from another at least somewhat coherent perspective. Analogies and disanalogies are drawn out. Bundles of facts can be sifted, spot-checked, and compared; predictions can be resolved into updates. The case, in being laid out to the debate partner, is also made clearer to the audience. The audience can see where another expert thinks the strong and weak points are in a position.
So there’s value on the table from debates. But debates usually suck. People talk past each other. They waste effort talking about non-cruxy claims. The truth about verifiable claims isn’t made available, missing the chance to bring one or the other perspective into contact with reality. People cite statistics and analyses which become meaningless in a context where they can’t be investigated and can’t take on their full meaning by relating in detail to the discourse context.
What would a better debate look like? Below is a speculative list of ways to structure a debate so that it finds and explains more truth. As armchair speculation, this list is gesture or inspiration, not tested advice.
Multiple rounds. Have multiple conversations with the same debaters on the same topic. Debaters sometimes have to pause a thread of conversation and go off to collect data, clarify their case, reevaluate and update, look into counterdata, come up with a more concise response to critiques, pass the other’s Ideological Turing Test, or mull over the other’s concepts and perspective.
Seconds. Debaters bring seconds—people who can step in to make certain points, or can (being already versed in the perspective) efficiently work in the background to bring up information for the main debater to use.
Free-flowing. Debates often have a regimented structure, with a fixed overall length, and alternating statements with prescribed length. Don’t do that, let the debaters have back and forth.
Technical facilitation. Offload cognitive labor from the debaters by having support people who (perhaps on big visible screens):
Check easily-checkable claims.
Bring in numbers and diagrams and exact statements from citations.
Map the argument. Summarize claims and counterclaims, define terms (especially clarifying how each debater uses important words), track sub-points and digressions.
Track stated cruxes and double-cruxes.
Remind the debaters of the current thread stack, if needed.
Filter comments from an audience.
Discourse facilitation. Have one or two facilitators who more directly guide the discourse:
Encourage debaters to paraphrase each other’s position, and sometimes try to pass their Ideological Turing Test.
Highlight when the debaters are using terms differently, or are making unshared background assumptions.
MC the conversation, e.g. sometimes briefly summarizing the conversation so far, and summarizing prior rounds at the beginning of subsequent rounds.
Help avoid blindspots. E.g. if one debater is evading answering a question, bring that to light. E.g. if there are conflicts and/or conflict-orientation in the debate, help bring that to light.
Check if a claim under discussion is a crux or a double-crux for the debaters.
Avoid the discourse getting bogged down in uninteresting and nonuseful regimes, e.g. by going back to a previous topic or asking a provoking question.
Process smoothing. E.g. if the debaters have two threads that they want to separately explore, the facilitator is in charge of remembering that there was the other thread and bringing it back in at some point.
Context holding. E.g. (somehow) make it more ok for debaters to change their mind. E.g. if one of the debaters is struggling with a thought, then if needed instruct the other to give them some moments to think quietly.
Spirits. Somehow evoke these spirits:
The debaters are not trying to convince the audience of something, they’re trying to communicate with each other.
The debaters are trying to understand where each other is coming from, e.g. they paraphrase each other and try to pass each other’s ITT.
The debaters are trying to seek the truth, trying to coconstruct a coherent total perspective that integrates all the facts. The debate isn’t a debate, it’s a collision. E.g. the debate doesn’t have “sides” or voting for a winner or even a fixed proposition. It’s fine and encouraged if the debaters just start truth seeking and hypothesizing—thinking out loud with each other—instead of making contradictory statements. The disagreement is used as a pointer to where models are incorrect, incomplete, or not yet integrated.
To find what is actually helpful and feasible, one would have to run iterated experiments. A path could go like: first start with debates on topics that are cool/important/interesting, and are very concrete with lots of real-world data that can resolve disagreements. For example, nutrition questions like “Should you supplement with XYZ?” or “Is ABC bad for you?”. Then, when the support systems are showing real value, expand into more abstract, more difficult debates, like between linguistic theories or something. Then expand into more fraught areas such as policy debates or big controversies, including policies around AI.
Here’s a debate protocol that I’d like to try. Both participants independently write statements of up to 10K words and send them to each other at the same time. (This can be done through an intermediary, to make sure both statements are sent before either is received.) Then they take a day to revise their statements, fixing the uncovered weak points and preemptively attacking the other’s weak points, and send them to each other again. This continues for multiple rounds, until both participants feel they have expressed their position well and don’t need to revise more, reaching a kind of Nash equilibrium. Then the final revisions of both statements are released to the public, side by side.
Note that in this kind of debate the participants don’t try to change each other’s mind. They just try to write something that will eventually sway the public. But they know that if they write wrong stuff that the other side can easily disprove, they won’t sway the public. So only the best arguments remain, within the size limit.
One note, based on my experience in across a variety of organizations, including holding a leadership role in a small political party, is that when a debate is “Free Flowing”, if it is taking place verbally (usually in-person or over video-call) the lack of definite structure and time-boxing can often lead to domination by whoever of the two or more interlocutors has either greater prowess in rhetorical skill, or is more willing to simply steamroll over the opportunity for the other to speak, or both. I think a balance may be struck by having structured rounds, with a pre-established limit for the number of claims each side may argue for or against, and then also allowing the debate to last some arbitrarily large number of rounds.
Much of the rest sounds desirable, though (as is also true of “Fact Checkers”) difficult to accomplish in a way which will satisfy all parties involved. Choosing someone or some group with a genuine openness to whatever the truth may be (or as close to that ideal as any person can have) is the most impactful action.
Does anyone here know of good examples of such forums for debate, either (recent) past or present?
Structured time boxes seem very suboptimal, steamrollering is easy enough to deal with by a moderator “Ok let’s pause there for X to respond to that point”
This would make a great YouTube series
Edit: I think I’m going to make this a YouTube series
You’ve noticed why debates suck for truth-seeking, but we seem to use them anyway. So I’m suspicious that the main purpose of debates isn’t actually truth-seeking (<-> Robin Hanson’s “X is not about Y”), but rather something else, like status contents.
In which case it’s all well and good to propose conversation schemas which are better for truth-seeking, but I’d rather call them by some new term, rather than “debates”.
Some of these problems vanish if you use an asynchronous textual format for debate. I’d really like to see decisionmaking processes that formalize the use of some sort of text forum, so I’ll propose some systems such a thing would use:
Flags indicating whether a claim has been refuted by deeper investigations, and who ran those investigations.
Comments from the general public, but subjectively filtered.
Methods for determining which authors are causing changes in peoples’ views, IE, which authors are doing the best work.
I wouldn’t call it ‘waste effort’ if the claims were not clearly and legibly indicated as such beforehand. The LW readerbase is easily many thousands or tens of thousands (?). So expecting every reader to reliably intuit which claims are which for even 10 front page posts a day seems too high of a bar to be practical.
Which ironically may be the case with this comment too.
The resolution is simple in theory, writers can restrict themselves to more parsimonious sets of claims, and/or add explicit indicators.