RSS

Ad­ver­sar­ial Collaboration

TagLast edit: 8 Mar 2021 14:36 UTC by Yoav Ravid

Adversarial collaboration is a protocol developed by Daniel Kahneman for two researchers advocating competing hypotheses to collaborate on a research project with the goal of resolving their differences, designed on the assumption that this will be more effective than each researcher conducting their own experiments individually and publishing replies to each others’ papers. Kahneman tested adversarial collaboration with Ralph Hertwig, aiming to resolve their dispute about whether the conjunction fallacy was primarily due to the representativeness heuristic (as advocated by Kahneman), or simply due to subjects interpreting the word “and” as a disjunction where the experimenters intended it to be used as a conjunction (as advocated by Hertwig). Kahneman and Hertwig collaborated on a series of experiments related to the issue, along with Barbara Mellers as an arbiter, and further refined the suggested adversarial collaboration protocol based on their experiences.

They suggest that the two disputing researchers should team up, along with a neutral arbiter to settle disputes that arise in the process, and agree on a procedure for an experiment that would distinguish between their hypotheses. The researchers would discuss ahead of time what results each of them expects, and what sorts of results would lead them to change their minds. Since the initial experiment may be inconclusive, each side would be allowed to propose follow-up studies afterwards, which they would continue to collaborate on. After the study is complete, all three researchers involved would collaborate on a paper describing the results, with the arbiter having responsibility for some sections, and each of the two disputing researchers given the chance to describe their own interpretation of the results.

Scott Alexander has arranged adversarial collaboration contests on his his blog, where people write an essay together.

2019:

2018:

External links

http://​​web.cenet.org.cn/​​upfile/​​21290.pdf (the protocol is summarized in table 1, at the top of page 2)

Guided By The Beauty Of Our Weapons

Scott Alexander24 Mar 2017 4:33 UTC
64 points
5 comments19 min readLW link

Look­ing for ad­ver­sar­ial col­lab­o­ra­tors to test our De­bate protocol

Beth Barnes19 Aug 2020 3:15 UTC
52 points
5 comments1 min readLW link

Iter­a­tive Ar­gu­ments: Alter­na­tive to Ad­ver­sar­ial Collaboration

matsn12 Sep 2018 15:56 UTC
27 points
9 comments1 min readLW link

Re­sults from an Ad­ver­sar­ial Col­lab­o­ra­tion on AI Risk (FRI)

11 Mar 2024 20:00 UTC
60 points
3 comments9 min readLW link
(forecastingresearch.org)