If we’re talking about the domain where we can assume “good human input”, why do we need a solution more complicated than direct human supervision/demonstration (perhaps amplified by reward models or models of human feedback)? I mean this non-rhetorically; I have my own opinion (that debate acts as an unprincipled way of inserting one round of optimization for meta-preferences [if confusing, see here]), but it’s probably not yours.
Identifying vs. evaluating flaws: In debate, human judges get to see critiques made by a superhuman system—some of which they probably wouldn’t have been able to identify themselves but can still evaluate (on the (IMO plausible) assumption that it’s easier to evaluate than generate). That makes human judges more likely to give correct signal.
Whole debate vs. single subclaim: In recursive debate, human judges evaluate only a single subclaim. The (superhuman) debaters do the work of picking out which subclaim they’re most likely to win on, and we use this as a proxy for who would’ve won the whole debate. This is a more efficient use of human judges’ time, and also probably it’s easier for human judges to evaluate smaller subproblems.
For those reasons, I think we can likely get “good human input” for a bigger set of questions with debate than direct human supervision
If we’re talking about the domain where we can assume “good human input”, why do we need a solution more complicated than direct human supervision/demonstration (perhaps amplified by reward models or models of human feedback)? I mean this non-rhetorically; I have my own opinion (that debate acts as an unprincipled way of inserting one round of optimization for meta-preferences [if confusing, see here]), but it’s probably not yours.
Two reasons:
Identifying vs. evaluating flaws: In debate, human judges get to see critiques made by a superhuman system—some of which they probably wouldn’t have been able to identify themselves but can still evaluate (on the (IMO plausible) assumption that it’s easier to evaluate than generate). That makes human judges more likely to give correct signal.
Whole debate vs. single subclaim: In recursive debate, human judges evaluate only a single subclaim. The (superhuman) debaters do the work of picking out which subclaim they’re most likely to win on, and we use this as a proxy for who would’ve won the whole debate. This is a more efficient use of human judges’ time, and also probably it’s easier for human judges to evaluate smaller subproblems.
For those reasons, I think we can likely get “good human input” for a bigger set of questions with debate than direct human supervision