However, this paper wants the answers to actually be correct. Thus, they claim that for sufficiently complicated questions, since the debate can’t reach the right answer, the debate isn’t truth-seeking—but in these cases, the answer is still in expectation more accurate than the answer the judge would come up with by themselves.
Truth-seeking: better than the answer the judge would have come up with by themself (how does this work? making an observation at random instead of choosing the observation that’s recommended by the debate?)
You have a prior; you choose to do the experiment with highest VOI to get a posterior, and then you choose the best answer given that posterior. I’m pretty sure I could calculate this for many of their scenarios.
Truth-seeking: better than the answer the judge would have come up with by themself (how does this work? making an observation at random instead of choosing the observation that’s recommended by the debate?)
Truth-finding: the truth is found.
You have a prior; you choose to do the experiment with highest VOI to get a posterior, and then you choose the best answer given that posterior. I’m pretty sure I could calculate this for many of their scenarios.