Correlated evidence can quietly break debate’s stability construction
Epistemic status: I’m formalizing a problem the original authors already flagged as open, not claiming to have discovered something out of nowhere. I don’t work on debate protocols and haven’t had this checked by anyone who does — that’s basically why I’m posting it.
Prover-estimator debate (Brown-Cohen, Irving & Piliouras, 2025) gets its completeness guarantee only when the recursive decomposition is stable, and stability is bought through a majority-vote construction (Theorem 8.3) that assumes each subclaim has independent pieces of supporting evidence. The authors are upfront that this independence “cannot be verified via calls to the oracles,” and they list finding a way around that as an open problem.
I wanted to know what actually happens if that independence fails. It’s worse than I expected, and in a specific way: a prover can hand over evidence that’s indistinguishable from honestly independent evidence at the level of any check a judge could run, while being correlated enough to completely gut the majority vote’s safety margin.
The setup
The relevant piece of the protocol takes redundant estimates of some ground-truth bit and combines them with . If the errors across the estimates are independent, a Chernoff bound drives the failure probability down exponentially in . That’s the whole mechanism — more redundancy, more safety.
Say the honest marginal error rate is , matching what the paper requires. Now instead of independent errors, imagine there’s a shared blind spot , firing with probability : when it fires, all pieces of evidence fail together, because they secretly all leaned on the same flawed assumption or upstream source. Otherwise everything’s fine.
A few things fall out of this pretty directly:
Every individual piece of evidence still has error rate exactly — for every — so nothing about spot-checking any single item distinguishes this from the honest case.
Conditioning on one vote being wrong tells you, with certainty, that all the others are wrong too. That’s about as far as you can get from the pointwise conditional-independence condition the stability proof actually needs.
And the failure probability of the whole gate is just — flat, with no dependence on at all. The honest construction’s error shrinks like ; this one just sits at forever. You can throw arbitrarily more “independent” evidence at it and buy nothing.
If the same blind spot happens to sit underneath evidence at multiple points in the recursion tree — which seems likely whenever one prover, or one family of similar models, is generating everything — a single failure can take out several supposedly-independent subclaims at once rather than needing to get lucky gate by gate.
Is this just a cherry-picked example?
I wondered the same thing, so I checked. Suppose the only thing a judge could realistically ever measure about an evidence source is its marginal error rate and its pairwise error-correlation — already a generous assumption, since actually estimating even that much would be hard in practice. Via the de Finetti representation for exchangeable sequences and the one-sided Chebyshev (Cantelli) inequality, the worst-case failure probability consistent with just those two numbers is
and this bound is tight — achieved, in the limit, by exactly a two-point mixing distribution. So the blind-spot construction above isn’t an arbitrary attack someone made up; it’s the extremal case for whatever correlation actually exists. It’s the ceiling, not a lucky guess.
There’s a cleaner way to see the same thing through variance. For exchangeable correlated binary errors,
which gives an effective sample size
A correlation as small as — nowhere near large enough to show up in a per-item check — caps your effective redundancy at 100 independent-equivalent votes, permanently, whether you’re actually gathering 100 pieces of evidence or ten billion.
Why you can’t just check for this
The marginal statistics are calibrated to match the honest case, so there’s nothing to catch at the individual-item level. The correlation only shows up jointly, and only if you already have a reason to suspect where to look for it. And the adversary picks what to correlate — the defender doesn’t get to choose which axis to audit. This is exactly the gap the original paper names but doesn’t resolve; they say independence “must be established via other means,” and leave open how to do that in general.
A partial defense
If a judge, or some process auditing many past debates with the same prover, could estimate empirically, the Cantelli bound above gives an actual computable ceiling on residual risk, without needing to know what the hidden correlation mechanism even is. A protocol could just refuse to trust a gate’s exponential-suppression guarantee whenever that ceiling comes out too high. It’s not a full fix — it needs history to work from, and a patient adversary could try to build up a clean track record before attacking — but it’s the one part of this that’s actually specified rather than a direction to explore. Beyond that, there’s probably something to borrow from diversity-forcing mechanisms like Estornell & Liu’s diversity-pruning or Pfau & Irving’s entropy penalty for the related exploitable-search problem, or from treating independence itself as something debated recursively rather than assumed — though that last one has an obvious regress problem worth taking seriously rather than hand-waving away.
What I’m not claiming
I’m not saying Theorem 8.3 is wrong — it’s correct given its hypotheses. And I’m not claiming to have spotted a new concern; the paper’s own discussion section and UK AISI’s Alignment Project page both already name the underlying independence-verification problem. What I think is new, as far as I’ve managed to check, is the explicit construction, the quantitative comparison, and the Cantelli argument showing it’s worst-case rather than a cherry-picked example. If any of this turns out to already be handled somewhere I haven’t found, or if there’s a hole in the reasoning, I’d genuinely like to know.
Author Note
I’m 14 and have been studying this area independently. This is my first technical note on debate protocols, so I’d especially appreciate the scrutiny of the probability arguments and pointers to prior work I’ve missed.
Correlated Evidence Can Quietly Break Debate’s Stability Construction
Correlated evidence can quietly break debate’s stability construction
Epistemic status: I’m formalizing a problem the original authors already flagged as open, not claiming to have discovered something out of nowhere. I don’t work on debate protocols and haven’t had this checked by anyone who does — that’s basically why I’m posting it.
Prover-estimator debate (Brown-Cohen, Irving & Piliouras, 2025) gets its completeness guarantee only when the recursive decomposition is stable, and stability is bought through a majority-vote construction (Theorem 8.3) that assumes each subclaim has
I wanted to know what actually happens if that independence fails. It’s worse than I expected, and in a specific way: a prover can hand over evidence that’s indistinguishable from honestly independent evidence at the level of any check a judge could run, while being correlated enough to completely gut the majority vote’s safety margin.
The setup
The relevant piece of the protocol takes
Say the honest marginal error rate is
A few things fall out of this pretty directly:
Every individual piece of evidence still has error rate exactly
Conditioning on one vote being wrong tells you, with certainty, that all the others are wrong too. That’s about as far as you can get from the pointwise conditional-independence condition the stability proof actually needs.
And the failure probability of the whole gate is just
If the same blind spot happens to sit underneath evidence at multiple points in the recursion tree — which seems likely whenever one prover, or one family of similar models, is generating everything — a single failure can take out several supposedly-independent subclaims at once rather than needing to get lucky gate by gate.
Is this just a cherry-picked example?
I wondered the same thing, so I checked. Suppose the only thing a judge could realistically ever measure about an evidence source is its marginal error rate
and this bound is tight — achieved, in the limit, by exactly a two-point mixing distribution. So the blind-spot construction above isn’t an arbitrary attack someone made up; it’s the extremal case for whatever correlation actually exists. It’s the ceiling, not a lucky guess.
There’s a cleaner way to see the same thing through variance. For exchangeable correlated binary errors,
which gives an effective sample size
A correlation as small as
Why you can’t just check for this
The marginal statistics are calibrated to match the honest case, so there’s nothing to catch at the individual-item level. The correlation only shows up jointly, and only if you already have a reason to suspect where to look for it. And the adversary picks what to correlate — the defender doesn’t get to choose which axis to audit. This is exactly the gap the original paper names but doesn’t resolve; they say independence “must be established via other means,” and leave open how to do that in general.
A partial defense
If a judge, or some process auditing many past debates with the same prover, could estimate
What I’m not claiming
I’m not saying Theorem 8.3 is wrong — it’s correct given its hypotheses. And I’m not claiming to have spotted a new concern; the paper’s own discussion section and UK AISI’s Alignment Project page both already name the underlying independence-verification problem. What I think is new, as far as I’ve managed to check, is the explicit construction, the quantitative comparison, and the Cantelli argument showing it’s worst-case rather than a cherry-picked example. If any of this turns out to already be handled somewhere I haven’t found, or if there’s a hole in the reasoning, I’d genuinely like to know.
Author Note
I’m 14 and have been studying this area independently. This is my first technical note on debate protocols, so I’d especially appreciate the scrutiny of the probability arguments and pointers to prior work I’ve missed.