With computationally unbounded debaters, debate with optimal play (and cross-examination) can answer any question in NEXP given polynomial time judges.
The crux being:
Interesting questions about the world can be formalised as problems with deterministic solutions to be resolved via an iterated proof process.
Yet consider that most “weighty” problems safe AI version N will have to resolve may be of the form:
Forecast the impact of policy X on the world. Is the impact good or bad?
Where X can be “deploy safe AI system version N+1” or “implement governance structure Y” or “release commercial technology patent Z”
In these cases the solution requires some simulation of a computationally irreducibly complex system (the world, which includes safe AI version N). Even assuming no deception, it is very hard to see how the debate process can in anyway approximate a proof of the validity of these propositions, in a mathematical sense.
I believe that this is the crux of debate:
The crux being:
Yet consider that most “weighty” problems safe AI version N will have to resolve may be of the form:
Where X can be “deploy safe AI system version N+1” or “implement governance structure Y” or “release commercial technology patent Z”
In these cases the solution requires some simulation of a computationally irreducibly complex system (the world, which includes safe AI version N). Even assuming no deception, it is very hard to see how the debate process can in anyway approximate a proof of the validity of these propositions, in a mathematical sense.