It might be worth making a choice about a single move which is unclear to weak players but where strong players have a consensus.
Mostly I think it would be faster and I think a lot less noisy per minute. I also think it’s a bit unrepresentative to be able to use “how well did this advisor’s suggestions work out in hindsight?” to learn which advisors are honest and so it’s nice to make the dishonest advisors’ job easier.
(In practice I think evaluating what worked well in hindsight is going to be very valuable, and is already enough for crazy research acceleration—e.g. it would be very valuable to just get predictions of which research direction will feel promising to me after spending a day thinking about it. But I think the main open question here is whether some kind of debate or decomposition can add value over and above the obvious big wins.)
For what it’s worth I think using chess might be kind of tough—if you provide significant time, the debaters can basically just play out the game.
It might be worth making a choice about a single move which is unclear to weak players but where strong players have a consensus.
Mostly I think it would be faster and I think a lot less noisy per minute. I also think it’s a bit unrepresentative to be able to use “how well did this advisor’s suggestions work out in hindsight?” to learn which advisors are honest and so it’s nice to make the dishonest advisors’ job easier.
(In practice I think evaluating what worked well in hindsight is going to be very valuable, and is already enough for crazy research acceleration—e.g. it would be very valuable to just get predictions of which research direction will feel promising to me after spending a day thinking about it. But I think the main open question here is whether some kind of debate or decomposition can add value over and above the obvious big wins.)
For what it’s worth I think using chess might be kind of tough—if you provide significant time, the debaters can basically just play out the game.