The specific thing I think wouldn’t work is trying to start the process without a bunch of pretraining data for at least the initial judge (i.e. pure self play from a randomized initialization with no human-generated data or judgments enteringthetraining the training run at any point). Not super insightful I know, just addressing what I meant by “zero” in my hypothetical ChatGPT-Zero.
The specific thing I think wouldn’t work is trying to start the process without a bunch of pretraining data for at least the initial judge (i.e. pure self play from a randomized initialization with no human-generated data or judgments enteringthetraining the training run at any point). Not super insightful I know, just addressing what I meant by “zero” in my hypothetical ChatGPT-Zero.
Thanks for clarifying! I do agree that that wouldn’t work, at least if we wanted what was produced to be in any way useful or meaningful to humans.