So it’s still not clear to me how much they delayed bc they had to, versus how much (if at all) they did due to the forecasters and/or acceleration considerations.
Yeah, completely agree.
I think “finished training” is the next-token prediction pre-training, and what they did since August is the fine-tuning and the RLHF + other stuff.
This seems most likely? But if so, I wish openai had used a different phrase, fine-tuning/RLHF/other stuff is also part of training (unless I’m badly mistaken), and we have this lovely phrase “pre-training” that they could have used instead.
Yeah, completely agree.
This seems most likely? But if so, I wish openai had used a different phrase, fine-tuning/RLHF/other stuff is also part of training (unless I’m badly mistaken), and we have this lovely phrase “pre-training” that they could have used instead.
Ah yeah, that does seem needlessly ambiguous.