This would be totally insane if they mean literally autonomously. 5.6 Luna is something like a $10 billion product, so even at $1k per human hour it would be worth spending 50 engineer-years to improve its quality by 1%. I would predict that well over 50 human engineer-years went into 5.6 Luna in total, and lots of this into post-training. Maybe 5.6 Sol was monitoring posttraining using a pipeline that needed tens of engineer years to work out all the issues in.
This assumes that economic outcomes are linearly correlated with quality, but historically that hasn’t really been the case. Especially not for something that the market is currently unable to meaningfully evaluate on the grounds of quality.
Getting 1% better on a cheap model doesn’t necessarily correlate to any tangible revenue for OpenAI; the economics of consumer behavior operates in step changes and vibes.
This would be totally insane if they mean literally autonomously. 5.6 Luna is something like a $10 billion product, so even at $1k per human hour it would be worth spending 50 engineer-years to improve its quality by 1%. I would predict that well over 50 human engineer-years went into 5.6 Luna in total, and lots of this into post-training. Maybe 5.6 Sol was monitoring posttraining using a pipeline that needed tens of engineer years to work out all the issues in.
This assumes that economic outcomes are linearly correlated with quality, but historically that hasn’t really been the case. Especially not for something that the market is currently unable to meaningfully evaluate on the grounds of quality.
Getting 1% better on a cheap model doesn’t necessarily correlate to any tangible revenue for OpenAI; the economics of consumer behavior operates in step changes and vibes.