(01:15:14): It is not completely clear at this moment whether that’s going to be useful. So there have been some studies where if you train models on regurgitated model outputs, there’s this phenomenon called “model collapse”, in which the quality of the model ends up degrading. And we don’t have a really good understanding of that, not just yet. But again, this is in a sense the early days of dealing with this problem, because it hasn’t been that big of an issue yet. Once it becomes [one], I expect the forces that be to push really hard to figure out how do we go past this?
The issue is that the model collapse paper makes such unrealistic assumptions such that I would not trust it as any sort of guide to future progress, like the idea that you will throw all real data away (AI companies obviously wouldn’t do this).
And that’s just one unrealistic assumption on a stack of unrealistic assumptions, which the tweet described below:
The issue is that the model collapse paper makes such unrealistic assumptions such that I would not trust it as any sort of guide to future progress, like the idea that you will throw all real data away (AI companies obviously wouldn’t do this).
And that’s just one unrealistic assumption on a stack of unrealistic assumptions, which the tweet described below:
https://x.com/RylanSchaeffer/status/1816881533795422404