If pre-training was the main reason you believed AGI was close—and now you believe pre-training has stalled—then you should update pretty strongly away from short timelines.
I’m not sure if this is true. Pretraining doesn’t need to scale indefinitely in order to reach AGI, it just needs to scale to the point where the base models + RL are capable enough to automate ML research.
Good point, and I think I somewhat agree. If you think we just reach an intelligence explosion at some level (seems pretty plausible), you wouldn’t update to previous pre-training levels because we’d be closer and what really matters is hitting that point (and post-training can possibly take you to that point). While it means that you shouldn’t update towards before pre-training, I still think the general point of being a large update back still stands (perhaps this point—the degree—depends on some other priors, though, which I didn’t want to get into).
I’m not sure if this is true. Pretraining doesn’t need to scale indefinitely in order to reach AGI, it just needs to scale to the point where the base models + RL are capable enough to automate ML research.
Good point, and I think I somewhat agree. If you think we just reach an intelligence explosion at some level (seems pretty plausible), you wouldn’t update to previous pre-training levels because we’d be closer and what really matters is hitting that point (and post-training can possibly take you to that point). While it means that you shouldn’t update towards before pre-training, I still think the general point of being a large update back still stands (perhaps this point—the degree—depends on some other priors, though, which I didn’t want to get into).