If I put on the “we need empirical feedback from neural nets to make progress on alignment” hat, along with my “prudence” hat, I’m thinking things more like, “okay let’s stop scaling now, and just work really hard on figuring out how exactly capabilities emerged between e.g., GPT-3 and GPT-4. Like, what exactly can we predict about GPT-4 based on GPT-3? Can we break down surprising and abrupt less-scary capabilities into understandable parts, and generalize from that to more-scary capabilities?” Basically, I’m hoping for a bunch more proof of concept that Anthropic is capable of understanding and controlling current systems, before they scale blindly. If they can’t do it now, why should I expect they’ll be able to do it then?
Looking back in 2026, this strikes me as a sad missed opportunity. We’re at a point where the next capabilities jump will probably not get us anything as easily describable as “gain the ability to multiply 3-digit numbers”. Instead it will be something fuzzy and hard to operationalize.
There was a window where we could do empirical work in the genre of “figure out how exactly model capabilities improve with successive generations”, but that work is much harder to do now—it might not even be possible at all.
Looking back in 2026, this strikes me as a sad missed opportunity. We’re at a point where the next capabilities jump will probably not get us anything as easily describable as “gain the ability to multiply 3-digit numbers”. Instead it will be something fuzzy and hard to operationalize.
There was a window where we could do empirical work in the genre of “figure out how exactly model capabilities improve with successive generations”, but that work is much harder to do now—it might not even be possible at all.