Vaguely feeling like OpenAI might be moving away from GPT-N+1 release model, for some combination of “political/frog-boiling” reasons and “scaling actually hitting a wall” reasons. Seems relevant to note, since in the worlds where they hadn’t been drip-feeding people incremental releases of slight improvements over the original GPT-4 capabilities, and instead just dropped GPT-5 (and it was as much of an improvement over 4 as 4 was over 3, or close), that might have prompted people to do an explicit orientation step. As it is, I expect less of that kind of orientation to happen. (Though maybe I’m speaking too soon and they will drop GPT-5 on us at some point, and it’ll still manage to be a step-function improvement over whatever the latest GPT-4* model is at that point.)
Eh, I think they’ll drop GPT-4.5/5 at some point. It’s just relatively natural for them to incrementally improve their existing model to ensure that users aren’t tempted to switch to competitors.
It also allows them to avoid people being underwhelmed.
I would wait another year or so before getting much evidence on “scaling actually hitting a wall” (or until we have models that are known to have training runs with >30x GPT-4 effective compute), training and deploying massive models isn’t that fast.
Yeah, I agree that it’s too early to call it re: hitting a wall. I also just realized that releasing 4o for free might be some evidence in favor of 4.5/5 dropping soon-ish.
Yeah. This prompts me to make a brief version of a post I’d had on my TODO list for awhile:
“In the 21st century, being quick and competent at ‘orienting’ is one of the most important skills.”
(in the OODA Loop sense, i.e. observe → orient → decide → act)
We don’t know exactly what’s coming with AI or other technologies, we can make plans informed by our best-guesses, but we should be on the lookout for things that should prompt some kind of strategic orientation. @jacobjacob has helped prioritize noticing things like “LLMs are pretty soon going to be affect the strategic landscape, we should be ready to take advantage of the technology and/or respond to a world where other people are doing that.”
I like Robert’s comment here because it feels skillful at noticing a subtle thing that is happening, and promoting it to strategic attention. The object-level observation seems important and I hope people in the AI landscape get good at this sort of noticing.
It also feels kinda related to the original context of OODA-looping, which was about fighter pilots dogfighting. One of the skills was “get inside of the enemy’s OODA loop and disrupt their ability to orient.” If this were intentional on OpenAI’s part (or part of subconscious strategy), it’d be a kinda clever attempt to disrupt our observation step.
Sam Altman and OpenAI have both said they are aiming for incremental releases/deployment for the primary purpose of allowing society to prepare and adapt. Opposed to, say, dropping large capabilities jumps out of the blue which surprise people.
I think “They believe incremental release is safer because it promotes societal preparation” should certainly be in the hypothesis space for the reasons behind these actions, along with scaling slowing and frog-boiling. My guess is that it is more likely than both of those reasons (they have stated it as their reasoning multiple times; I don’t think scaling is hitting a wall).
Yeah, “they’re following their stated release strategy for the reasons they said motivated that strategy” also seems likely to share some responsibility. (I might not think those reasons justify that release strategy, but that’s a different argument.)
I wonder if that is actually a sound view though. I just started reading Like War (interesting and seems correct/on target so far but really just starting it). Given the subject area of impact, reaction and use of social media and networking technologies and the general results socially, seems like society generally is not really even yet prepared and adapted for that inovation. If all the fears about AI are even close to getting things right I suspect the “allowing society to prepare and adapt” suggests putting everything on hold, freezing in place, for at least a decade and probably longer.
Altman’s and OpenAI’s intentions might be towards that stated goal but I think they are basing that approach on how “the smartest people in the room” react to AI and not the general public, or the most opportinistic people in the room.
I’m not sure if you’d categorize this under “scaling actually hitting a wall” but the main possibility that feels relevant in my mind is that progress simply is incremental in this case, as a fact about the world, rather than being a strategic choice on behalf of OpenAI. When underlying progress is itself incremental, it makes sense to release frequent small updates. This is common in the software industry, and would not at all be surprising if what’s often true for most software development holds for OpenAI as well.
(Though I also expect GPT-5 to be medium-sized jump, once it comes out.)
Vaguely feeling like OpenAI might be moving away from GPT-N+1 release model, for some combination of “political/frog-boiling” reasons and “scaling actually hitting a wall” reasons. Seems relevant to note, since in the worlds where they hadn’t been drip-feeding people incremental releases of slight improvements over the original GPT-4 capabilities, and instead just dropped GPT-5 (and it was as much of an improvement over 4 as 4 was over 3, or close), that might have prompted people to do an explicit orientation step. As it is, I expect less of that kind of orientation to happen. (Though maybe I’m speaking too soon and they will drop GPT-5 on us at some point, and it’ll still manage to be a step-function improvement over whatever the latest GPT-4* model is at that point.)
Eh, I think they’ll drop GPT-4.5/5 at some point. It’s just relatively natural for them to incrementally improve their existing model to ensure that users aren’t tempted to switch to competitors.
It also allows them to avoid people being underwhelmed.
I would wait another year or so before getting much evidence on “scaling actually hitting a wall” (or until we have models that are known to have training runs with >30x GPT-4 effective compute), training and deploying massive models isn’t that fast.
Yeah, I agree that it’s too early to call it re: hitting a wall. I also just realized that releasing 4o for free might be some evidence in favor of 4.5/5 dropping soon-ish.
Yeah. This prompts me to make a brief version of a post I’d had on my TODO list for awhile:
“In the 21st century, being quick and competent at ‘orienting’ is one of the most important skills.”
(in the OODA Loop sense, i.e. observe → orient → decide → act)
We don’t know exactly what’s coming with AI or other technologies, we can make plans informed by our best-guesses, but we should be on the lookout for things that should prompt some kind of strategic orientation. @jacobjacob has helped prioritize noticing things like “LLMs are pretty soon going to be affect the strategic landscape, we should be ready to take advantage of the technology and/or respond to a world where other people are doing that.”
I like Robert’s comment here because it feels skillful at noticing a subtle thing that is happening, and promoting it to strategic attention. The object-level observation seems important and I hope people in the AI landscape get good at this sort of noticing.
It also feels kinda related to the original context of OODA-looping, which was about fighter pilots dogfighting. One of the skills was “get inside of the enemy’s OODA loop and disrupt their ability to orient.” If this were intentional on OpenAI’s part (or part of subconscious strategy), it’d be a kinda clever attempt to disrupt our observation step.
Sam Altman and OpenAI have both said they are aiming for incremental releases/deployment for the primary purpose of allowing society to prepare and adapt. Opposed to, say, dropping large capabilities jumps out of the blue which surprise people.
I think “They believe incremental release is safer because it promotes societal preparation” should certainly be in the hypothesis space for the reasons behind these actions, along with scaling slowing and frog-boiling. My guess is that it is more likely than both of those reasons (they have stated it as their reasoning multiple times; I don’t think scaling is hitting a wall).
Yeah, “they’re following their stated release strategy for the reasons they said motivated that strategy” also seems likely to share some responsibility. (I might not think those reasons justify that release strategy, but that’s a different argument.)
I wonder if that is actually a sound view though. I just started reading Like War (interesting and seems correct/on target so far but really just starting it). Given the subject area of impact, reaction and use of social media and networking technologies and the general results socially, seems like society generally is not really even yet prepared and adapted for that inovation. If all the fears about AI are even close to getting things right I suspect the “allowing society to prepare and adapt” suggests putting everything on hold, freezing in place, for at least a decade and probably longer.
Altman’s and OpenAI’s intentions might be towards that stated goal but I think they are basing that approach on how “the smartest people in the room” react to AI and not the general public, or the most opportinistic people in the room.
I’m not sure if you’d categorize this under “scaling actually hitting a wall” but the main possibility that feels relevant in my mind is that progress simply is incremental in this case, as a fact about the world, rather than being a strategic choice on behalf of OpenAI. When underlying progress is itself incremental, it makes sense to release frequent small updates. This is common in the software industry, and would not at all be surprising if what’s often true for most software development holds for OpenAI as well.
(Though I also expect GPT-5 to be medium-sized jump, once it comes out.)