With how everyone seems to be constrained on inference compute and good datasets, there might be less pressure towards making something GPT-5 scaled for now, given uncertainty about its economically relevant capability improvement. So I expect things significantly larger than GPT-4 (with appropriately scaled datasets) to only get deployed in ~early 2025 (65% for no publicly available GPT-5 equivalent in this sense before 2025), after enough new AI datacenters get built and enough datasets are prepared. See also this market on GPT-5 specifically.
If this holds, the next two years see everyone catching up to GPT-4 and doing less essential stuff like multimodality and productization of GPT-4 level capabilities. Less visibly, there is work towards GPT-5 scale runs or on fine-tuning such models in-house before deployment. There are no new fire alarms, other than gradually building economic impact. It’s not unlikely algorithmic improvement breaks this, but in a way that’s not predictable in detail right now. Or alternatively first GPT-5 level runs show enough capability improvement to accelerate the race, making inference costs less relevant.
With how everyone seems to be constrained on inference compute and good datasets, there might be less pressure towards making something GPT-5 scaled for now, given uncertainty about its economically relevant capability improvement. So I expect things significantly larger than GPT-4 (with appropriately scaled datasets) to only get deployed in ~early 2025 (65% for no publicly available GPT-5 equivalent in this sense before 2025), after enough new AI datacenters get built and enough datasets are prepared. See also this market on GPT-5 specifically.
If this holds, the next two years see everyone catching up to GPT-4 and doing less essential stuff like multimodality and productization of GPT-4 level capabilities. Less visibly, there is work towards GPT-5 scale runs or on fine-tuning such models in-house before deployment. There are no new fire alarms, other than gradually building economic impact. It’s not unlikely algorithmic improvement breaks this, but in a way that’s not predictable in detail right now. Or alternatively first GPT-5 level runs show enough capability improvement to accelerate the race, making inference costs less relevant.