Is that the correct way to model this, though? My current impression is that only OpenAI (and maybe Anthropic) are actually pushing the frontier in a qualitative way, whereas everyone else is just copying them. (Test-time compute, the current push for agents, scaling LLMs to begin with...)
That process of copying/reverse-engineering is indeed very fast. But I’m not convinced that if OpenAI decided to sit still doing nothing, or stopped disclosing its advancements publicly, the other labs’ progress wouldn’t stagnate around incremental improvements to OpenAI’s latest paradigm. Likely by diffusing into a thousand abortive attempts to make qualitative progress that mostly sum up to nothing, the way the open source community has mostly been.
Like, I can’t help but notice that prior to o1, nobody seems to have been going full-tilt on reasoning models. “RL on CoTs” was an obvious idea everyone discussed since 2022, and of course everyone is now claiming to have been working on stuff like this for a while… But no-one seem to have actually implemented it well prior to OpenAI.
Once OpenAI showed that it’s possible and that there’s gold there, obviously everyone and their grandmother coordinated around reverse-engineering that. But if OpenAI’s Strawberry spontaneously caught fire, would the other labs have actually gotten there on their own?
Eventually, sure. But would it have taken 6 months, or more like 12-24?
Unclear, I think. (Sources with counter-evidence welcome.)
I’m skeptical to which extent the latter can be done. That’s like saying an AI lab should suddenly care about AI safety. One can’t really bolt a security mandate onto an existing institution and expect a competent result.
Is that the correct way to model this, though? My current impression is that only OpenAI (and maybe Anthropic) are actually pushing the frontier in a qualitative way, whereas everyone else is just copying them. (Test-time compute, the current push for agents, scaling LLMs to begin with...)
That process of copying/reverse-engineering is indeed very fast. But I’m not convinced that if OpenAI decided to sit still doing nothing, or stopped disclosing its advancements publicly, the other labs’ progress wouldn’t stagnate around incremental improvements to OpenAI’s latest paradigm. Likely by diffusing into a thousand abortive attempts to make qualitative progress that mostly sum up to nothing, the way the open source community has mostly been.
Like, I can’t help but notice that prior to o1, nobody seems to have been going full-tilt on reasoning models. “RL on CoTs” was an obvious idea everyone discussed since 2022, and of course everyone is now claiming to have been working on stuff like this for a while… But no-one seem to have actually implemented it well prior to OpenAI.
Once OpenAI showed that it’s possible and that there’s gold there, obviously everyone and their grandmother coordinated around reverse-engineering that. But if OpenAI’s Strawberry spontaneously caught fire, would the other labs have actually gotten there on their own?
Eventually, sure. But would it have taken 6 months, or more like 12-24?
Unclear, I think. (Sources with counter-evidence welcome.)
Does this matter all that much, given lack of opsec, relationships between or poaching of employees of other labs, corporate espionage, etc.?
That’s a valid point, yes. But shoddy opsec doesn’t mean no opsec.
Or replace “stopped disclosing its advancements” with “started caring about opsect related to its most important projects”.
I’m skeptical to which extent the latter can be done. That’s like saying an AI lab should suddenly care about AI safety. One can’t really bolt a security mandate onto an existing institution and expect a competent result.