Apparently the rumor is that they are working on test-time training (at 39:25 in the podcast), which I think is the most glaring of the remaining hobbles on LLMs (compared to human cognitive faculties). That is, ability to deeply adapt to a given situation, instead of always needing to figure things out quickly from the position of “first day on the job” (the situation with the current reasoning models), only with notes and work artifacts from your predecessors, but never any job-specific skills (that are specific to this particular source of tasks you are currently dealing with, rather than some broader competence such as “a full-stack programmer”).
If that works out, they plausibly get access or get acquired, since even currently, in 2025, you need to raise $35-45bn to build a frontier AI training system of 2026, otherwise you are at the mercy of one of the handful of existing players. And even if you do raise that, you still need at least a year to build the system. If they only solve the problem in 2026, the bill only goes up, maybe to $100bn.
Of course a working test-time training method is a giant advantage with sufficient value (even without giving a broadly invention-capable AGI) to possibly build even a $500bn training system (2029-2030 frontier AI training compute that probably won’t be built at all otherwise), but look at what happened with OpenAI o1. Ideas leak or get convergently rediscovered by all the competitors, a year of lead time might be too much to actually secure, so even if SSI is first to figure it out, the competitors might still quickly create AIs more capable than SSI does because they have frontier compute, while SSI is still building their own $500bn training system.
Do you think SSI is a player? How much compute do they have relative to the others?
There was a funding round a few months ago where SSI raised $2bn at $32bn valuation.
Apparently the rumor is that they are working on test-time training (at 39:25 in the podcast), which I think is the most glaring of the remaining hobbles on LLMs (compared to human cognitive faculties). That is, ability to deeply adapt to a given situation, instead of always needing to figure things out quickly from the position of “first day on the job” (the situation with the current reasoning models), only with notes and work artifacts from your predecessors, but never any job-specific skills (that are specific to this particular source of tasks you are currently dealing with, rather than some broader competence such as “a full-stack programmer”).
If that works out, they plausibly get access or get acquired, since even currently, in 2025, you need to raise $35-45bn to build a frontier AI training system of 2026, otherwise you are at the mercy of one of the handful of existing players. And even if you do raise that, you still need at least a year to build the system. If they only solve the problem in 2026, the bill only goes up, maybe to $100bn.
Of course a working test-time training method is a giant advantage with sufficient value (even without giving a broadly invention-capable AGI) to possibly build even a $500bn training system (2029-2030 frontier AI training compute that probably won’t be built at all otherwise), but look at what happened with OpenAI o1. Ideas leak or get convergently rediscovered by all the competitors, a year of lead time might be too much to actually secure, so even if SSI is first to figure it out, the competitors might still quickly create AIs more capable than SSI does because they have frontier compute, while SSI is still building their own $500bn training system.