Maybe 2 years, in 2028? Active params are cost per token in chip-hours, and 100B active params of Kimi K3 are 10x less than about 1T active params of Mythos 5 (both being the higher end models in their regions). Thus to serve as many tokens per year, China needs 10x less datacenter capacity than the US, which fits the estimates of what’s currently available. This is ignoring the likely technical impossibility of serving a 1T active 10T total param model on H100s (or similar) at a reasonable time per output token.
A better anchor is maybe 2x smaller than Mythos 5, since Mythos 5 is likely compute optimally trained, while Opus 5 (not Opus 4.5+) might be 4x overtrained and 2x smaller (500B-750B active params), that is the D/N ratio is off by 4x compared to compute optimal, and it uses the same amount of pretraining compute as Mythos 5, but with 2x fewer active params. (It’s not completely clear if Opus 5 could be similarly dangerous, but probably; on the other hand, Sonnet doesn’t seem to be in the ballpark, and 100B active params is probably less than Sonnet.) Compared to today’s models in China, serving the same number of tokens per year with the 500B-750B active param models needs 5-10x more compute to be available, even if somehow decode can still run on scale-out networks (expert parallelism rather than pipeline parallelism across many scale-up systems) at a tolerable speed (which gets less plausible for the models with more active params; using more total params doesn’t matter for this, feasibility of even 10T total param models with 100B active params won’t be evidence against the difficulty with more active params). Getting 5-10x more compute is not something that happens in 1 year.
Pretraining doesn’t need big scale-up systems, and it’s plausible RL training could somehow happen on the sufficiently big and thus fast-for-a-big-model Oberon racks. But setting up inference despite access restrictions seems less plausible, and the 5-10x greater cost in chip-hours makes it infeasible to use big models even if a smaller amount of big scale-up systems becomes available (such as if Huawei makes progress with Atlas). So a Mythos-level model can probably be trained in China (especially with some circumvention of access restrictions to bigger scale-up systems for RL training), but it can’t be usefully served for some time yet, and so it probably won’t be trained. Hence maybe 2 years.
Maybe 2 years, in 2028? Active params are cost per token in chip-hours, and 100B active params of Kimi K3 are 10x less than about 1T active params of Mythos 5 (both being the higher end models in their regions). Thus to serve as many tokens per year, China needs 10x less datacenter capacity than the US, which fits the estimates of what’s currently available. This is ignoring the likely technical impossibility of serving a 1T active 10T total param model on H100s (or similar) at a reasonable time per output token.
A better anchor is maybe 2x smaller than Mythos 5, since Mythos 5 is likely compute optimally trained, while Opus 5 (not Opus 4.5+) might be 4x overtrained and 2x smaller (500B-750B active params), that is the D/N ratio is off by 4x compared to compute optimal, and it uses the same amount of pretraining compute as Mythos 5, but with 2x fewer active params. (It’s not completely clear if Opus 5 could be similarly dangerous, but probably; on the other hand, Sonnet doesn’t seem to be in the ballpark, and 100B active params is probably less than Sonnet.) Compared to today’s models in China, serving the same number of tokens per year with the 500B-750B active param models needs 5-10x more compute to be available, even if somehow decode can still run on scale-out networks (expert parallelism rather than pipeline parallelism across many scale-up systems) at a tolerable speed (which gets less plausible for the models with more active params; using more total params doesn’t matter for this, feasibility of even 10T total param models with 100B active params won’t be evidence against the difficulty with more active params). Getting 5-10x more compute is not something that happens in 1 year.
Pretraining doesn’t need big scale-up systems, and it’s plausible RL training could somehow happen on the sufficiently big and thus fast-for-a-big-model Oberon racks. But setting up inference despite access restrictions seems less plausible, and the 5-10x greater cost in chip-hours makes it infeasible to use big models even if a smaller amount of big scale-up systems becomes available (such as if Huawei makes progress with Atlas). So a Mythos-level model can probably be trained in China (especially with some circumvention of access restrictions to bigger scale-up systems for RL training), but it can’t be usefully served for some time yet, and so it probably won’t be trained. Hence maybe 2 years.