The recent trend is towards shorter lag times between OAI et al. performance and Chinese competitors.
Just today, Deepseek claimed to match O1-preview performance—that is a two month delay.
I do not know about CCP intent, and I don’t know on what basis the authors of this report base their claims, but “China is racing towards AGI … It’s critical that we take them extremely seriously” strikes me as a fair summary of the recent trend in model quality and model quantity from Chinese companies (Deepseek, Qwen, Yi, Stepfun, etc.)
I recommend lmarena.ai s leaderboard tab as a one-stop-shop overview of the state of AI competition.
Claiming that China as a country is racing toward AGI != Chinese AI companies aren’t fast following US AI companies, which are explicitly trying to build AGI. This is a big distinction!
Chinese companies explicitly have a rule not to release things that are ahead of SOTA (I’ve seen comments of the form “trying to convince my boss this isn’t SOTA so we can release it” on github repos). So “publicly release Chinese models are always slightly behind American ones” doesn’t prove much.
Benchmarks for o1 were included in the o1/o1-preview announcement, and you could eyeball the jumps as roughly equal for 4o → o1-preview → o1. (Another way to put it: the o1-preview you have access to has only half the total gain.) So if you only match o1-preview at its announcement, you are far behind o1 back then, and further behind now.
The recent trend is towards shorter lag times between OAI et al. performance and Chinese competitors.
Just today, Deepseek claimed to match O1-preview performance—that is a two month delay.
I do not know about CCP intent, and I don’t know on what basis the authors of this report base their claims, but “China is racing towards AGI … It’s critical that we take them extremely seriously” strikes me as a fair summary of the recent trend in model quality and model quantity from Chinese companies (Deepseek, Qwen, Yi, Stepfun, etc.)
I recommend lmarena.ai s leaderboard tab as a one-stop-shop overview of the state of AI competition.
Claiming that China as a country is racing toward AGI != Chinese AI companies aren’t fast following US AI companies, which are explicitly trying to build AGI. This is a big distinction!
Chinese companies explicitly have a rule not to release things that are ahead of SOTA (I’ve seen comments of the form “trying to convince my boss this isn’t SOTA so we can release it” on github repos). So “publicly release Chinese models are always slightly behind American ones” doesn’t prove much.
Interesting, do you have a link for that?
US companies are racing toward AGI but the USG isn’t. As someone else mentioned, Dylan Patel from Semianalysis does not think China is scale-pilled.
Why is that comparison not to the much better GPT-4 o1 then, or the doubtless better o1 now?
I’m not sure the o1 model has even been benchmarked yet, let alone used, so that’s why they are focused on o1 preview.
Benchmarks for o1 were included in the o1/o1-preview announcement, and you could eyeball the jumps as roughly equal for 4o → o1-preview → o1. (Another way to put it: the o1-preview you have access to has only half the total gain.) So if you only match o1-preview at its announcement, you are far behind o1 back then, and further behind now.