if you’re Xi Jinping and you’re scaling pilled, you can just centralize all the compute and build all the substations for it. You can just hide it inside one of the factories you already have that’s drawing power for steel production and re-purpose it as a data center.
Given the success we’ve seen in training SOTA models with constrained GPU resources (Deepseek), I don’t think it’s far fetched to think you can hide bleeding edge development. It turns out all you need is a few hundred of the smartest people in your country and a few thousand GPUs.
Hrm...sounds like the size of the Manhattan Project.
re: 1b (development likely impossible to kept secret given scale required)
I’m remind of Dylan Patel’s comments (semianalysis) on a recent episode of the Dwarkesh Podcast which goes something like:
Given the success we’ve seen in training SOTA models with constrained GPU resources (Deepseek), I don’t think it’s far fetched to think you can hide bleeding edge development. It turns out all you need is a few hundred of the smartest people in your country and a few thousand GPUs.
Hrm...sounds like the size of the Manhattan Project.