The human brain should be seen as evidence that a certain scale is ~sufficient, but not that it is necessary.
The human brain is often estimated to have 10^14 synapses, which would be a 100T model, give or take. Except that individual neurons also have a bunch of internal parameters, which might complicate things.
If you told me that the human brain was massively inefficient, and that you had squeezed human level AGI into 1T parameters, I would be only mildly surprised.
For that matter, if you told me you had squeezed a weak AGI into 30B parameters, I’d be interested in the claim. Qwen3 really is surprisingly capable in that size range. If you told me 4B, I’d be very skeptical, but then again, Gemma 3n does implausibly well on my diverse private benchmarks, and it’s technically multi-modal. At the very least, I’d accept it as the premise of a science fiction horror story about tiny, unaligned AIs.
But if we drop all the way to 30 million parameters, I am profoundly suspicious of any kind of general model with language skills and reasonable world knowledge. Even if you store language and world knowledge as compressed text files, you’re going to his some pretty hard limits at that size. That’s a 60MB ZIP file or less. You’d be taking about needing only 1⁄3,000,000th of parameters of the brain. Which is a lot of orders of magnitude.
At that size, I’m assuming that any kind of genuinely interesting model would be something like AlphaGo, that demonstrates impressive knowledge and learning abilities in a very narrow domain. Which is fine! It might even be the final warning that AGI is inevitable. But I would still expect more than 6 months would be required to scale back up from such a tiny model to something with general world knowledge, language and common sense.
The human brain is often estimated to have 10^14 synapses, which would be a 100T model, give or take. Except that individual neurons also have a bunch of internal parameters, which might complicate things.
If you told me that the human brain was massively inefficient, and that you had squeezed human level AGI into 1T parameters, I would be only mildly surprised.
For that matter, if you told me you had squeezed a weak AGI into 30B parameters, I’d be interested in the claim. Qwen3 really is surprisingly capable in that size range. If you told me 4B, I’d be very skeptical, but then again, Gemma 3n does implausibly well on my diverse private benchmarks, and it’s technically multi-modal. At the very least, I’d accept it as the premise of a science fiction horror story about tiny, unaligned AIs.
But if we drop all the way to 30 million parameters, I am profoundly suspicious of any kind of general model with language skills and reasonable world knowledge. Even if you store language and world knowledge as compressed text files, you’re going to his some pretty hard limits at that size. That’s a 60MB ZIP file or less. You’d be taking about needing only 1⁄3,000,000th of parameters of the brain. Which is a lot of orders of magnitude.
At that size, I’m assuming that any kind of genuinely interesting model would be something like AlphaGo, that demonstrates impressive knowledge and learning abilities in a very narrow domain. Which is fine! It might even be the final warning that AGI is inevitable. But I would still expect more than 6 months would be required to scale back up from such a tiny model to something with general world knowledge, language and common sense.