The brain contains roughly 10^14 synapses. Assuming the strength of a synapse contains at least 4 bits of information, and that there’s at least 4 bits of information per synapse in the choice of which nearby cells this cell has synapse connections versus which it don’t, that’s roughly 10^14 bytes of information. The human genome is roughly 10^9 bytes of information [Dwarkesh’s numbers are wrong, 3.1 billion base pairs is less than 1GB, not 3GB: a base-pair is two bits, a byte is eight bits], not all of which is devoted to the brain. In fact, since the difference between the chimp and human genomes is only about 2%, while the human brain is roughly 4 times as large as the chimp one, it would seem that probably, at least for the neocortex rather then the brainstem (since the neocortex is where the differences between chimps and humans are concentrated: our brainstems do fairly similar things), probably pretty small proportion of the genome is devoted to that part of the brain. So at least 99.999% of the information in the human brain comes from its environment or is built up internally during development rather than being directly hard-coded by the genome.
Nevertheless, the genome clearly gives a very developmentally useful jump-start and set of priors (like “here’s a good start on several useful ways to do depth perception”, and “pay attention to anything that looks roughly like a primate face”). But likely it’s focused on what to learn and how to learn it, rather than providing bulk of the actual information learnt: there simply isn’t space for the latter, by many orders of magnitude. And in AI terms, the genome has to be worth less than a well-chosen 1B parameters.
Thanks for the numbers, I completely agree on the information counting. I was trying to make (and failed) a slightly different point. Let me try to put it in more precise language.
the genome clearly gives a very developmentally useful jump-start and set of priors
This is the main sentiment I was trying to push against. It is not that I think DNA provides the bulk of the information. It is just that it doesn’t start from nothing, create a few priors and then hand things off. Imagine some basic brain structure + sensory inputs encoded by some simple set of genes. When facing the real world, this brain will reliably be “filled” (i.e., learn lots of information). But, DNA doesn’t have to only encode priors or loss functions for this learning process. It can also encode transformations to perform on the completed thing. For example, you could have things like “copy structure from region X to region Y”, “extract half of circuit X and combine it with circuit Y to create a new circuit Z”. This still does not mean that the minute details are in the DNA itself. But, I think you can build practically anything you want with tricks like that, because many specific learned circuits will contain general-purpose reusable subnetworks.
My main point is that evolution isn’t limited to designs like “create a starting point with good priors and let go”. It can encode things that start with a completed brain, and then copy/paste/merge/shuffle parts around to create something new. Given that most algorithms you might want to encode likely can be expressed as parts of already existing learned ones, a complicated design can fit in very few bits. Just like you could write a complicated program with very few lines of code in a high-level language.
The brain contains roughly 10^14 synapses. Assuming the strength of a synapse contains at least 4 bits of information, and that there’s at least 4 bits of information per synapse in the choice of which nearby cells this cell has synapse connections versus which it don’t, that’s roughly 10^14 bytes of information. The human genome is roughly 10^9 bytes of information [Dwarkesh’s numbers are wrong, 3.1 billion base pairs is less than 1GB, not 3GB: a base-pair is two bits, a byte is eight bits], not all of which is devoted to the brain. In fact, since the difference between the chimp and human genomes is only about 2%, while the human brain is roughly 4 times as large as the chimp one, it would seem that probably, at least for the neocortex rather then the brainstem (since the neocortex is where the differences between chimps and humans are concentrated: our brainstems do fairly similar things), probably pretty small proportion of the genome is devoted to that part of the brain. So at least 99.999% of the information in the human brain comes from its environment or is built up internally during development rather than being directly hard-coded by the genome.
Nevertheless, the genome clearly gives a very developmentally useful jump-start and set of priors (like “here’s a good start on several useful ways to do depth perception”, and “pay attention to anything that looks roughly like a primate face”). But likely it’s focused on what to learn and how to learn it, rather than providing bulk of the actual information learnt: there simply isn’t space for the latter, by many orders of magnitude. And in AI terms, the genome has to be worth less than a well-chosen 1B parameters.
Thanks for the numbers, I completely agree on the information counting. I was trying to make (and failed) a slightly different point. Let me try to put it in more precise language.
This is the main sentiment I was trying to push against. It is not that I think DNA provides the bulk of the information. It is just that it doesn’t start from nothing, create a few priors and then hand things off.
Imagine some basic brain structure + sensory inputs encoded by some simple set of genes. When facing the real world, this brain will reliably be “filled” (i.e., learn lots of information). But, DNA doesn’t have to only encode priors or loss functions for this learning process. It can also encode transformations to perform on the completed thing. For example, you could have things like “copy structure from region X to region Y”, “extract half of circuit X and combine it with circuit Y to create a new circuit Z”.
This still does not mean that the minute details are in the DNA itself. But, I think you can build practically anything you want with tricks like that, because many specific learned circuits will contain general-purpose reusable subnetworks.
My main point is that evolution isn’t limited to designs like “create a starting point with good priors and let go”. It can encode things that start with a completed brain, and then copy/paste/merge/shuffle parts around to create something new. Given that most algorithms you might want to encode likely can be expressed as parts of already existing learned ones, a complicated design can fit in very few bits. Just like you could write a complicated program with very few lines of code in a high-level language.