Thank you for this excellent post. Here are some thoughts I had while reading.
The hard paths hypothesis:
I think there’s another side to the hard paths hypothesis. We are clearly the first technology-using species to evolve on Earth. However, it’s entirely possible that we’re not the first species with human-level intelligence. If a species with human level intelligence but no opposable thumbs evolved millions of years ago, they could have died out without leaving any artifacts we’d recognize as signs of intelligence.
Besides our intelligence, humans seem odd in many ways that could plausibly contribute to developing a technological civilization.
We are pretty long-lived.
We are fairly social.
Feral children raised outside of human culture experience serious and often permanent mental disabilities (Wikipedia).
A species with human-level intelligence, but whose members live mostly independently may not develop technological civilization.
We have very long childhoods.
We have ridiculously high manual dexterity (even compared to other primates).
We live on land.
Most animals are aquatic.
It’s hard to have an industrial revolution when you can’t burn things.
Given how well-tuned our biology seems for developing civilization, I think it’s plausible that multiple human-level intelligent species arose in Earth’s history, but additional bottlenecks prevented them from developing technological civilization. However, most of these bottlenecks wouldn’t be an issue for an intelligence generated by simulated evolution. E.g., we could intervene in such a simulation to give low-dexterity species other means of manipulating their environment. Perhaps Earth’s evolutionary history actually contains n human-level intelligent species, only one of which developed technology. That implies the true compute required to evolve human-level intelligence is far lower.
Brain imitation learning:
I also think the discussion of neuromophic AI and whole brain emulation misses an important possibility that Gwern calls “brain imitation learning”. In essence, you record a bunch of data about human brain activity (using EEG, implanted electrodes, etc.), then you train a deep neural network to model the recorded data (similar to how GPT-3 or BERT model text). The idea is that modeling brain activity will cause the deep network to learn some of the brain’s neurological algorithms. Then, you train the deep network on some downstream task and hope its learned brain algorithms generalize to the task in question.
I think brain imitation learning is pretty likely to work. We’ve repeatedly seen in deep learning that knowledge distillation (training a smaller student model to imitate a larger teacher model) is FAR more computationally efficient than trying to train the student model from scratch, while also giving superior performance (Wikipedia, distilling BERT, distilling CLIP). Admittedly, brain activity data is pretty expensive. However, the project that finally builds human-level AI will plausibly cost billions of dollars in compute for training. If brain imitation learning can cut the price by even 10%, it will be worth hundreds of millions in terms of saved compute costs.
I think it’s very unlikely that Earth has seen other species as intelligent as humans (with the possible exception of other Homo species). In short, I suspect there is strong selection pressure for (at least many of) the different traits that allow humans to have civilization to go together. Consider dexterity – such skills allow one to use intelligence to make tools; that is, the more dexterous one is, the greater the evolutionary value of high intelligence, and the more intelligent one is, the greater the evolutionary value of dexterity. Similar positive feedback loops also seem likely between intelligence and: longevity, being omnivorous, having cumulative culture, hypersociality, language ability, vocal control, etc.
Additionally, birds and mammals are both considered unusually intelligent for animals (more so than reptiles, amphibians, fish, etc), and both birds and mammals have seen (neurological evidence of) gradual trends of increasing (maximum) intelligence over the course of the past 100 MY or more (and even extant nonhuman great apes seem most likely to be somewhat smarter than their last common ancestors with humans). So if there was a previously intelligent species, I’d be scratching my head about when it would have evolved. While we can’t completely rule out a previous species as smart as humans (we also can’t completely rule out a previous technological species, for which all artifacts have been destroyed), I think the balance of evidence is pretty strongly against, though I’ll admit that not everyone shares this view. Personally, I’d be absolutely shocked if there were 10+ (not very closely related) previous intelligent species, which is what would be required to reduce compute by just 1 OOM. (And even then, insofar as the different species shared a common ancestor, there still could be a hard step that the ancestor passed.)
But I do think it’s the case that certain bottlenecks on Earth wouldn’t be a bottleneck for engineers. For instance, I think there’s a good chance that we simply got lucky in the past several hundred million years for the climate staying ~stable instead of spiraling into uninhabitable hothouse or snowball states (i.e., we may be subject to survivorship bias here); this seems very easy for human engineers to work around in simulations. The same is plausibly true for other bottlenecks as well.
Re: Brain imitation learning
My cop-out answer here is that this is already covered by the “other methods” section. My real answer is that the model isn’t great at handling approaches that are intermediate between different methods. I agree it makes sense to continue to watch this space.
Thank you for this excellent post. Here are some thoughts I had while reading.
The hard paths hypothesis:
I think there’s another side to the hard paths hypothesis. We are clearly the first technology-using species to evolve on Earth. However, it’s entirely possible that we’re not the first species with human-level intelligence. If a species with human level intelligence but no opposable thumbs evolved millions of years ago, they could have died out without leaving any artifacts we’d recognize as signs of intelligence.
Besides our intelligence, humans seem odd in many ways that could plausibly contribute to developing a technological civilization.
We are pretty long-lived.
We are fairly social.
Feral children raised outside of human culture experience serious and often permanent mental disabilities (Wikipedia).
A species with human-level intelligence, but whose members live mostly independently may not develop technological civilization.
We have very long childhoods.
We have ridiculously high manual dexterity (even compared to other primates).
We live on land.
Most animals are aquatic.
It’s hard to have an industrial revolution when you can’t burn things.
Note that by Wikipedia’s listed estimates for cortical neuron counts, there are multiple dolphin/whale species with higher counts than us.
Given how well-tuned our biology seems for developing civilization, I think it’s plausible that multiple human-level intelligent species arose in Earth’s history, but additional bottlenecks prevented them from developing technological civilization. However, most of these bottlenecks wouldn’t be an issue for an intelligence generated by simulated evolution. E.g., we could intervene in such a simulation to give low-dexterity species other means of manipulating their environment. Perhaps Earth’s evolutionary history actually contains n human-level intelligent species, only one of which developed technology. That implies the true compute required to evolve human-level intelligence is far lower.
Brain imitation learning:
I also think the discussion of neuromophic AI and whole brain emulation misses an important possibility that Gwern calls “brain imitation learning”. In essence, you record a bunch of data about human brain activity (using EEG, implanted electrodes, etc.), then you train a deep neural network to model the recorded data (similar to how GPT-3 or BERT model text). The idea is that modeling brain activity will cause the deep network to learn some of the brain’s neurological algorithms. Then, you train the deep network on some downstream task and hope its learned brain algorithms generalize to the task in question.
I think brain imitation learning is pretty likely to work. We’ve repeatedly seen in deep learning that knowledge distillation (training a smaller student model to imitate a larger teacher model) is FAR more computationally efficient than trying to train the student model from scratch, while also giving superior performance (Wikipedia, distilling BERT, distilling CLIP). Admittedly, brain activity data is pretty expensive. However, the project that finally builds human-level AI will plausibly cost billions of dollars in compute for training. If brain imitation learning can cut the price by even 10%, it will be worth hundreds of millions in terms of saved compute costs.
Thanks for the comments!
Re: The Hard Paths Hypothesis
I think it’s very unlikely that Earth has seen other species as intelligent as humans (with the possible exception of other Homo species). In short, I suspect there is strong selection pressure for (at least many of) the different traits that allow humans to have civilization to go together. Consider dexterity – such skills allow one to use intelligence to make tools; that is, the more dexterous one is, the greater the evolutionary value of high intelligence, and the more intelligent one is, the greater the evolutionary value of dexterity. Similar positive feedback loops also seem likely between intelligence and: longevity, being omnivorous, having cumulative culture, hypersociality, language ability, vocal control, etc.
Regarding dolphins and whales, it is true that many have more neurons than us, but they also have thin cortices, low neuronal packing densities, and low axonal conduction velocities (in addition to lower EQs than humans).
Additionally, birds and mammals are both considered unusually intelligent for animals (more so than reptiles, amphibians, fish, etc), and both birds and mammals have seen (neurological evidence of) gradual trends of increasing (maximum) intelligence over the course of the past 100 MY or more (and even extant nonhuman great apes seem most likely to be somewhat smarter than their last common ancestors with humans). So if there was a previously intelligent species, I’d be scratching my head about when it would have evolved. While we can’t completely rule out a previous species as smart as humans (we also can’t completely rule out a previous technological species, for which all artifacts have been destroyed), I think the balance of evidence is pretty strongly against, though I’ll admit that not everyone shares this view. Personally, I’d be absolutely shocked if there were 10+ (not very closely related) previous intelligent species, which is what would be required to reduce compute by just 1 OOM. (And even then, insofar as the different species shared a common ancestor, there still could be a hard step that the ancestor passed.)
But I do think it’s the case that certain bottlenecks on Earth wouldn’t be a bottleneck for engineers. For instance, I think there’s a good chance that we simply got lucky in the past several hundred million years for the climate staying ~stable instead of spiraling into uninhabitable hothouse or snowball states (i.e., we may be subject to survivorship bias here); this seems very easy for human engineers to work around in simulations. The same is plausibly true for other bottlenecks as well.
Re: Brain imitation learning
My cop-out answer here is that this is already covered by the “other methods” section. My real answer is that the model isn’t great at handling approaches that are intermediate between different methods. I agree it makes sense to continue to watch this space.