Is there a follow up? I was expecting to see some kind of 2023 update because the existence of GPT-4 + plugins allows us to reject out of hand almost everything from bio-anchors.
Eliezer happens to be correct.
The reason for the out of hand rejection is simple. The human brain has 7 properties not considered here that make this a garbage analysis. Though to be fair, we didn’t know it was garbage until models started to show signs of generality in late 2022.
(1) Heavy internal processing noise, where a computation must be repeated many times to get reliability. This can let you pull 1-2 OOM off the top of training and inference time. One piece of evidence that this is correct is, well, GPT-3/4. Assuming the brain has 86 billion neurons, with ~1000 synapses each, and each stores 1 byte of state, then you need 86 terabytes of memory to represent the weights of 1 brain. Dendrite computation must be learnable or we can just ignore that. Yet we seem to be able to store more information than a human being is able to learn in 0.35 terabytes, the weights of GPT-3, of ~3.2 for GPT-4.
There are are also papers on the high noise rate and poor determinism for neurons and high timing jitter you can examine.
(2) Entire modalities, or large chunks of the circuitry the brain has, obviously don’t matter at all during modalities. This includes the computation they do. For example, consider a student sitting in class. They are processing the teacher’s audio back to phonemes, which I will pretend are the same as tokens, and processing each image they see presented as a slide back to a set of relational and object identity tokens. The human may fidget or pay attention to the needs of their body, all of which are not increasing their knowledge of the subject.
The “training equivalent” of that classroom is simply feeding the contents of the course material as clean images and text right into the AI. You should be able to learn everything in an entire course of education with a fraction of the compute, and it appears that you can. The thing that GPT-3/4 are missing is they don’t have additional cognitive components to practice and apply what they know, as well as they do not have modalities for all the I/O a human has. But in terms of sheer breadth of facts? Superhuman.
(3) Entire roles the human brain does aren’t needed at all because very cheap to run conventional software can solve them. Proprioception of the human body → read the encoders for every joint in the robot. Stereo depth extraction → lidar. Hunger → query the BMS for the battery state of charge. Sound input → whisper → tokens. (robots don’t need to survive like humans, so it’s fine if they cannot hear danger, since the AI driving them does not lose information when a robot is destroyed)
(4) The brain is stuck building everything out of components it has the cellular machinery to construct and maintain. Entire architectural possibilities are unavailable to it. This combined with RSI—where we use AI models only slightly stronger than what are currently released today to explore a search space of possible architectures—would make algorithm improvement much faster than bioanchors models.
(5) The brain is on a self replicating robot. We do not need anywhere near the brain’s full suite of capabilities for AI to be transformative. This is another big miss. Entire capabilities like emotional intelligence are not even needed. Transformative means the robots can build each other and create more value than the GDP of the earth within a few years. And to accomplish this, there is a subset of brain capabilities you need. Anything not ‘left brain’ so to speak is useless, and that saves half your compute and training right there.
6. Another massive savings is most robots, most of the time, are going to be doing rote tasks that require minimal compute. One robot mines straight down a tunnel. Another picks up a rock, puts it into a cart, picks up a rock, puts it into a cart, … Another waits and waits and waits for the ore train to arrive. Another waits for an ingot to reach it, then puts it in the CNC machine, and waits for the next.
And so on. Industry has lots of very low brainpower tasks and a lot of waiting.
Humans run their brains this whole time for no purpose. We could simply use clusters of shared inference hardware, and not run any jobs for robots but a small local model that can do repetitive tasks and wait. Only when something outside the input distribution of the small local model causes jobs to run that actually use humanlike intelligence.
7. The human brain cannot simply write itself a software solver for problems it finds difficult. GPT-4, were it extended very slightly, could author itself a solver in Python for every math question it has trouble with, every “letters in this sentence” problem it gets wrong, and so on. It would use a pool of candidate solvers for each question and gradually upvote the best ones until it has reliable and robust python scripts that solve any of Gary Marcus style “gotcha” questions.
That would be like a human realizing math is annoying in the 3rd grade, writing a general program that uses mathematica and other general tools to solve all math below a certain difficulty, and then testing out of all future math courses.
Or a human realizing that history is difficult to memorize, so they write a python script that maintains a database searchable by text vector embeddings of all historical facts. They can immediately then just memorize the textbooks and test out.
These software solvers are stupid cheap, using billions of times less compute than AI accelerators do.
8. The scaling was also way off. OpenAI 10xed their funding since this was written, and there appears to be a real possibility that they will get 100 billion before 2030 to train AGI. Multiple other labs and parties are in the 1-10 billion range, including several Chinese companies.
Is there a follow up? I was expecting to see some kind of 2023 update because the existence of GPT-4 + plugins allows us to reject out of hand almost everything from bio-anchors.
Eliezer happens to be correct.
The reason for the out of hand rejection is simple. The human brain has 7 properties not considered here that make this a garbage analysis. Though to be fair, we didn’t know it was garbage until models started to show signs of generality in late 2022.
(1) Heavy internal processing noise, where a computation must be repeated many times to get reliability. This can let you pull 1-2 OOM off the top of training and inference time. One piece of evidence that this is correct is, well, GPT-3/4. Assuming the brain has 86 billion neurons, with ~1000 synapses each, and each stores 1 byte of state, then you need 86 terabytes of memory to represent the weights of 1 brain. Dendrite computation must be learnable or we can just ignore that. Yet we seem to be able to store more information than a human being is able to learn in 0.35 terabytes, the weights of GPT-3, of ~3.2 for GPT-4.
There are are also papers on the high noise rate and poor determinism for neurons and high timing jitter you can examine.
(2) Entire modalities, or large chunks of the circuitry the brain has, obviously don’t matter at all during modalities. This includes the computation they do. For example, consider a student sitting in class. They are processing the teacher’s audio back to phonemes, which I will pretend are the same as tokens, and processing each image they see presented as a slide back to a set of relational and object identity tokens. The human may fidget or pay attention to the needs of their body, all of which are not increasing their knowledge of the subject.
The “training equivalent” of that classroom is simply feeding the contents of the course material as clean images and text right into the AI. You should be able to learn everything in an entire course of education with a fraction of the compute, and it appears that you can. The thing that GPT-3/4 are missing is they don’t have additional cognitive components to practice and apply what they know, as well as they do not have modalities for all the I/O a human has. But in terms of sheer breadth of facts? Superhuman.
(3) Entire roles the human brain does aren’t needed at all because very cheap to run conventional software can solve them. Proprioception of the human body → read the encoders for every joint in the robot. Stereo depth extraction → lidar. Hunger → query the BMS for the battery state of charge. Sound input → whisper → tokens. (robots don’t need to survive like humans, so it’s fine if they cannot hear danger, since the AI driving them does not lose information when a robot is destroyed)
(4) The brain is stuck building everything out of components it has the cellular machinery to construct and maintain. Entire architectural possibilities are unavailable to it. This combined with RSI—where we use AI models only slightly stronger than what are currently released today to explore a search space of possible architectures—would make algorithm improvement much faster than bioanchors models.
(5) The brain is on a self replicating robot. We do not need anywhere near the brain’s full suite of capabilities for AI to be transformative. This is another big miss. Entire capabilities like emotional intelligence are not even needed. Transformative means the robots can build each other and create more value than the GDP of the earth within a few years. And to accomplish this, there is a subset of brain capabilities you need. Anything not ‘left brain’ so to speak is useless, and that saves half your compute and training right there.
6. Another massive savings is most robots, most of the time, are going to be doing rote tasks that require minimal compute. One robot mines straight down a tunnel. Another picks up a rock, puts it into a cart, picks up a rock, puts it into a cart, … Another waits and waits and waits for the ore train to arrive. Another waits for an ingot to reach it, then puts it in the CNC machine, and waits for the next.
And so on. Industry has lots of very low brainpower tasks and a lot of waiting.
Humans run their brains this whole time for no purpose. We could simply use clusters of shared inference hardware, and not run any jobs for robots but a small local model that can do repetitive tasks and wait. Only when something outside the input distribution of the small local model causes jobs to run that actually use humanlike intelligence.
7. The human brain cannot simply write itself a software solver for problems it finds difficult. GPT-4, were it extended very slightly, could author itself a solver in Python for every math question it has trouble with, every “letters in this sentence” problem it gets wrong, and so on. It would use a pool of candidate solvers for each question and gradually upvote the best ones until it has reliable and robust python scripts that solve any of Gary Marcus style “gotcha” questions.
That would be like a human realizing math is annoying in the 3rd grade, writing a general program that uses mathematica and other general tools to solve all math below a certain difficulty, and then testing out of all future math courses.
Or a human realizing that history is difficult to memorize, so they write a python script that maintains a database searchable by text vector embeddings of all historical facts. They can immediately then just memorize the textbooks and test out.
These software solvers are stupid cheap, using billions of times less compute than AI accelerators do.
8. The scaling was also way off. OpenAI 10xed their funding since this was written, and there appears to be a real possibility that they will get 100 billion before 2030 to train AGI. Multiple other labs and parties are in the 1-10 billion range, including several Chinese companies.