Hey Steve! Thanks for writing this, it was an interesting and useful read! After our discussion in the LW comments, I wanted to get a better understanding of your thinking and this sequence is doing the job. Now I feel I can better engage in a technical discussion.
I can sympathize well with your struggle in section 2.6. A lot of the “big picture” neuroscience is in the stage where it’s not even wrong. That being said, I don’t think you’ll find a lot of neuroscientists who nod along with your line of argument without raising objections here and there (neuroscientists love their trivia). They might be missing the point, but I think that still makes your theory (by definition) controversial. (I think the term “scientific consensus” should be used carefully and very selectively).
In that spirit, there are a few points that I could push back on:
Cortical uniformity (and by extension canonical microcircuits) are extremely useful concepts for thinking about the brain. But they are not literally 100% accurate. There are a lot of differences between different regions of the cortex, not only in thickness but also in the developmental process (here or here). I don’t think anyone except for Jeff Hawkin believes in literal cortical uniformity.
In section 2.5.4.1 you are being a bit dismissive of biologically-”realistic” implementations of backpropagation. I used to be pretty skeptical too, but some of the recent studies are beginning to make a lot of sense. This one (a collaboration of Deepmind and some of the established neuroscience bigshots) is really quite elegant and offers some great insights on how interneurons and dendritic branches might interact.
A more theoretical counter: If evolution could initialize certain parts of the cortex so that they are faster “up and running” why wouldn’t it? (Just so that we can better understand it? How nice!) From the perspective of evolution, it makes a lot of sense to initialize the cortex with an idea of what an oriented edge is because oriented edges have always been around since the inception of the eye. Or, in terms of computation theory, learning from scratch is computationally intractable. Strong, informative priors over hypothesis space might just be necessary to learn anything worthwhile at all.
But perhaps I’m missing the point with that nitpicking. I think the broader conceptual question I have is: What does “randomly initialized” even mean in the brain? At what point is the brain initialized? When the neural tube forms? When interneurons begin to migrate to the cortex? When the first synapses are established? When the subplate is gone? When the pruning of excess synapses and the apoptosis of cells is over? When the animal/human is born? When all the senses begin to transmit input? After college graduation?
Perhaps this is the point that the “old-timer” also wanted to make. It doesn’t really make sense to separate the “initialization” from the “refinement”. They happen at the same time, and whether you put a certain thing into one category or the other is up to individual taste.
All of this being said, I’m very curious to read the next parts of this sequence! :) Perhaps my points don’t even affect your core argument about AI Safety.
Hey Steve! Thanks for writing this, it was an interesting and useful read! After our discussion in the LW comments, I wanted to get a better understanding of your thinking and this sequence is doing the job. Now I feel I can better engage in a technical discussion.
I can sympathize well with your struggle in section 2.6. A lot of the “big picture” neuroscience is in the stage where it’s not even wrong. That being said, I don’t think you’ll find a lot of neuroscientists who nod along with your line of argument without raising objections here and there (neuroscientists love their trivia). They might be missing the point, but I think that still makes your theory (by definition) controversial. (I think the term “scientific consensus” should be used carefully and very selectively).
In that spirit, there are a few points that I could push back on:
Cortical uniformity (and by extension canonical microcircuits) are extremely useful concepts for thinking about the brain. But they are not literally 100% accurate. There are a lot of differences between different regions of the cortex, not only in thickness but also in the developmental process (here or here). I don’t think anyone except for Jeff Hawkin believes in literal cortical uniformity.
In section 2.5.4.1 you are being a bit dismissive of biologically-”realistic” implementations of backpropagation. I used to be pretty skeptical too, but some of the recent studies are beginning to make a lot of sense. This one (a collaboration of Deepmind and some of the established neuroscience bigshots) is really quite elegant and offers some great insights on how interneurons and dendritic branches might interact.
A more theoretical counter: If evolution could initialize certain parts of the cortex so that they are faster “up and running” why wouldn’t it? (Just so that we can better understand it? How nice!) From the perspective of evolution, it makes a lot of sense to initialize the cortex with an idea of what an oriented edge is because oriented edges have always been around since the inception of the eye.
Or, in terms of computation theory, learning from scratch is computationally intractable. Strong, informative priors over hypothesis space might just be necessary to learn anything worthwhile at all.
But perhaps I’m missing the point with that nitpicking. I think the broader conceptual question I have is: What does “randomly initialized” even mean in the brain? At what point is the brain initialized? When the neural tube forms? When interneurons begin to migrate to the cortex? When the first synapses are established? When the subplate is gone? When the pruning of excess synapses and the apoptosis of cells is over? When the animal/human is born? When all the senses begin to transmit input? After college graduation?
Perhaps this is the point that the “old-timer” also wanted to make. It doesn’t really make sense to separate the “initialization” from the “refinement”. They happen at the same time, and whether you put a certain thing into one category or the other is up to individual taste.
All of this being said, I’m very curious to read the next parts of this sequence! :) Perhaps my points don’t even affect your core argument about AI Safety.