Artificial Neural Networks have been trained with millions of parameters. There are a lot of different methods of regularization like dropconnect or sparsity constraints. But the brain does online learning. Overfitting isn’t as big of a concern because it doesn’t see the data more than once.
On the other hand, architecture matters. The most successful neural network for a given task has connections designed for the structure of that task, so that it will learn much more quickly than a fully-connected or arbitrarily connected network.
The human brain appears to have a great deal of information and structure in its architecture right off the bat.
I’m not saying that you’re wrong, but the state of the art in computer vision is weight sharing which biological NNs probably can’t do. Hyper parameters like the number of layers and how local the connections should be, are important but they don’t give that much prior information about the task.
I may be completely wrong, but I do suspect that biological NNs are far more general purpose and less “pre-programmed” than is usually thought. The learning rules for a neural network are far simpler than the functions they learn. Training neural networks with genetic algorithms is extremely slow.
Architecture of the V1 and V2 areas of the brain, which Convolutional Neural Networks and other ANNs for vision borrow heavily from, is highly geared towards vision, and includes basic filters that detect stripes, dots, corners, etc. that appear in all sorts of computer vision work. Yes, no backpropagation or weight-sharing is directly responsible for this, but the presence of local filters is still what I would call very specific architecture (I’ve studied computer vision and inspiration it draws from early vision specifically, so I can say more about this).
The way genetic algorithms tune weights in an ANN (and yes, this is an awful way to train an ANN) is very different from the way they work in actually evolving a brain; working on the genetic code that develops the brain. I’d say they are so wildly different that no conclusions from the first can be applied to the second.
During a single individual’s life, Hebbian and other learning mechanisms in the brain are distinct from gradient learning, but can achieve somewhat similar things.
The human brain appears to engage in hierarchical learning, which is what allows it to leverage huge amounts of “general case” abstract knowledge in attacking novel specific problems put before it.
Artificial Neural Networks have been trained with millions of parameters. There are a lot of different methods of regularization like dropconnect or sparsity constraints. But the brain does online learning. Overfitting isn’t as big of a concern because it doesn’t see the data more than once.
On the other hand, architecture matters. The most successful neural network for a given task has connections designed for the structure of that task, so that it will learn much more quickly than a fully-connected or arbitrarily connected network.
The human brain appears to have a great deal of information and structure in its architecture right off the bat.
I’m not saying that you’re wrong, but the state of the art in computer vision is weight sharing which biological NNs probably can’t do. Hyper parameters like the number of layers and how local the connections should be, are important but they don’t give that much prior information about the task.
I may be completely wrong, but I do suspect that biological NNs are far more general purpose and less “pre-programmed” than is usually thought. The learning rules for a neural network are far simpler than the functions they learn. Training neural networks with genetic algorithms is extremely slow.
Architecture of the V1 and V2 areas of the brain, which Convolutional Neural Networks and other ANNs for vision borrow heavily from, is highly geared towards vision, and includes basic filters that detect stripes, dots, corners, etc. that appear in all sorts of computer vision work. Yes, no backpropagation or weight-sharing is directly responsible for this, but the presence of local filters is still what I would call very specific architecture (I’ve studied computer vision and inspiration it draws from early vision specifically, so I can say more about this).
The way genetic algorithms tune weights in an ANN (and yes, this is an awful way to train an ANN) is very different from the way they work in actually evolving a brain; working on the genetic code that develops the brain. I’d say they are so wildly different that no conclusions from the first can be applied to the second.
During a single individual’s life, Hebbian and other learning mechanisms in the brain are distinct from gradient learning, but can achieve somewhat similar things.
The human brain appears to engage in hierarchical learning, which is what allows it to leverage huge amounts of “general case” abstract knowledge in attacking novel specific problems put before it.