it would be impossible to establish that much order with pruning alone.
That seems straightforwardly falsified by ANNs like lotterytickets or weight agnostic NNs, where the weights are randomized and all the learning is done by adding/pruning away connections (you can think of it as searching in fully-connected net space). Most on point here would be Ramanujan et al 2019 which proves that as random NNs get bigger, you expect there to be a sub-network which computes whatever you need (which makes sense, really, why wouldn’t there be? NNs express such rich classes of functions, that if you can pick and choose from within a very large NN, it’d make sense you could sculpt whatever you need).
To clarify, I meant, within a human brain. Not within an artificial neural net. The human brain is highly ordered before birth by genetically determined chemotaxis signals guiding axon growth. Synapses are formed within relatively small regions of where the guided axons have ended up. This results in a large capacity for ‘fine’ adjustments, while having a preset large scale structure which is very similar between individuals. The amount of this large-scale order seen in the human brain, as evidenced by the broad similarity of Brodmann regions between individuals, is the thing I’m claiming couldn’t be established by pruning neurons.
That seems straightforwardly falsified by ANNs like lottery tickets or weight agnostic NNs, where the weights are randomized and all the learning is done by adding/pruning away connections (you can think of it as searching in fully-connected net space). Most on point here would be Ramanujan et al 2019 which proves that as random NNs get bigger, you expect there to be a sub-network which computes whatever you need (which makes sense, really, why wouldn’t there be? NNs express such rich classes of functions, that if you can pick and choose from within a very large NN, it’d make sense you could sculpt whatever you need).
To clarify, I meant, within a human brain. Not within an artificial neural net. The human brain is highly ordered before birth by genetically determined chemotaxis signals guiding axon growth. Synapses are formed within relatively small regions of where the guided axons have ended up. This results in a large capacity for ‘fine’ adjustments, while having a preset large scale structure which is very similar between individuals. The amount of this large-scale order seen in the human brain, as evidenced by the broad similarity of Brodmann regions between individuals, is the thing I’m claiming couldn’t be established by pruning neurons.
For more info: https://www.lesswrong.com/posts/Wr7N9ji36EvvvrqJK/response-to-quintin-pope-s-evolution-provides-no-evidence?commentId=r72N4LxupbmTtgN3i