This is an interesting view on AI, but IMO I don’t really share this view, and think that the evolutionary/memetic aspect of AI is way overplayed, compared to other factors that make AI powerful.
A big reason for that is that there will be higher-level bounds on what exactly is selected for, and in particular one big difference between computer code used on AI and genetic code is that genetic code has way less ability to error-correct than basically all AI code, and it’s in a weird spot of reliability where random mutations are frequent enough to drive evolution, but not so frequent as to cause organisms to outright collapse within seconds or minutes.
Another reason is that effective AI architectures can’t go through simulated evolution, since that would use up too much compute for training to work (We forget that evolution had at a lower bound 10e46 FLOPs to 10e48 FLOPs to get to humans).
A better analogy is within human-lifetime learning.
I basically agree with Steven Byrnes’s case against evolution, and think that evolutionary analogies are very overplayed in the popular press:
I agree that evolutionary arguments are frequently confused and oversimplified, but your argument is proving too much.
[the difference between] AI and genetic code is that genetic code has way less ability to error-correct than basically all AI code, and it’s in a weird spot of reliability where random mutations are frequent enough to drive evolution, but not so frequent as to cause organisms to outright collapse within seconds or minutes.
This “weird spot of reliability” is itself an evolved trait, and even with the effects of mutation rate variation between species, the variation within populations is heavily constrained (see Lewontin’s paradox of diversity). Even discounting purely genetic/code-based(?) factors, the amount of plasticity (?search) in behaviour is also an evolvable trait (see canalisation) - I think it’s likely there are already terms for this within the AI field but it’s not obvious to me how best to link the two ideas together. I’m more curious about the value drift evolutionary arguments but I don’t see an a priori reason that these ideas don’t apply.
It would be good if we could understand the conditions under which greater plasticity/evolvability is selected for, and whether we expect its effects to occur in a timeframe relevant to near-term alignment/safety.
Another reason is that effective AI architectures can’t go through simulated evolution, since that would use up too much compute for training to work (We forget that evolution had at a lower bound 10e46 FLOPs to 10e48 FLOPs to get to humans).
It’s not obvious to me that this is a sharp lower-bound, particularly when AI are already receiving the benefits of prior human computation in the form of culture. Human evolution had to achieve the hard part of reifying the world into semantic objects whereas AI has a major head-start. If language is the key idea (as some have argued), then I think there’s a decent chance that the lower bound is smaller than this.
The ‘evolutionary pressures’ being discussed by CGP Grey is not the direct gradient descent used to train an individual model. Instead, he is referring to the whole set of incentives we as a society put on AI models. Similar to memes—there is no gradient descent on memes.
(Apologies if you already understood this, but it seems your post and Steven Byrne’s post focus on training of individual models)
Fair enough on that difference between the societial level incentives on AI models and the individual selection incentives on AI models.
My main current response is to say that I think the incentives are fairly weak predictors of the variance in outcomes, compared to non-evolutionary forces at this time.
However, I do think this has interesting consequences for AI governance (since one of the effects is to make societal level incentives become more relevant, compared to non-evolutionary forces.)
This is an interesting view on AI, but IMO I don’t really share this view, and think that the evolutionary/memetic aspect of AI is way overplayed, compared to other factors that make AI powerful.
A big reason for that is that there will be higher-level bounds on what exactly is selected for, and in particular one big difference between computer code used on AI and genetic code is that genetic code has way less ability to error-correct than basically all AI code, and it’s in a weird spot of reliability where random mutations are frequent enough to drive evolution, but not so frequent as to cause organisms to outright collapse within seconds or minutes.
Another reason is that effective AI architectures can’t go through simulated evolution, since that would use up too much compute for training to work (We forget that evolution had at a lower bound 10e46 FLOPs to 10e48 FLOPs to get to humans).
A better analogy is within human-lifetime learning.
I basically agree with Steven Byrnes’s case against evolution, and think that evolutionary analogies are very overplayed in the popular press:
https://www.lesswrong.com/posts/pz7Mxyr7Ac43tWMaC/against-evolution-as-an-analogy-for-how-humans-will-create
I agree that evolutionary arguments are frequently confused and oversimplified, but your argument is proving too much.
This “weird spot of reliability” is itself an evolved trait, and even with the effects of mutation rate variation between species, the variation within populations is heavily constrained (see Lewontin’s paradox of diversity). Even discounting purely genetic/code-based(?) factors, the amount of plasticity (?search) in behaviour is also an evolvable trait (see canalisation) - I think it’s likely there are already terms for this within the AI field but it’s not obvious to me how best to link the two ideas together. I’m more curious about the value drift evolutionary arguments but I don’t see an a priori reason that these ideas don’t apply.
It would be good if we could understand the conditions under which greater plasticity/evolvability is selected for, and whether we expect its effects to occur in a timeframe relevant to near-term alignment/safety.
It’s not obvious to me that this is a sharp lower-bound, particularly when AI are already receiving the benefits of prior human computation in the form of culture. Human evolution had to achieve the hard part of reifying the world into semantic objects whereas AI has a major head-start. If language is the key idea (as some have argued), then I think there’s a decent chance that the lower bound is smaller than this.
prompts already go through undesigned evolution through reproductive fitness (rendered in 4k artstation flickr 2014)
The ‘evolutionary pressures’ being discussed by CGP Grey is not the direct gradient descent used to train an individual model. Instead, he is referring to the whole set of incentives we as a society put on AI models. Similar to memes—there is no gradient descent on memes.
(Apologies if you already understood this, but it seems your post and Steven Byrne’s post focus on training of individual models)
Fair enough on that difference between the societial level incentives on AI models and the individual selection incentives on AI models.
My main current response is to say that I think the incentives are fairly weak predictors of the variance in outcomes, compared to non-evolutionary forces at this time.
However, I do think this has interesting consequences for AI governance (since one of the effects is to make societal level incentives become more relevant, compared to non-evolutionary forces.)