It’s not clear to me how badly EY erred. It seems that he was comparing the size of code designed by humans to the size of code “designed” by evolution, which would seem to be his primary mistake. I also concur that he shouldn’t get from complexity of the evolved brain to “number of insights needed to create AI” (charitably: he doesn’t claim to know the exact conversion ratio, but in principle there should be one).
I agree with your “information in genes+environment” although the example of needing light (and other inputs) for the brain to develop normally isn’t the best. I consider the womb-environment to be more impressive—how easy is it to reverse engineer the appropriate womb+mitochondria+??? (sorry, I’m no expert) given the genome?
This was nicely presented. Nitpick: I would not say “make it highly compressible”; I would say “make it smaller”. You can make something compressible by adding redundancy, which is not what you intended.
I think the post needlessly interprets EY as bounding the complexity of the brain to at most that of the genome. Of course the brain’s complexity reflects the learning environment—but, important as that is, in this context it doesn’t seem very relevant. It’s not that hard to “raise” an AI in an environment much like those humans are raised in. (Maybe that’s not a good way to create Friendly AI—or maybe it is—but I take it EY’s argument was about AI in general.)
I agree. More charitably, even, he could be counterarguing:
A lot of people think human intelligence must be immensely complicated, and therefore general AI is far from achievable. But look at what we now know about human brain architecture! perhaps it is simpler than you thought?
(not a quote and possibly not a faithful paraphrase)
It’s not clear to me how badly EY erred. It seems that he was comparing the size of code designed by humans to the size of code “designed” by evolution, which would seem to be his primary mistake. I also concur that he shouldn’t get from complexity of the evolved brain to “number of insights needed to create AI” (charitably: he doesn’t claim to know the exact conversion ratio, but in principle there should be one).
I agree with your “information in genes+environment” although the example of needing light (and other inputs) for the brain to develop normally isn’t the best. I consider the womb-environment to be more impressive—how easy is it to reverse engineer the appropriate womb+mitochondria+??? (sorry, I’m no expert) given the genome?
This was nicely presented. Nitpick: I would not say “make it highly compressible”; I would say “make it smaller”. You can make something compressible by adding redundancy, which is not what you intended.
I think the post needlessly interprets EY as bounding the complexity of the brain to at most that of the genome. Of course the brain’s complexity reflects the learning environment—but, important as that is, in this context it doesn’t seem very relevant. It’s not that hard to “raise” an AI in an environment much like those humans are raised in. (Maybe that’s not a good way to create Friendly AI—or maybe it is—but I take it EY’s argument was about AI in general.)
I agree. More charitably, even, he could be counterarguing:
(not a quote and possibly not a faithful paraphrase)
Or “make it highly compressed” perhaps.