I’d say your post focused on convincing the average techie or academic that Eliezer is wrong, but didn’t try to focus on what Eliezer would see as cruxes. That might be a reasonable choice of where to focus, given the results of prior attempts to address Eliezer’s cruxes. You gave a gave a better summary of why Eliezer’s cruxes are controversial in this section of Age of Em.
I’ll make another attempt to focus on Eliezer’s cruxes.
Intelligence Explosion Microeconomics seems to be the main place where Eliezer attempts to do more than say “my model is better than your reference class”.
Here’s a summary of what I consider his most interesting evidence:
most of the differences between humans and chimps are almost certainly algorithmic. If just taking an Australopithecus brain and scaling it up by a factor of four produced a human, the evolutionary road from Australopithecus to Homo sapiens would probably have been much shorter; simple factors like the size of an organ can change quickly in the face of strong evolutionary pressures.
At the time that was written, there may have been substantial expert support for those ideas. But more recent books have convinced me Eliezer is very wrong here.
Henrich’s book The Secret of Our Success presents fairly strong evidence that humans did not stumble on any algorithmic improvement that could be confused with a core of general intelligence. Human uniqueness derives mainly from better transmission of knowledge.
Herculano-Houzel’s book The Human Advantage presents clear evidence that large primates are pushing the limits of how big their brains can be. Getting enough calories takes more than 8 hours per day of foraging and feeding. That suggests strong evolutionary pressures for bigger brains, enough to reach a balance with the pressure from starvation risk. The four-fold increase in human brain size was likely less important than culture, but I see plenty of inconclusive hints that it was more important than new algorithms.
I’d say your post focused on convincing the average techie or academic that Eliezer is wrong, but didn’t try to focus on what Eliezer would see as cruxes. That might be a reasonable choice of where to focus, given the results of prior attempts to address Eliezer’s cruxes. You gave a gave a better summary of why Eliezer’s cruxes are controversial in this section of Age of Em.
I’ll make another attempt to focus on Eliezer’s cruxes.
Intelligence Explosion Microeconomics seems to be the main place where Eliezer attempts to do more than say “my model is better than your reference class”.
Here’s a summary of what I consider his most interesting evidence:
At the time that was written, there may have been substantial expert support for those ideas. But more recent books have convinced me Eliezer is very wrong here.
Henrich’s book The Secret of Our Success presents fairly strong evidence that humans did not stumble on any algorithmic improvement that could be confused with a core of general intelligence. Human uniqueness derives mainly from better transmission of knowledge.
Herculano-Houzel’s book The Human Advantage presents clear evidence that large primates are pushing the limits of how big their brains can be. Getting enough calories takes more than 8 hours per day of foraging and feeding. That suggests strong evolutionary pressures for bigger brains, enough to reach a balance with the pressure from starvation risk. The four-fold increase in human brain size was likely less important than culture, but I see plenty of inconclusive hints that it was more important than new algorithms.