“interesting difference here: he seems to think that it would only require, say, 2-3x more computing power, and thus only 2-3x more budget, to go from human-level intelligence to ultraintelligence, noting that the human brain is about that much of a factor larger than a chimpanzee brain”
That seems obviously unjustified. I expect you can reduce a human brain by 2x and still have them be essentially human. The metric could be the max intelligence you can get with a chimp number of neurons/synapses which I expect is probably an IQ 80+ human. There was an “overhang” situation with chimp brains where they could have been optimized to be much better at abstract thought with the same brainpower but it didn’t happen.
“interesting difference here: he seems to think that it would only require, say, 2-3x more computing power, and thus only 2-3x more budget, to go from human-level intelligence to ultraintelligence, noting that the human brain is about that much of a factor larger than a chimpanzee brain”
That seems obviously unjustified. I expect you can reduce a human brain by 2x and still have them be essentially human. The metric could be the max intelligence you can get with a chimp number of neurons/synapses which I expect is probably an IQ 80+ human. There was an “overhang” situation with chimp brains where they could have been optimized to be much better at abstract thought with the same brainpower but it didn’t happen.