I generally agree with this, so I’ll just elaborate on disagreements, anything that I don’t mention you should assume I agree with it.
On Amdahl’s law:
These AIs wouldn’t be able to automate some tasks (without a human helping them) and this bottleneck would limit the speed-up due to Amdahl’s law.
While I agree in the context of the post, I generally don’t like Amdahl’s law arguments, and tend to think they’re a midwit trap, because people forget that more resources don’t just cause people to solve old problems more efficiently, but to make new problems practical at all, and this is why I believe parallelization is usually better than pessimists argue, due to Gustafson-Barsis’s law.
This doesn’t matter here, but it does matter once you fully automate a field.
There is an obvious consequence that this will cause increased awareness and salience of: AI, AI automating AI R&D, and the potential for powerful capabilities in the short term.
So I agree there will be more salience, but I generally expect this to be pretty restrained, and in genpop, I expect much more discontinuous salience and responses, and I expect much weaker responses until we have full automation of AI R&D at least, and maybe even longer than that.
A key worldview difference is I expect genpop already believes in/is motivated to hear this argument for a very long time, regardless of whether this is correct:
“Now that we’ve seen AIs automate AI R&D and no one is even claiming that we’re seeing explosive capabilities growth, the intelligence explosion has been disproven; compute bottlenecks really are decisive. (Or insert whichever bottleneck this person believes is most important.) The intelligence explosion must have been bullshit all along and look, we don’t see any of these intelligence explosion proponents apologizing for being wrong, probably they’re off inventing some new milestone of AI to fearmonger about.”
I generally agree with this, so I’ll just elaborate on disagreements, anything that I don’t mention you should assume I agree with it.
On Amdahl’s law:
While I agree in the context of the post, I generally don’t like Amdahl’s law arguments, and tend to think they’re a midwit trap, because people forget that more resources don’t just cause people to solve old problems more efficiently, but to make new problems practical at all, and this is why I believe parallelization is usually better than pessimists argue, due to Gustafson-Barsis’s law.
This doesn’t matter here, but it does matter once you fully automate a field.
So I agree there will be more salience, but I generally expect this to be pretty restrained, and in genpop, I expect much more discontinuous salience and responses, and I expect much weaker responses until we have full automation of AI R&D at least, and maybe even longer than that.
A key worldview difference is I expect genpop already believes in/is motivated to hear this argument for a very long time, regardless of whether this is correct: