Temporarily adopting this sort of model of “AI capabilities are useful compared to human IQs”:
With IQ 100 AGI (i.e. could do about the same fraction of tasks as well as a sample of IQ 100 humans), progress may well be hyper exponentially fast: but the lead-in to a hyper-exponentially fast function could be very, very slow. The majority of even relatively incompetent humans in technical fields like AI development have greater than IQ 100. Eventually quantity may have a quality of its own, e.g. after there were very large numbers of these sub-par researcher equivalents running at faster than human and coordinated better than I would expect average humans to be.
Absent enormous numerical or speed advantages, I wouldn’t expect substantial changes in research speed until something vaguely equivalent to IQ 160 or so.
Though in practice, I’m not sure that human measures of IQ are usefully applicable to estimating rates of AI-assisted research. They are not human, and only hindsight could tell what capabilities turn out to be the most useful to advancing research. A narrow tool along the lines of AlphaFold could turn out to be radically important to research rate without having anything that you could characterize as IQ. On the other hand, it may turn out that exceeding human research capabilities isn’t practically possible from any system pretrained on material steeped in existing human paradigms and ontology.
Perhaps thinking about IQ conflates too things: correctness and speed. For individual humans, these seem correlated: people with higher IQ are usually able to get more correct results, more quickly.
But it becomes relevant when talking about groups of people: Whether a group of average people is better than a genius, depends on the nature of the task. The genius will be better at doing novel research. The group of normies will be better at doing lots of trivial paperwork.
Currently, the AIs seem comparable to having an army of normies on steroids.
The performance of a group of normies (literal or metaphorical) can sometimes be improved by error checking. For example, if you have them solve mathematical problems, they will probably make a lot of errors; adding more normies would allow you to solve more problems, but the fraction of correct solutions would remain the same. But if you give them instructions how the verify the solutions, you could increase the correctness (at a cost of slowing them down somewhat). Similarly, an LLM can give me hallucinated solutions to math / programming problems, but that is less of a concern if I can verify the solutions in Lean / using unit tests, and reject the incorrect ones; and who knows, maybe trying again will result in a better solution. (In a hypothetical extreme case, an army of monkeys with typewriters could produce Shakespeare, if we had a 100% reliable automatic verifier of their outputs.)
So it seems to me, the question is how much we can compensate for the errors caused by “lower IQ”. Depending on the answer, that’s how long we have to wait until the AIs become that intelligent.
More important is “size of a team of humans” vs “peak capabilities of a team of humans” (or maybe sum of the cubes of their IQs?)
A given person thinking faster than average is more equivalent to a multiplier on the number of people of exactly that intelligence you have.
(Of course, if you could radically increase brain speed, I would expect IQ to increase rather than remain constant, but that isn’t yet an option for humans).
Temporarily adopting this sort of model of “AI capabilities are useful compared to human IQs”:
With IQ 100 AGI (i.e. could do about the same fraction of tasks as well as a sample of IQ 100 humans), progress may well be hyper exponentially fast: but the lead-in to a hyper-exponentially fast function could be very, very slow. The majority of even relatively incompetent humans in technical fields like AI development have greater than IQ 100. Eventually quantity may have a quality of its own, e.g. after there were very large numbers of these sub-par researcher equivalents running at faster than human and coordinated better than I would expect average humans to be.
Absent enormous numerical or speed advantages, I wouldn’t expect substantial changes in research speed until something vaguely equivalent to IQ 160 or so.
Though in practice, I’m not sure that human measures of IQ are usefully applicable to estimating rates of AI-assisted research. They are not human, and only hindsight could tell what capabilities turn out to be the most useful to advancing research. A narrow tool along the lines of AlphaFold could turn out to be radically important to research rate without having anything that you could characterize as IQ. On the other hand, it may turn out that exceeding human research capabilities isn’t practically possible from any system pretrained on material steeped in existing human paradigms and ontology.
Perhaps thinking about IQ conflates too things: correctness and speed. For individual humans, these seem correlated: people with higher IQ are usually able to get more correct results, more quickly.
But it becomes relevant when talking about groups of people: Whether a group of average people is better than a genius, depends on the nature of the task. The genius will be better at doing novel research. The group of normies will be better at doing lots of trivial paperwork.
Currently, the AIs seem comparable to having an army of normies on steroids.
The performance of a group of normies (literal or metaphorical) can sometimes be improved by error checking. For example, if you have them solve mathematical problems, they will probably make a lot of errors; adding more normies would allow you to solve more problems, but the fraction of correct solutions would remain the same. But if you give them instructions how the verify the solutions, you could increase the correctness (at a cost of slowing them down somewhat). Similarly, an LLM can give me hallucinated solutions to math / programming problems, but that is less of a concern if I can verify the solutions in Lean / using unit tests, and reject the incorrect ones; and who knows, maybe trying again will result in a better solution. (In a hypothetical extreme case, an army of monkeys with typewriters could produce Shakespeare, if we had a 100% reliable automatic verifier of their outputs.)
So it seems to me, the question is how much we can compensate for the errors caused by “lower IQ”. Depending on the answer, that’s how long we have to wait until the AIs become that intelligent.
In fact, speed and accuracy in humans are at least somewhat mechanistically different
More important is “size of a team of humans” vs “peak capabilities of a team of humans” (or maybe sum of the cubes of their IQs?) A given person thinking faster than average is more equivalent to a multiplier on the number of people of exactly that intelligence you have. (Of course, if you could radically increase brain speed, I would expect IQ to increase rather than remain constant, but that isn’t yet an option for humans).