What would you say if I said I still think there’s a good chance that meaningful intelligence explosion (accelerating absent compute growth) is difficult or impossible prior to roughly-human sample-efficiency?
As you suggest, I think the parity must be achievable substantially before this point. AI will have such massive advantages in other areas, by the time they match human sample efficiency they will be way way way important than humans to overall R&D. And so parity must come substantially sooner.
It’s believable to me that sample efficiency undergoes a meaningful jump at this (slightly indeterminate) stage. Still unclear if that’s an accelerating SIE/singularity. Do you consider your and others’ previous work on estimating the returns to R&D to be the key summary on this question?
Not sure if I follow. I expect sample efficiency to improve continuously and significant acceleration to happen before it reaches human level. If the r<1 then acceleration will be limited, and so maybe AI progress is steady at 2x faster even as sample efficiency reaches and surpasses human-level.
I’m on board with, in principle, massive advantages in other places outweighing sample-efficiency deficiency—at least, we haven’t given an argument against that here.
But what are the massive advantages? Can they really meaningfully outweigh a sample-efficiency deficiency at the frontier of research in a big way?
Taking two obvious possible advantages:
faster thinking (but not of experimenting)
parallel count
I think both of the above have very diminishing returns. How diminishing? Not sure! Some theory suggests logarithmic at best, and empirics maybe line up with that. My sense is this largely neuters these as providing non-negligible frontier-research-relevant advantages, especially in a compute-static context. But I haven’t thoroughly explored arguments for or against.
Other possible advantages:
don’t sleep etc (this is largely the same as ‘faster’)
more ‘in context’ at once (debatable where the threshold is, but surely a prospect)
how helpful? Don’t know! Strikes me as not especially helpful, or at least quite diminishing returns again
Glad you like it!
As you suggest, I think the parity must be achievable substantially before this point. AI will have such massive advantages in other areas, by the time they match human sample efficiency they will be way way way important than humans to overall R&D. And so parity must come substantially sooner.
Not sure if I follow. I expect sample efficiency to improve continuously and significant acceleration to happen before it reaches human level. If the r<1 then acceleration will be limited, and so maybe AI progress is steady at 2x faster even as sample efficiency reaches and surpasses human-level.
I’m on board with, in principle, massive advantages in other places outweighing sample-efficiency deficiency—at least, we haven’t given an argument against that here.
But what are the massive advantages? Can they really meaningfully outweigh a sample-efficiency deficiency at the frontier of research in a big way?
Taking two obvious possible advantages:
faster thinking (but not of experimenting)
parallel count
I think both of the above have very diminishing returns. How diminishing? Not sure! Some theory suggests logarithmic at best, and empirics maybe line up with that. My sense is this largely neuters these as providing non-negligible frontier-research-relevant advantages, especially in a compute-static context. But I haven’t thoroughly explored arguments for or against.
Other possible advantages:
don’t sleep etc (this is largely the same as ‘faster’)
more ‘in context’ at once (debatable where the threshold is, but surely a prospect)
how helpful? Don’t know! Strikes me as not especially helpful, or at least quite diminishing returns again
broader background knowledge
could provide more sources of ‘inspiration’?
Any more that seem relevant?