(I also acknowledge that I failed to parse what Ryan is saying in the parenthetical — does he mean that he mostly expects such research-capable AIs to be wildly superhuman, or that he mostly doesn’t? I infer from context that he doesn’t, but this is a guess.)
When I said “Let’s say these AIs aren’t much more capable than needed for this capability bar (and the capability profile works out such that these AIs aren’t qualitatively wildly superhuman as seems likely to me).” I meant:
The AIs that dominate top human experts in these fields (as in, can fully automate these fields better than top human experts) are only just over the level of capability needed to do this and the capability profile works out such that you can achieve this level of capability without these AIs needing to be wildly superhuman in terms of general purpose capability. As in, there isn’t some otherwise trailing capability required for this automation that trails so hard that to get it high enough this requires that the AI is wildly superhuman in general. (I think this is likely, at least if we try to improve the capability profile which seems like it might help a lot.)
When I said “Let’s say these AIs aren’t much more capable than needed for this capability bar (and the capability profile works out such that these AIs aren’t qualitatively wildly superhuman as seems likely to me).” I meant:
The AIs that dominate top human experts in these fields (as in, can fully automate these fields better than top human experts) are only just over the level of capability needed to do this and the capability profile works out such that you can achieve this level of capability without these AIs needing to be wildly superhuman in terms of general purpose capability. As in, there isn’t some otherwise trailing capability required for this automation that trails so hard that to get it high enough this requires that the AI is wildly superhuman in general. (I think this is likely, at least if we try to improve the capability profile which seems like it might help a lot.)