I think it is often useful to talk about the point when human cognitive labor is totally obsolete. Thus, I think it also makes sense to separately talk about Top-human-Expert-Dominating AI (TEDAI): AIs which strictly dominate top human experts[5] in virtually all cognitive tasks (i.e., doable via remote work) while being at least 2x faster[6] and within a factor of 5 on cost[7].
Hm, but human cognitive labor isn’t totally obsolete, because there’s still those tasks where humans are a factor 5 cheaper.
Similarly for the definition in footnote 7: If AIs simultaneously became uniformly 2x cheaper than humans on all the tasks holding current compute prices fixed, then that’s not actually that impressive. Currently compute spend is a small fraction of spend on human cognitive labor. So even if we got AI software that could generate $2 of cognitive labor revenue for every $1 spent on compute, most of the cognitive labor income would still be accruing to humans.
Of course, TED-AI won’t be uniformly cheaper. It’ll have spiky capabilities compared to the human capability distribution. So the impressiveness comes from how, at the point where it’s only 5x more expensive than humans on its least comparative advantaged cognitive task (or 2x cheaper according to extrapolated compute prices on its most expensive task) then surely it’s far superhuman on most things, and humans are totally obsolete on most things.
It does feel kind of off to me that this would correspond to:
The AI generally feels roughly as smart as a top human expert and is able to dominate across virtually all domains via increasing capabilities further with other advantages
I would imagine that the AI would feel much smarter than a top human expert in more than half of domains, but then for there to be a few domains where it both has surprising weaknesses and where those “other advantages” doesn’t help that much.
(Somewhat less relevant point: Probably the AI will also feel very smart because “feel smart” is easier to train for than to actually do a good job in a bunch of domains, so the AI will succeed at that before it’s actually doing a good job.)
A more specific argument to expect spikiness and therefore TED-AI to be vastly superhuman in most areas:
The human brain is much more sample efficient than machine learning.
In some areas, we can’t generate many more samples than what humans can engage with in a lifetime. (E.g.: Forecasting of rare events or results of extremely expensive experiments.) As long as human sample efficiency is better than AI sample efficiency, it will be great to employ humans in these areas.
Once ML beats human sample efficiency, ML will be vastly superhuman in all the areas where there’s vastly more data than what humans could absorb in a lifetime.
Possible defeaters to that argument:
Something about generalization/transfer learning?
Maybe you can use some tricks for sample-efficient learning in the low-data regime that doesn’t transfer to the high-data regime. (E.g. put all the data points in-context and reason about them—this wouldn’t generalize to a number of data points that’s much greater than what fits in the context window.) And therefore beat humans in the low-data regime without necessarily being able to vastly outperform them in the high-data regime.
Maybe I’m just confused about what sense of “feel smart” and what “other advantages” you have in mind such that the advantages helps the AI dominate a bunch of domains but doesn’t cause the AI to feel much smarter than top experts. Are you imagining really limiting the parallel compute that can be used (perhaps to one “copy” of the model if those concepts still make sense in the future) and most of the advantages come from increasing parallel compute and coordination of that?
Hm, but human cognitive labor isn’t totally obsolete, because there’s still those tasks where humans are a factor 5 cheaper.
Similarly for the definition in footnote 7: If AIs simultaneously became uniformly 2x cheaper than humans on all the tasks holding current compute prices fixed, then that’s not actually that impressive. Currently compute spend is a small fraction of spend on human cognitive labor. So even if we got AI software that could generate $2 of cognitive labor revenue for every $1 spent on compute, most of the cognitive labor income would still be accruing to humans.
Of course, TED-AI won’t be uniformly cheaper. It’ll have spiky capabilities compared to the human capability distribution. So the impressiveness comes from how, at the point where it’s only 5x more expensive than humans on its least comparative advantaged cognitive task (or 2x cheaper according to extrapolated compute prices on its most expensive task) then surely it’s far superhuman on most things, and humans are totally obsolete on most things.
It does feel kind of off to me that this would correspond to:
I would imagine that the AI would feel much smarter than a top human expert in more than half of domains, but then for there to be a few domains where it both has surprising weaknesses and where those “other advantages” doesn’t help that much.
(Somewhat less relevant point: Probably the AI will also feel very smart because “feel smart” is easier to train for than to actually do a good job in a bunch of domains, so the AI will succeed at that before it’s actually doing a good job.)
A more specific argument to expect spikiness and therefore TED-AI to be vastly superhuman in most areas:
The human brain is much more sample efficient than machine learning.
In some areas, we can’t generate many more samples than what humans can engage with in a lifetime. (E.g.: Forecasting of rare events or results of extremely expensive experiments.) As long as human sample efficiency is better than AI sample efficiency, it will be great to employ humans in these areas.
Once ML beats human sample efficiency, ML will be vastly superhuman in all the areas where there’s vastly more data than what humans could absorb in a lifetime.
Possible defeaters to that argument:
Something about generalization/transfer learning?
Maybe you can use some tricks for sample-efficient learning in the low-data regime that doesn’t transfer to the high-data regime. (E.g. put all the data points in-context and reason about them—this wouldn’t generalize to a number of data points that’s much greater than what fits in the context window.) And therefore beat humans in the low-data regime without necessarily being able to vastly outperform them in the high-data regime.
Maybe I’m just confused about what sense of “feel smart” and what “other advantages” you have in mind such that the advantages helps the AI dominate a bunch of domains but doesn’t cause the AI to feel much smarter than top experts. Are you imagining really limiting the parallel compute that can be used (perhaps to one “copy” of the model if those concepts still make sense in the future) and most of the advantages come from increasing parallel compute and coordination of that?