I think an issue with the definition of TED-AI is that there are lots of tasks that benefit a ton from detailed experience and knowledge that AIs won’t have by default. For example, there’s probably a bunch of engineers deep in the compute supply chain who have very detailed knowledge and experience with managing their particular operation. I think we’ll spend an important time in the intelligence explosion where AIs would be able to automate their job given, say, a month of learning and assistance from them, but couldn’t do it on their own. This could really delay the point of “TED-AI” far beyond the point where “The AI generally feels roughly as smart as a top human expert” and where I’ll feel like AIs are crushing humans on all the important tasks. (Because for all the most important tasks, people will have put in the work to make AIs dominate humans on those tasks. And it doesn’t matter much that there’s a bunch of tasks in the old human economy that hasn’t been worth that attention yet.)
(IIRC, Tom Davidson’s take-off speed model tries to get around something similar by talking about what tasks are “readily” automatable, meaning “i) it would be profitable for organisations to do the engineering and workflow adjustments necessary for AI to perform the task in practice, and ii) they could make these adjustments within 1 year if they made this one of their priorities.” I think the 1 year thing sounds too slow though, given that we want to track milestones that may be only months apart. Also, I’m not sure we should condition on it being profitable for orgs to do this given that the opp-cost of AI labor might be really high at the time and it seems pretty fine and cheap to just have humans keep doing some of their old jobs for a bit longer, if they’re not in a bottlenecking part of the economy.)
When I use the term TED-AI these days, I mean “you’d strictly prefer to hire the AI over any human expert putting aside narrow experience that’s very specific to this job in important domains like mechanical engineering, drone R&D, etc (like the AI dominates putting aside this narrow expertise)”. So I do mean something more like readily automatable. The definition I gave in this post does underspecify this.
I think an issue with the definition of TED-AI is that there are lots of tasks that benefit a ton from detailed experience and knowledge that AIs won’t have by default. For example, there’s probably a bunch of engineers deep in the compute supply chain who have very detailed knowledge and experience with managing their particular operation. I think we’ll spend an important time in the intelligence explosion where AIs would be able to automate their job given, say, a month of learning and assistance from them, but couldn’t do it on their own. This could really delay the point of “TED-AI” far beyond the point where “The AI generally feels roughly as smart as a top human expert” and where I’ll feel like AIs are crushing humans on all the important tasks. (Because for all the most important tasks, people will have put in the work to make AIs dominate humans on those tasks. And it doesn’t matter much that there’s a bunch of tasks in the old human economy that hasn’t been worth that attention yet.)
(IIRC, Tom Davidson’s take-off speed model tries to get around something similar by talking about what tasks are “readily” automatable, meaning “i) it would be profitable for organisations to do the engineering and workflow adjustments necessary for AI to perform the task in practice, and ii) they could make these adjustments within 1 year if they made this one of their priorities.” I think the 1 year thing sounds too slow though, given that we want to track milestones that may be only months apart. Also, I’m not sure we should condition on it being profitable for orgs to do this given that the opp-cost of AI labor might be really high at the time and it seems pretty fine and cheap to just have humans keep doing some of their old jobs for a bit longer, if they’re not in a bottlenecking part of the economy.)
When I use the term TED-AI these days, I mean “you’d strictly prefer to hire the AI over any human expert putting aside narrow experience that’s very specific to this job in important domains like mechanical engineering, drone R&D, etc (like the AI dominates putting aside this narrow expertise)”. So I do mean something more like readily automatable. The definition I gave in this post does underspecify this.