How long would it take (in months) to train a smart recent college graduate with no specialized training in my field to complete this task?
This doesn’t seem like a great metric because there are many tasks that a college grad can do with 0 training that current AI can’t do, including:
Download and play a long video game to completion
Read and summarize a whole book
Spend a month planning an event
I do think that there’s something important about this metric, but I think it’s basically subsumed by my metric: if the task is “spend a month doing novel R&D for lidar”, then my framework predicts that we’ll need 1-month AGI for that. If the task is instead “answer the specific questions about lidar which this expert has been studying”, then I claim that this is overfitting and therefore not a fair comparison; even if you expand it to “questions about lidar in general” there’s probably a bunch of stuff that GPT-4 will know that the expert won’t.
For the t-AGI framework, maybe you should also specify that the human starts the task only knowing things that are written multiple times on the internet. For example, Ed Witten could give snap (1-second) responses to lots of string theory questions that are WAY beyond current AI, using idiosyncratic intuitions he built up over many years. Likewise a chess grandmaster thinking about a board state for 1 second could crush GPT-4 or any other AI that wasn’t specifically and extensively trained on chess by humans.
I feel pretty uncertain about this, actually. Sure, there are some questions that don’t appear at all on the internet, but most human knowledge is, so you’d have to cherry-pick questions. And presumably GPT-4 has also inferred a bunch of intuitions from internet data which weren’t explicitly written down there. In other words: even if this is true, it doesn’t feel centrally relevant.
Sure, there are some questions that don’t appear at all on the internet, but most human knowledge is, so you’d have to cherry-pick questions.
I think you’re saying “there are questions about string theory whose answers are obvious to Ed Witten because he happened to have thought about them in the course of some unpublished project, but these questions are hyper-specific, so bringing them up at all would be unfair cherry-picking.”
But then we could just ask the question: “Can you please pose a question about string theory that no AI would have any prayer of answering, and then answer it yourself?” That’s not cherry-picking, or at least not in the same way.
And it points to an important human capability, namely, figuring out which areas are promising and tractable to explore, and then exploring them. Like, if a human wants to make money or do science or take over the world, then they get to pick, endogenously, which areas or avenues to explore.
But then we could just ask the question: “Can you please pose a question about string theory that no AI would have any prayer of answering, and then answer it yourself?” That’s not cherry-picking, or at least not in the same way.
But can’t we equivalently just ask an AI to pose a question that no human would have a prayer of answering in one second? It wouldn’t even need to be a trivial memorization thing, it could also be a math problem complex enough that humans can’t do it that quickly, or drawing a link between two very different domains of knowledge.
I think the “in one second” would be cheating. The question for Ed Witten didn’t specify “the AI can’t answer it in one second”, but rather “the AI can’t answer it period”. Like, if GPT-4 can’t answer the string theory question in 5 minutes, then it probably can’t answer it in 1000 years either.
Why is it cheating? That seems like the whole point of my framework—that we’re comparing what AIs can do in any amount of time to what humans can do in a bounded amount of time.
Whatever. Maybe I was just jumping on an excuse to chit-chat about possible limitations of LLMs :) And maybe I was thread-hijacking by not engaging sufficiently with your post, sorry.
This part you wrote above was the most helpful for me:
if the task is “spend a month doing novel R&D for lidar”, then my framework predicts that we’ll need 1-month AGI for that
I guess I just want to state my opinion that (1) summarizing a 10,000-page book is a one-month task but could come pretty soon if indeed it’s not already possible, (2) spending a month doing novel R&D for lidar is a one-month task that I think is forever beyond LLMs and would require new algorithmic breakthroughs. That’s not disagreeing with you per se, because you never said in OP that all 1-month human tasks are equally hard for AI and will fall simultaneously! (And I doubt you believe it!) But maybe you conveyed that vibe slightly, from your talk about “coherence over time” etc., and I want to vibe in the opposite direction, by saying that what the human is doing during that month matters a lot, with building-from-scratch and exploring a rich hierarchical interconnected space of novel concepts being a hard-for-AI example, and following a very long fiction plot being an easy-for-AI example (somewhat related to its parallelizability).
Yeah, I agree I convey the implicit prediction that, even though not all one-month tasks will fall at once, they’ll be closer than you would otherwise expect not using this framework.
I think I still disagree with your point, as follows: I agree that AI will soon do passably well at summarizing 10k word books, because the task is not very “sharp”—i.e. you get gradual rather than sudden returns to skill differences. But I think it will take significantly longer for AI to beat the quality of summary produced by a median expert in 1 month, because that expert’s summary will in fact explore a rich hierarchical interconnected space of concepts from the novel (novel concepts, if you will).
This doesn’t seem like a great metric because there are many tasks that a college grad can do with 0 training that current AI can’t do, including:
Download and play a long video game to completion
Read and summarize a whole book
Spend a month planning an event
I do think that there’s something important about this metric, but I think it’s basically subsumed by my metric: if the task is “spend a month doing novel R&D for lidar”, then my framework predicts that we’ll need 1-month AGI for that. If the task is instead “answer the specific questions about lidar which this expert has been studying”, then I claim that this is overfitting and therefore not a fair comparison; even if you expand it to “questions about lidar in general” there’s probably a bunch of stuff that GPT-4 will know that the expert won’t.
I feel pretty uncertain about this, actually. Sure, there are some questions that don’t appear at all on the internet, but most human knowledge is, so you’d have to cherry-pick questions. And presumably GPT-4 has also inferred a bunch of intuitions from internet data which weren’t explicitly written down there. In other words: even if this is true, it doesn’t feel centrally relevant.
Ah, that’s helpful, thanks.
I think you’re saying “there are questions about string theory whose answers are obvious to Ed Witten because he happened to have thought about them in the course of some unpublished project, but these questions are hyper-specific, so bringing them up at all would be unfair cherry-picking.”
But then we could just ask the question: “Can you please pose a question about string theory that no AI would have any prayer of answering, and then answer it yourself?” That’s not cherry-picking, or at least not in the same way.
And it points to an important human capability, namely, figuring out which areas are promising and tractable to explore, and then exploring them. Like, if a human wants to make money or do science or take over the world, then they get to pick, endogenously, which areas or avenues to explore.
But can’t we equivalently just ask an AI to pose a question that no human would have a prayer of answering in one second? It wouldn’t even need to be a trivial memorization thing, it could also be a math problem complex enough that humans can’t do it that quickly, or drawing a link between two very different domains of knowledge.
I think the “in one second” would be cheating. The question for Ed Witten didn’t specify “the AI can’t answer it in one second”, but rather “the AI can’t answer it period”. Like, if GPT-4 can’t answer the string theory question in 5 minutes, then it probably can’t answer it in 1000 years either.
(If the AI can get smarter and smarter, and figure out more and more stuff, without bound, in any domain, by just running it longer and longer, then (1) it would be quite disanalogous to current LLMs [btw I’ve been assuming all along that this post is implicitly imagining something vaguely like current LLMs but I guess you didn’t say that explicitly], (2) I would guess that we’re already past end-of-the-world territory.)
Why is it cheating? That seems like the whole point of my framework—that we’re comparing what AIs can do in any amount of time to what humans can do in a bounded amount of time.
Whatever. Maybe I was just jumping on an excuse to chit-chat about possible limitations of LLMs :) And maybe I was thread-hijacking by not engaging sufficiently with your post, sorry.
This part you wrote above was the most helpful for me:
I guess I just want to state my opinion that (1) summarizing a 10,000-page book is a one-month task but could come pretty soon if indeed it’s not already possible, (2) spending a month doing novel R&D for lidar is a one-month task that I think is forever beyond LLMs and would require new algorithmic breakthroughs. That’s not disagreeing with you per se, because you never said in OP that all 1-month human tasks are equally hard for AI and will fall simultaneously! (And I doubt you believe it!) But maybe you conveyed that vibe slightly, from your talk about “coherence over time” etc., and I want to vibe in the opposite direction, by saying that what the human is doing during that month matters a lot, with building-from-scratch and exploring a rich hierarchical interconnected space of novel concepts being a hard-for-AI example, and following a very long fiction plot being an easy-for-AI example (somewhat related to its parallelizability).
Yeah, I agree I convey the implicit prediction that, even though not all one-month tasks will fall at once, they’ll be closer than you would otherwise expect not using this framework.
I think I still disagree with your point, as follows: I agree that AI will soon do passably well at summarizing 10k word books, because the task is not very “sharp”—i.e. you get gradual rather than sudden returns to skill differences. But I think it will take significantly longer for AI to beat the quality of summary produced by a median expert in 1 month, because that expert’s summary will in fact explore a rich hierarchical interconnected space of concepts from the novel (novel concepts, if you will).