I mean, beating a chess engine in 2005 might be a “years-long task” for a human? The time METR is measuring is how long it would hypothetically take a human to do the task, not how long it takes the AI.
And even if—they try to argue in the other direction: If it takes the human time X at time T it will take the AI duration L. That didn’t work for chess either.
I mean, beating a chess engine in 2005 might be a “years-long task” for a human? The time METR is measuring is how long it would hypothetically take a human to do the task, not how long it takes the AI.
Yes, but it didn’t mean that AIs could do all kinds of long tasks in 2005. And that is the conclusion many people seem to draw from the METR paper.
No? It means you can’t beat the chess engine.
And even if—they try to argue in the other direction: If it takes the human time X at time T it will take the AI duration L. That didn’t work for chess either.