The ‘rank’ doesn’t really matter, you are missing the point. What matters is which cognitive moves were required for the agent to arrive at an answer to those problems and what that allows us to predict about future progress.
Please focus on the specific underlying capabilities instead of assuming I am a “goalpost-mover.” These LLMs are in fact continuing to solve the type of problems I’ve come to expect they will be good at and have so far failed at the type I expect matters even more for AI timelines.
ok, so your model is that AI won’t be able to do capability X until they’re able to do capability Y, and so far the evidence shows that they’re not close to doing capability Y. what exactly are X and Y here?
The ‘rank’ doesn’t really matter, you are missing the point. What matters is which cognitive moves were required for the agent to arrive at an answer to those problems and what that allows us to predict about future progress.
Please focus on the specific underlying capabilities instead of assuming I am a “goalpost-mover.” These LLMs are in fact continuing to solve the type of problems I’ve come to expect they will be good at and have so far failed at the type I expect matters even more for AI timelines.
ok, so your model is that AI won’t be able to do capability X until they’re able to do capability Y, and so far the evidence shows that they’re not close to doing capability Y. what exactly are X and Y here?
It’s a bit difficult to explain quickly, but some of my thoughts on the matter are here.
And here: https://www.lesswrong.com/posts/jXjeYYPXipAtA2zmj/jacquesthibs-s-shortform?commentId=pCiAsJ7NLXrtBymw2