Fair enough, I reworded slightly. (Although I’d be pretty surprised if LLM algorithmic progress were way faster today than in, say, 2020, given that there was presumably much more low-hanging fruit in 2020.)
This might be some sort of crux between you and him, where he might point to growth in researcher focus/headcount as well as perhaps some uplift from AI by now. (I might endorse the first of those a bit, and probably not the second, thought it’s conceptually reasonable and not out of the question.)
FWIW, I would not be surprised if LLM algorithmic progress was considerably faster today than in 2020. Per my recent post, I think catch-up algorithmic progress is very fast today, and it seems like it wasn’t so fast a few years ago.
Fair enough, I reworded slightly. (Although I’d be pretty surprised if LLM algorithmic progress were way faster today than in, say, 2020, given that there was presumably much more low-hanging fruit in 2020.)
This might be some sort of crux between you and him, where he might point to growth in researcher focus/headcount as well as perhaps some uplift from AI by now. (I might endorse the first of those a bit, and probably not the second, thought it’s conceptually reasonable and not out of the question.)
FWIW, I would not be surprised if LLM algorithmic progress was considerably faster today than in 2020. Per my recent post, I think catch-up algorithmic progress is very fast today, and it seems like it wasn’t so fast a few years ago.