Tom Davidson comments on The nature of LLM algorithmic progress (v2)

[Tom Davidson]

I guess i’m saying that doing that experiments at small scale could very easily cause large inaccuracies, and this is a plausible reason you might be underestimating progress in your 1.1 bucket!