particlemania comments on Recent AI model progress feels mostly like bullshit

[particlemania]

I expect it matters to the extent we care about whether the generalizing to the new question is taking place in the expensive pretraining phase, or in the active in-context phase.