It feels like this sort of thing is better than nothing, but I wish we had something better.
The existing epoch paper is pretty good, but doesn’t directly target LLMs in a way which seems somewhat sad.
The thing I’d be most excited about is:
Epoch does an in depth investigation using an estimation methodology which is directly targeting LLMs (rather than looking at returns in some other domains).
They use public data and solicit data from companies about algorithmic improvement, head count, compute on experiments etc.
(Some) companies provide this data. Epoch potentially doesn’t publish this exact data and instead just publishes the results of the final analysis to reduce capabilities externalities. (IMO, companies are somewhat unlikely to do this, but I’d like to be proven wrong!)
The existing epoch paper is pretty good, but doesn’t directly target LLMs in a way which seems somewhat sad.
The thing I’d be most excited about is:
Epoch does an in depth investigation using an estimation methodology which is directly targeting LLMs (rather than looking at returns in some other domains).
They use public data and solicit data from companies about algorithmic improvement, head count, compute on experiments etc.
(Some) companies provide this data. Epoch potentially doesn’t publish this exact data and instead just publishes the results of the final analysis to reduce capabilities externalities. (IMO, companies are somewhat unlikely to do this, but I’d like to be proven wrong!)