I just want to make it clear that both are paper and Epoch’s paper addresses innovations that occur from 2012-2023 (and only the first half of 2023). We are aware of MLA, muon optimizer, long context unlocks, and RL, and think they are important contributors. However, all these innovations are explicitly outside of the scope of our current paper which seeks to account for Epoch’s estimates in that time period.
I just want to make it clear that both are paper and Epoch’s paper addresses innovations that occur from 2012-2023 (and only the first half of 2023). We are aware of MLA, muon optimizer, long context unlocks, and RL, and think they are important contributors. However, all these innovations are explicitly outside of the scope of our current paper which seeks to account for Epoch’s estimates in that time period.