For the 2024 prediction “So, the most compute spent on a single training run is something like 5x10^25 FLOPs.” you cite v3 as having been trained on 3.5e24 FLOP, but that is outside an OOM. Whereas Grok-2 was trained in 2024 with 3e25, so seems to be a better model to cite?
Nice!
For the 2024 prediction “So, the most compute spent on a single training run is something like 5x10^25 FLOPs.” you cite v3 as having been trained on 3.5e24 FLOP, but that is outside an OOM. Whereas Grok-2 was trained in 2024 with 3e25, so seems to be a better model to cite?
That’s a much better source, I’ve updated the spreadsheet accordingly, thanks!