See footnote 2 (the assumed pretraining time is 3 months). Some sort of assumption is necessary to talk about the size of pretraining runs at all. Pretraining can take longer in principle, but things go wrong (so it’s important to have enough time for another attempt), and it’s not yet a settled manufacturing procedure when new levels of scale get attempted for the first time, using new hardware and new low-level code. In 2035 there might be pretraining runs that take a year, but right now the largest pretraining runs probably take even less than 3 months.
See footnote 2 (the assumed pretraining time is 3 months). Some sort of assumption is necessary to talk about the size of pretraining runs at all. Pretraining can take longer in principle, but things go wrong (so it’s important to have enough time for another attempt), and it’s not yet a settled manufacturing procedure when new levels of scale get attempted for the first time, using new hardware and new low-level code. In 2035 there might be pretraining runs that take a year, but right now the largest pretraining runs probably take even less than 3 months.
Aha. I had missed that footnote; my apologies.