Yep, looks very likely the data centers will be more expensive than this. It’s a bit complicated because Epoch estimates that GPT-4.5 only cost $400 million to train, vs OpenAI’s $7 billion total compute costs in 2024. If the costs of an individual training run are ~5% of an org’s compute budget, rather than more like 50%, that saves the model’s predictions somewhat (though it wasn’t what I had in mind when playing out the scenario, and means the model can’t really say “this is financially feasible” because labs’ total compute costs will be much higher than the stated costs of training).
Yep, looks very likely the data centers will be more expensive than this. It’s a bit complicated because Epoch estimates that GPT-4.5 only cost $400 million to train, vs OpenAI’s $7 billion total compute costs in 2024. If the costs of an individual training run are ~5% of an org’s compute budget, rather than more like 50%, that saves the model’s predictions somewhat (though it wasn’t what I had in mind when playing out the scenario, and means the model can’t really say “this is financially feasible” because labs’ total compute costs will be much higher than the stated costs of training).