You seem to be assuming that there’s not significant overhead or delays from negotiating leases, entering bankruptcy, or dealing with specialized hardware, which is very plausibly false.
If nobody is buying new datacenter GPU’s, that will cut GPU progress to ~zero or negative (because production is halted and implicit knowledge is lost). (It will also probably damage broader semiconductor progress.)
This proportionally reduces cost of inference (and also of training).
This reduces the cost to rent a GPU-hour, but it doesn’t reduce the cost to the owner. (OpenAI, and every frontier lab but Anthropic, will own much or all[1] of their own compute. So this doesn’t do much to help OpenAI in particular.)
I think you have a misconception about accounting. GPU depreciation is considered on an income statement, it is part of the operating expenses, subtracted from gross profit to get net profit. Depreciation due to obsolescence vs. breakdowns isn’t treated differently. If OpenAI drops its prices below the level needed to pay for that depreciation, they won’t be running a (net) profit. Since they won’t be buying new GPU’s, they will die in a few years, once their existing stock of GPU’s breaks down or becomes obsolete. To phrase it another way, if you reduce GPU-time prices 3-5x, the global AI compute buildout has not in fact paid for itself.
OpenAI has deals with CoreWeave and Azure; they may specify fixed prices; even if not, CoreWeave’s independence doesn’t matter here, as they also need to make enough money to buy new GPU’s/repay debt. (Azure is less predictable.)
The point of the first two paragraphs was to establish relevance and an estimate for the lowest market price of compute in case of a significant AI slowdown, a level at which some datacenters will still prefer to sell GPU-time rather than stay idle (some owners of datacenters will manage to avoid bankruptcy and will be selling GPU-time even with no hope of recouping capex, as long as it remains at an opex profit, assuming nobody will be willing to buy out their second hand hardware either). So it’s not directly about OpenAI’s datacenter situation, rather it’s a context in which OpenAI might find itself, which is with access to a lot of cheap compute from others.
I’m using “cost of inference” in a narrow sense of cost of running a model at a market price of the necessary compute, with no implications about costs of unfortunate steps taken in pursuit of securing inference capacity, such as buying too much hardware directly. In case of an AI slowdown, I’m assuming that inference compute will remain abundant, so securing the necessary capacity won’t be difficult.
I’m guessing one reason Stargate is an entity separate from OpenAI is to have an option to walk away from it if future finances of OpenAI can’t sustain the hardware Stargate is building, in which case OpenAI might need or want to find compute elsewhere, hence relevance of market prices of compute. Right now they are in for $18bn with Stargate specifically out of $30-40bn they’ve raised (depending on success of converting into a for-profit).
You seem to be assuming that there’s not significant overhead or delays from negotiating leases, entering bankruptcy, or dealing with specialized hardware, which is very plausibly false.
If nobody is buying new datacenter GPU’s, that will cut GPU progress to ~zero or negative (because production is halted and implicit knowledge is lost). (It will also probably damage broader semiconductor progress.)
This reduces the cost to rent a GPU-hour, but it doesn’t reduce the cost to the owner. (OpenAI, and every frontier lab but Anthropic, will own much or all[1] of their own compute. So this doesn’t do much to help OpenAI in particular.)
I think you have a misconception about accounting. GPU depreciation is considered on an income statement, it is part of the operating expenses, subtracted from gross profit to get net profit. Depreciation due to obsolescence vs. breakdowns isn’t treated differently. If OpenAI drops its prices below the level needed to pay for that depreciation, they won’t be running a (net) profit. Since they won’t be buying new GPU’s, they will die in a few years, once their existing stock of GPU’s breaks down or becomes obsolete. To phrase it another way, if you reduce GPU-time prices 3-5x, the global AI compute buildout has not in fact paid for itself.
OpenAI has deals with CoreWeave and Azure; they may specify fixed prices; even if not, CoreWeave’s independence doesn’t matter here, as they also need to make enough money to buy new GPU’s/repay debt. (Azure is less predictable.)
The point of the first two paragraphs was to establish relevance and an estimate for the lowest market price of compute in case of a significant AI slowdown, a level at which some datacenters will still prefer to sell GPU-time rather than stay idle (some owners of datacenters will manage to avoid bankruptcy and will be selling GPU-time even with no hope of recouping capex, as long as it remains at an opex profit, assuming nobody will be willing to buy out their second hand hardware either). So it’s not directly about OpenAI’s datacenter situation, rather it’s a context in which OpenAI might find itself, which is with access to a lot of cheap compute from others.
I’m using “cost of inference” in a narrow sense of cost of running a model at a market price of the necessary compute, with no implications about costs of unfortunate steps taken in pursuit of securing inference capacity, such as buying too much hardware directly. In case of an AI slowdown, I’m assuming that inference compute will remain abundant, so securing the necessary capacity won’t be difficult.
I’m guessing one reason Stargate is an entity separate from OpenAI is to have an option to walk away from it if future finances of OpenAI can’t sustain the hardware Stargate is building, in which case OpenAI might need or want to find compute elsewhere, hence relevance of market prices of compute. Right now they are in for $18bn with Stargate specifically out of $30-40bn they’ve raised (depending on success of converting into a for-profit).