The point of the first two paragraphs was to establish relevance and an estimate for the lowest market price of compute in case of a significant AI slowdown, a level at which some datacenters will still prefer to sell GPU-time rather than stay idle (some owners of datacenters will manage to avoid bankruptcy and will be selling GPU-time even with no hope of recouping capex, as long as it remains at an opex profit, assuming nobody will be willing to buy out their second hand hardware either). So it’s not directly about OpenAI’s datacenter situation, rather it’s a context in which OpenAI might find itself, which is with access to a lot of cheap compute from others.
I’m using “cost of inference” in a narrow sense of cost of running a model at a market price of the necessary compute, with no implications about costs of unfortunate steps taken in pursuit of securing inference capacity, such as buying too much hardware directly. In case of an AI slowdown, I’m assuming that inference compute will remain abundant, so securing the necessary capacity won’t be difficult.
I’m guessing one reason Stargate is an entity separate from OpenAI is to have an option to walk away from it if future finances of OpenAI can’t sustain the hardware Stargate is building, in which case OpenAI might need or want to find compute elsewhere, hence relevance of market prices of compute. Right now they are in for $18bn with Stargate specifically out of $30-40bn they’ve raised (depending on success of converting into a for-profit).
The point of the first two paragraphs was to establish relevance and an estimate for the lowest market price of compute in case of a significant AI slowdown, a level at which some datacenters will still prefer to sell GPU-time rather than stay idle (some owners of datacenters will manage to avoid bankruptcy and will be selling GPU-time even with no hope of recouping capex, as long as it remains at an opex profit, assuming nobody will be willing to buy out their second hand hardware either). So it’s not directly about OpenAI’s datacenter situation, rather it’s a context in which OpenAI might find itself, which is with access to a lot of cheap compute from others.
I’m using “cost of inference” in a narrow sense of cost of running a model at a market price of the necessary compute, with no implications about costs of unfortunate steps taken in pursuit of securing inference capacity, such as buying too much hardware directly. In case of an AI slowdown, I’m assuming that inference compute will remain abundant, so securing the necessary capacity won’t be difficult.
I’m guessing one reason Stargate is an entity separate from OpenAI is to have an option to walk away from it if future finances of OpenAI can’t sustain the hardware Stargate is building, in which case OpenAI might need or want to find compute elsewhere, hence relevance of market prices of compute. Right now they are in for $18bn with Stargate specifically out of $30-40bn they’ve raised (depending on success of converting into a for-profit).