The scale of training and R&D spending by AI companies can be reduced on short notice, while global inference buildout costs much more and needs years of use to pay for itself. So an AI slowdown mostly hurts clouds and makes compute cheap due to oversupply, which might be a wash for AI companies. Confusingly major AI companies are closely tied to cloud providers, but OpenAI is distancing itself from Microsoft, and Meta and xAI are not cloud providers, so wouldn’t suffer as much. In any case the tech giants will survive, it’s losing their favor that seems more likely to damage AI companies, making them no longer able to invest as much in R&D.
This is a solid point that I forgot to take into account here.
What happens to GPU clusters inside the data centers build out before the market crash?
If user demand slips and/or various companies stop training, that means that compute prices will slump. As a result, cheap compute will be available for remaining R&D teams, for the three years at least that the GPUs last.
I find that concerning. Because not only is compute cheap, but many of the researchers left using that compute will have reached an understanding that scaling transformer architectures on internet-available data has become a dead end. With investor and managerial pressure to release LLM-based products gone, researchers will explore their own curiosities. This is the time you’d expect the persistent researchers to invent and tinker with new architectures – that could end up being more compute and data efficient at encoding functionality.
~ ~ ~
I don’t want to skip over your main point. Is your argument that AI companies will be protected from a crash, since their core infrastructure is already build?
Or more precisely:
that since data centers were build out before the crash, that compute prices end up converging on mostly just the cost of the energy and operations needed to run the GPU clusters inside,
which in turn acts as a financial cushion for companies like OpenAI and Anthropic, for whom inference costs are now lower,
where those companies can quickly scale back expensive training and R&D, while offering their existing products to remaining users at lower cost.
as a result of which, those companies can continue to operate during the period that funding has dried up, waiting out the ‘AI winter’ until investors and consumers are willing to commit their money again.
That sounds right, given that compute accounts for over half of their costs. Particularly if the companies secure another large VC round ahead of a crash, then they should be able to weather the storm. E.g. the $40 billion just committed to OpenAI (assuming that by the end of this year OpenAI exploits a legal loophole to become for-profit, that their main backer SoftBank can lend enough money, etc).
Just realised that your point seems similar to Sequoia Capital’s: “declining prices for GPU computing is actually good for long-term innovation and good for startups. If my forecast comes to bear, it will cause harm primarily to investors. Founders and company builders will continue to build in AI—and they will be more likely to succeed, because they will benefit both from lower costs and from learnings accrued during this period of experimentation.”
~ ~ ~
A market crash is by itself not enough to deter these companies – from continuing to integrate increasingly automated systems into society.
I think a coordinated movement is needed; one that exerts legitimate pressure on our failing institutions. The next post will be about that.
E.g. the $40 billion just committed to OpenAI (assuming that by the end of this year OpenAI exploits a legal loophole to become for-profit, that their main backer SoftBank can lend enough money, etc).
VC money, in my experience, doesn’t typically mean that the VC writes a check and then the startup has it to do with as they want; it’s typically given out in chunks and often there are provisions for the VC to change their mind if they don’t think it’s going well. This may be different for loans, and it’s possible that a sufficiently hot startup can get the money irrevocably; I don’t know.
The scale of training and R&D spending by AI companies can be reduced on short notice, while global inference buildout costs much more and needs years of use to pay for itself. So an AI slowdown mostly hurts clouds and makes compute cheap due to oversupply, which might be a wash for AI companies. Confusingly major AI companies are closely tied to cloud providers, but OpenAI is distancing itself from Microsoft, and Meta and xAI are not cloud providers, so wouldn’t suffer as much. In any case the tech giants will survive, it’s losing their favor that seems more likely to damage AI companies, making them no longer able to invest as much in R&D.
This is a solid point that I forgot to take into account here.
What happens to GPU clusters inside the data centers build out before the market crash?
If user demand slips and/or various companies stop training, that means that compute prices will slump. As a result, cheap compute will be available for remaining R&D teams, for the three years at least that the GPUs last.
I find that concerning. Because not only is compute cheap, but many of the researchers left using that compute will have reached an understanding that scaling transformer architectures on internet-available data has become a dead end. With investor and managerial pressure to release LLM-based products gone, researchers will explore their own curiosities. This is the time you’d expect the persistent researchers to invent and tinker with new architectures – that could end up being more compute and data efficient at encoding functionality.
~ ~ ~
I don’t want to skip over your main point. Is your argument that AI companies will be protected from a crash, since their core infrastructure is already build?
Or more precisely:
that since data centers were build out before the crash, that compute prices end up converging on mostly just the cost of the energy and operations needed to run the GPU clusters inside,
which in turn acts as a financial cushion for companies like OpenAI and Anthropic, for whom inference costs are now lower,
where those companies can quickly scale back expensive training and R&D, while offering their existing products to remaining users at lower cost.
as a result of which, those companies can continue to operate during the period that funding has dried up, waiting out the ‘AI winter’ until investors and consumers are willing to commit their money again.
That sounds right, given that compute accounts for over half of their costs. Particularly if the companies secure another large VC round ahead of a crash, then they should be able to weather the storm. E.g. the $40 billion just committed to OpenAI (assuming that by the end of this year OpenAI exploits a legal loophole to become for-profit, that their main backer SoftBank can lend enough money, etc).
Just realised that your point seems similar to Sequoia Capital’s:
“declining prices for GPU computing is actually good for long-term innovation and good for startups. If my forecast comes to bear, it will cause harm primarily to investors. Founders and company builders will continue to build in AI—and they will be more likely to succeed, because they will benefit both from lower costs and from learnings accrued during this period of experimentation.”
~ ~ ~
A market crash is by itself not enough to deter these companies – from continuing to integrate increasingly automated systems into society.
I think a coordinated movement is needed; one that exerts legitimate pressure on our failing institutions. The next post will be about that.
VC money, in my experience, doesn’t typically mean that the VC writes a check and then the startup has it to do with as they want; it’s typically given out in chunks and often there are provisions for the VC to change their mind if they don’t think it’s going well. This may be different for loans, and it’s possible that a sufficiently hot startup can get the money irrevocably; I don’t know.