Flowers are selective about what kind of pollinator they attract. Diurnal flowers use diverse colours to stand out in a competition against their neighbours for visual salience. But flowers with nocturnal anthesis are generally white, as they aim only to outshine the night.
O mysterious temple of chance, for what did the scattered parts of your whole link arms to shape you? And the mysterious ceasefire they declared to heed you? O quiescent eye of a hurricane, what holds you together? O molecular weave of water and sunlight, why do you even bother?
Wow, this is a good argument. Especially if assumptions hold.
The ALU computes the input much faster than the results can be moved to the next layer.
So if the AI only receives a single user’s prompt, the ALUs waste a lot of time waiting for input.
But if many users are sending prompts all the time, the ALUs can be sent many more operations at once (assuming the wires are bottlenecked by speed rather than amount of information they can carry).
So if your AI is extremely popular (e.g., OpenAI), your ALUs have to spend less time idling, so the GPUs you use are much more cost-effective.
Compute is much more expensive for less popular AIs (plausibly >1000x).