Thanks for your input. This is one of our concerns, i.e. “utilization optimization makes more compute available, without moving the roof-line, and hence go under the radar”. The deployment factor plays a big role in improving utilization, hence it is one of the layers we want to look at. ”I wonder if that 50% could even end up being used by rogue workloads”—I am a bit unclear on how this might work. Do you have any thoughts to share on possible scenarios?
I meant that there’s a large fraction of the capacity that is up for the taking by a sophisticated actor. E.g., see the PipeFill paper.
In that paper, that actor is the original operator optimizing their workloads. But a different, undeclared workload could be injected to take advantage of the unused hardware: from cryptomining, to training or inference.
I hesitate to suggest scenarios, but I’d imagine that a rogue AI’s first need would be to find hardware to run on. One would think that available GPUs/TPUs/etc aren’t easy to find; but turns out that there’s ~50% of them waiting to be used, and already collocated with the AI!, if only it manages to deal with the complexities.
Thanks for your input.
This is one of our concerns, i.e. “utilization optimization makes more compute available, without moving the roof-line, and hence go under the radar”. The deployment factor plays a big role in improving utilization, hence it is one of the layers we want to look at.
”I wonder if that 50% could even end up being used by rogue workloads”—I am a bit unclear on how this might work. Do you have any thoughts to share on possible scenarios?
I meant that there’s a large fraction of the capacity that is up for the taking by a sophisticated actor. E.g., see the PipeFill paper.
In that paper, that actor is the original operator optimizing their workloads. But a different, undeclared workload could be injected to take advantage of the unused hardware: from cryptomining, to training or inference.
I hesitate to suggest scenarios, but I’d imagine that a rogue AI’s first need would be to find hardware to run on. One would think that available GPUs/TPUs/etc aren’t easy to find; but turns out that there’s ~50% of them waiting to be used, and already collocated with the AI!, if only it manages to deal with the complexities.
This is a very good point. I am studying the paper, and will incorporate it into our upcoming work. Thanks for bringing it up.