Current AI algorithms are highly parallelizable for training (though obviously not perfectly parallelizable). This would be more important for inference speed, but at least you can do 2x faster inference with 4x more compute at current margins. So, it’s as good as somewhere between 2x and 4x serially faster compute, I’d guess a bit over 3x. Another relevant point here: we seem to have ended up in a paradigm that’s very good at absorbing parallel training compute relative to benefiting from serial speed at the same FLOP count, so more parallel compute is pretty reasonable (like we already needed to have ~extremely parallelizable training/experiment algorithms for anything to work at all).
Current AI algorithms are highly parallelizable for training (though obviously not perfectly parallelizable). This would be more important for inference speed, but at least you can do 2x faster inference with 4x more compute at current margins. So, it’s as good as somewhere between 2x and 4x serially faster compute, I’d guess a bit over 3x. Another relevant point here: we seem to have ended up in a paradigm that’s very good at absorbing parallel training compute relative to benefiting from serial speed at the same FLOP count, so more parallel compute is pretty reasonable (like we already needed to have ~extremely parallelizable training/experiment algorithms for anything to work at all).