If something is just barely possible today with massive compute, maybe it will be possible with much much less compute very soon (e.g. <1 year).
I don’t think it really matters for the argument whether it’s a “small team” that improves the compute-efficiency, or the original team, or a different big team, or whatever. Just that it happens.
Anyway, is the hypothesis true? I would say, it’s very likely if we’re talking about a pioneering new algorithm, because with pioneering new algorithms, we don’t yet have best practices for parallelization, GPU-acceleration, clever shortcuts, etc. etc. On the other hand, if a known, widely-used algorithm is just barely able to do something on the world’s biggest GPU cluster, then it might take longer before it becomes really easy and cheap for anyone to do that thing. Like, maybe it will take a couple years, instead of <1 year :-P
I think the more important hypothesis is:
If something is just barely possible today with massive compute, maybe it will be possible with much much less compute very soon (e.g. <1 year).
I don’t think it really matters for the argument whether it’s a “small team” that improves the compute-efficiency, or the original team, or a different big team, or whatever. Just that it happens.
Anyway, is the hypothesis true? I would say, it’s very likely if we’re talking about a pioneering new algorithm, because with pioneering new algorithms, we don’t yet have best practices for parallelization, GPU-acceleration, clever shortcuts, etc. etc. On the other hand, if a known, widely-used algorithm is just barely able to do something on the world’s biggest GPU cluster, then it might take longer before it becomes really easy and cheap for anyone to do that thing. Like, maybe it will take a couple years, instead of <1 year :-P