A possible explanation for this phenomenon that feels somewhat natural with hindsight: there’s a relatively large minimum amount of compute required to get certain kinds of capabilities working at all. But once you’re above that minimum, you have lots of options: you can continue to scale things up, if you know how to scale them and you have the compute resources to do so, or you can look for algorithmic improvements which enable you to do more and get better results with less compute. Once you’re at this point, perhaps the main determiner of the relative rate of progress between algorithms and compute is which option researchers at the capabilities frontier choose to work on.
A possible explanation for this phenomenon that feels somewhat natural with hindsight: there’s a relatively large minimum amount of compute required to get certain kinds of capabilities working at all. But once you’re above that minimum, you have lots of options: you can continue to scale things up, if you know how to scale them and you have the compute resources to do so, or you can look for algorithmic improvements which enable you to do more and get better results with less compute. Once you’re at this point, perhaps the main determiner of the relative rate of progress between algorithms and compute is which option researchers at the capabilities frontier choose to work on.