On the one hand this is an interesting and useful piece of data on AI scaling and the progress of algorithms. It’s also important because it makes the point that the very notion of “progress of algorithms” implies hardware overhang as important as >10 years of Moore’s law. I also enjoyed the follow-up work that this spawned in 2021.
On the one hand this is an interesting and useful piece of data on AI scaling and the progress of algorithms. It’s also important because it makes the point that the very notion of “progress of algorithms” implies hardware overhang as important as >10 years of Moore’s law. I also enjoyed the follow-up work that this spawned in 2021.