The acceleration of the work as a whole is not determined by the mean of the accelerations experienced by individual employees. If only the tightest bottleneck widens by 4x, that means you go roughly as fast as the second tightest bottleneck is wide, not 4x faster. So long as there is any bottleneck that isn’t widened and that’s less than 4x as wide as the former tightest bottleneck, the work as a whole will be sped up by less than 4x. It would be entirely possible for many or most employees to experience >4x speedup without the overall org moving all that much faster.[1]
Additionally, this continues at the individual level. in my experience, if you ask people how much speedup they got from a major new model after they just got their hands on it, there’s some tendency for them to think about the tasks that used to occupy a lot of their time and that the model just sped up massively when giving their estimate, and not yet really think about the tasks the model didn’t speed up massively and that are now the new bottleneck in their workflow.
kinda agree, but a consideration worth noting: if your company currently carries out process by spending of work on tasks of type and of work on tasks of type , then if doing type stuff gets sped up while type stuff isn’t sped up, Amdahl’s law style reasoning like what you say in your comment would give that you get a roughly speedup, but really you can quite plausibly get like a speedup because in reality [a sufficient quantity of type work can partly substitute for type work in pushing forward] / [it wasn’t really necessary to do the type work, just good to do it at the previous relative speeds]. (the overall speedup number will of course depend on specifics of the example.) e.g.:
if algo research is sped up 1000x but compute buildup isn’t sped up, I think you will still have AI progress for some time even though in the past the two might have contributed similarly to AI progress
maybe: if high-iq algo research isn’t sped up much but kinda dumb algo research tasks are sped up and previously these contributed equally to AI progress, you could still get a speedup on AI progress
so, i think what you’re saying is technically true for things which are really bottlenecks — like, in the sense that you will really have to keep doing the same amount of the same thing later for each unit of AI progress — but i’m concerned about various things one would want to apply this thinking to not actually being bottlenecks in that sense
I agree in general, but presumably this would result in the company redistributing resources towards whatever the now-most-critical-bottleneck activities are? Maybe that’s impossible for humans at the current pace of AI development (organizations are usually not this responsive and adaptable)? Alternatively, couldn’t accelerating the acceleratable activities plausibly cause bigger gains per model iteration in ways that might subsequently loosen remaining bottlenecks?
The acceleration of the work as a whole is not determined by the mean of the accelerations experienced by individual employees. If only the tightest bottleneck widens by 4x, that means you go roughly as fast as the second tightest bottleneck is wide, not 4x faster. So long as there is any bottleneck that isn’t widened and that’s less than 4x as wide as the former tightest bottleneck, the work as a whole will be sped up by less than 4x. It would be entirely possible for many or most employees to experience >4x speedup without the overall org moving all that much faster.[1]
Additionally, this continues at the individual level. in my experience, if you ask people how much speedup they got from a major new model after they just got their hands on it, there’s some tendency for them to think about the tasks that used to occupy a lot of their time and that the model just sped up massively when giving their estimate, and not yet really think about the tasks the model didn’t speed up massively and that are now the new bottleneck in their workflow.
Yes, they take a geometric mean rather than an arithmetic mean. I still don’t buy it.
kinda agree, but a consideration worth noting: if your company currently carries out process
if algo research is sped up 1000x but compute buildup isn’t sped up, I think you will still have
maybe: if high-iq algo research isn’t sped up much but kinda dumb algo research tasks are sped up
so, i think what you’re saying is technically true for things which are really bottlenecks — like, in the sense that you will really have to keep doing the same amount of the same thing later for each unit of AI progress — but i’m concerned about various things one would want to apply this thinking to not actually being bottlenecks in that sense
I agree in general, but presumably this would result in the company redistributing resources towards whatever the now-most-critical-bottleneck activities are? Maybe that’s impossible for humans at the current pace of AI development (organizations are usually not this responsive and adaptable)? Alternatively, couldn’t accelerating the acceleratable activities plausibly cause bigger gains per model iteration in ways that might subsequently loosen remaining bottlenecks?
I agree with this, but think this is unlikely to be the main reason the overall serial labor acceleration is smaller.