Assume that all that doubling compute does is double labor
This isn’t what I want because: (1) doubling parallel labor is extremely conservative and (2) doubling overall labor effort is both probably conservative and also totally unprincipled, like why not 10x? (Tt will depend on returns from training compute and experiment of more compue right?)
It doesn’t compound further bc there’s no feedback loop to more experiments like there is to more labor
I don’t understand, the whole point of the experiments is to get us better labor.
I don’t understand, the whole point of the experiments is to get us better labor.
yeah, so they do—a doubling of cumulative experiments drives 0.7 doublings of software. And then that better software does more cognitive work to improve software further still. But it doesn’t increase the amount of compute available for experiments, so the feedback loop doesn’t go full circle.
For “cognitive labour” we have: more compute → more cog labour → beter software → more cog labour → better software… So you get and initial boost from extra compute which then ratchets up with the software feedback loop
But for “experimental compute” we have: more compute → more experiments → better software → more cog labour → better software...
So while it feeds into the software feedback loop, it doesn’t loop back to “more experimental compute”.
I’m not sure if my math full priced this in. I’ll think about that more.
This isn’t what I want because: (1) doubling parallel labor is extremely conservative and (2) doubling overall labor effort is both probably conservative and also totally unprincipled, like why not 10x? (Tt will depend on returns from training compute and experiment of more compue right?)
I don’t understand, the whole point of the experiments is to get us better labor.
yeah i adjust for this in my other comment
yeah, so they do—a doubling of cumulative experiments drives 0.7 doublings of software. And then that better software does more cognitive work to improve software further still. But it doesn’t increase the amount of compute available for experiments, so the feedback loop doesn’t go full circle.
For “cognitive labour” we have: more compute → more cog labour → beter software → more cog labour → better software… So you get and initial boost from extra compute which then ratchets up with the software feedback loop
But for “experimental compute” we have: more compute → more experiments → better software → more cog labour → better software...
So while it feeds into the software feedback loop, it doesn’t loop back to “more experimental compute”.
I’m not sure if my math full priced this in. I’ll think about that more.