I think there’s a bit of a modus ponens/modus tollens here, where you’re saying “surely they wouldn’t reallocate time suboptimally!” (so the observed allocation constrains parameters to imply meaningful uplift) while another might conclude “surely the substitutability isn’t that high!” (so the observed allocation must be ‘suboptimal’).
For my part, I think a fair bit (most) of the new code is ‘barely useful’ in the sense that it’s product code, stuff like claude code, cowork, etc. That’s actually ‘rational’ in the sense that it grabs market share and improves retention etc. Further, from individual contributors’ POV, code contribution is at least a bit reinforced since it’s such a legible metric (witness the internal and external headlines reporting on this!) but it’s a Goodhart proxy vs actual progress.
I’m still very unsure where it shakes out, but I really appreciate the key contributions from this analysis, which (to me) are to point out that under CES and some reasonable parameter assumptions, certain other parameters are quite constrained by what’s observed.
I think there’s a bit of a modus ponens/modus tollens here, where you’re saying “surely they wouldn’t reallocate time suboptimally!” (so the observed allocation constrains parameters to imply meaningful uplift) while another might conclude “surely the substitutability isn’t that high!” (so the observed allocation must be ‘suboptimal’).
For my part, I think a fair bit (most) of the new code is ‘barely useful’ in the sense that it’s product code, stuff like claude code, cowork, etc. That’s actually ‘rational’ in the sense that it grabs market share and improves retention etc. Further, from individual contributors’ POV, code contribution is at least a bit reinforced since it’s such a legible metric (witness the internal and external headlines reporting on this!) but it’s a Goodhart proxy vs actual progress.
I’m still very unsure where it shakes out, but I really appreciate the key contributions from this analysis, which (to me) are to point out that under CES and some reasonable parameter assumptions, certain other parameters are quite constrained by what’s observed.