If we’re going to evaluate AI’s impact on software engineering, or really any industry, we need to measure throughput. I suppose with code this could be something like unaltered lines that go directly from AI to prod, but even that might not capture the effectiveness of the AI vs the effectiveness of the prompt writer.
I have no doubt it’s been a boon for research (it has helped me do rapid prototyping), but we do need some better measures of its productivity. Right now all we have are high-level proxies like labor market impacts and low-level noise like volume of code.
If we’re going to evaluate AI’s impact on software engineering, or really any industry, we need to measure throughput. I suppose with code this could be something like unaltered lines that go directly from AI to prod, but even that might not capture the effectiveness of the AI vs the effectiveness of the prompt writer.
I have no doubt it’s been a boon for research (it has helped me do rapid prototyping), but we do need some better measures of its productivity. Right now all we have are high-level proxies like labor market impacts and low-level noise like volume of code.