This. LLMs make it much easier to write one-off scripts and custom tools for a bunch of stuff like plotting and visualizations. Because the cost is lower you produce much more of this stuff. But this doesn’t mean that AI is contributing 90% of the value. The ideal metric would be something like “how much longer would it take to ship Claude n+1 if we didn’t use Claude n for coding”?
I think a more traditional software company would be a much better measuring stick than Anthropic. Then your metric could be something closer to “lines of production code committed” and that code would account for most of the meaningful coding work done by the company (versus an AI dev company where a lot of the effort goes into experiments, training, data analysis, and other non-production code). Though, of course, “90% of the code written by AI” still wouldn’t mean that the AI did 90% of the work, since the humans would probably do the hardest parts and also supervise the AI and check its output.
This. LLMs make it much easier to write one-off scripts and custom tools for a bunch of stuff like plotting and visualizations. Because the cost is lower you produce much more of this stuff. But this doesn’t mean that AI is contributing 90% of the value. The ideal metric would be something like “how much longer would it take to ship Claude n+1 if we didn’t use Claude n for coding”?
I think a more traditional software company would be a much better measuring stick than Anthropic. Then your metric could be something closer to “lines of production code committed” and that code would account for most of the meaningful coding work done by the company (versus an AI dev company where a lot of the effort goes into experiments, training, data analysis, and other non-production code). Though, of course, “90% of the code written by AI” still wouldn’t mean that the AI did 90% of the work, since the humans would probably do the hardest parts and also supervise the AI and check its output.