Here’s an evalution claiming it isn’t even true at Anthropic in terms of lines of code (I haven’t checked independently)
The kind of thing I don’t want to count is that, for example, it would be easy to write a script that generates many many lines of code. It’s just not interesting to count “how are most lines of code written” globally. “Code actually merged at particular big tech companies” is much closer and I expect that was less than 90 percent at the time.
It’s just not interesting to count “how are most lines of code written” globally. “Code actually merged at particular big tech companies” is much closer and I expect that was less than 90 percent at the time.
Yeah I definitely agree with the first sentence, and I was assuming some kind of sensible qualifier like the one you suggest in the second. From a quick look at the transcript, I don’t think it’s clear precisely what he meant, but (confirming the impression given by the clip you linked) the context was “the job side of this”, so I’d guess something like “90% of economically useful code”.
Here’s an evalution claiming it isn’t even true at Anthropic in terms of lines of code
Thanks for the link. I’m a bit sceptical of the claim that “interpret[ing] the prediction as being about Anthropic itself” is the charitable read; I would expect AI usage, as a proportion of lines written, to be highest at companies doing relatively boring, unoriginal work with lots of boilerplate and minor variations on established patterns (CRUD apps etc.) and much lower at companies that are creating something truly new. But of course that’s offset by the fact that Anthropic has extra reasons to use its own product and the lowest possible barrier to doing so. Ultimately, I do think Amodei was probably wrong, but I wouldn’t be surprised if the true figure was/is pretty high.
Here’s an evalution claiming it isn’t even true at Anthropic in terms of lines of code (I haven’t checked independently)
The kind of thing I don’t want to count is that, for example, it would be easy to write a script that generates many many lines of code. It’s just not interesting to count “how are most lines of code written” globally. “Code actually merged at particular big tech companies” is much closer and I expect that was less than 90 percent at the time.
Yeah I definitely agree with the first sentence, and I was assuming some kind of sensible qualifier like the one you suggest in the second. From a quick look at the transcript, I don’t think it’s clear precisely what he meant, but (confirming the impression given by the clip you linked) the context was “the job side of this”, so I’d guess something like “90% of economically useful code”.
Thanks for the link. I’m a bit sceptical of the claim that “interpret[ing] the prediction as being about Anthropic itself” is the charitable read; I would expect AI usage, as a proportion of lines written, to be highest at companies doing relatively boring, unoriginal work with lots of boilerplate and minor variations on established patterns (CRUD apps etc.) and much lower at companies that are creating something truly new. But of course that’s offset by the fact that Anthropic has extra reasons to use its own product and the lowest possible barrier to doing so. Ultimately, I do think Amodei was probably wrong, but I wouldn’t be surprised if the true figure was/is pretty high.