Those numbers seem radically low. I’ve been using Codex and enjoying it a lot. I would expect a somewhat higher percentage of my code coming from it, and a much higher adoption rate among programmers like me.
Developers can be very picky and famously autistic, especially when it comes to holy wars (hence the need for mandates about permitted languages or resorts to tools like gofmt). So I’m not surprised if even very good things are hard to spread—the individual differences in programming skills seem to be much larger than the impact of most tooling tweaks, impeding evolution. (Something something Planck.) But so far on Twitter and elsewhere with Googlers, I’ve only seen praise for how well the tool works, including crusty IDE-hating old-timers like Christian Szegedy (which is a little surprising), so we’ll see how it spreads.
(There might also be limited availability. The blog post is unclear on whether this group of 10k is a limited-access group, or if everyone has access to it but all of them except the 10k haven’t bothered/disabled it. If so, then if the users accept about a quarter of completions and a completion is roughly each line, and there’s at least 100k coders at Google, then presumably that implies that, even without any quality improvement, it could go to at least a quarter of all new Google code as adoption increases?)
It seems to me that the proportion will take a while to move up much further from 3% since that 3% is low hanging fruit. But considering the last 3 years of AI improvement e.g. GPT-3, it would make sense for that proportion to be vastly higher than 3% in 10-20 years.
Well, speaking of Google and code writing and slippery slopes and finding out later, did you know Google has apparently for a while been plugging Codex-style models into their editors and now “With 10k+ Google-internal developers using the completion setup in their IDE, we measured a user acceptance rate of 25-34%...3% of new code (measured in characters) is now generated from accepting ML completion suggestions.”?
Those numbers seem radically low. I’ve been using Codex and enjoying it a lot. I would expect a somewhat higher percentage of my code coming from it, and a much higher adoption rate among programmers like me.
Developers can be very picky and famously autistic, especially when it comes to holy wars (hence the need for mandates about permitted languages or resorts to tools like
gofmt
). So I’m not surprised if even very good things are hard to spread—the individual differences in programming skills seem to be much larger than the impact of most tooling tweaks, impeding evolution. (Something something Planck.) But so far on Twitter and elsewhere with Googlers, I’ve only seen praise for how well the tool works, including crusty IDE-hating old-timers like Christian Szegedy (which is a little surprising), so we’ll see how it spreads.(There might also be limited availability. The blog post is unclear on whether this group of 10k is a limited-access group, or if everyone has access to it but all of them except the 10k haven’t bothered/disabled it. If so, then if the users accept about a quarter of completions and a completion is roughly each line, and there’s at least 100k coders at Google, then presumably that implies that, even without any quality improvement, it could go to at least a quarter of all new Google code as adoption increases?)
Oh, so AI already writes code that Google uses. Amazing.
It seems to me that the proportion will take a while to move up much further from 3% since that 3% is low hanging fruit. But considering the last 3 years of AI improvement e.g. GPT-3, it would make sense for that proportion to be vastly higher than 3% in 10-20 years.