My guess is neither of you is very good at using them, and getting value out of them somewhat scales with skill.
Models can easily replace on the order of 50% of my coding work these days, and if I have any major task, my guess is I quite reliably get 20%-30% productivity improvements out of them. It does take time to figure out at which things they are good at, and how to prompt them.
I think you’re right, but I rarely hear this take. Probably because “good at both coding and LLMs” is a light tail end of the distribution, and most of the relative value of LLMs in code is located at the other, much heavier end of “not good at coding” or even “good at neither coding nor LLMs”.
(Speaking as someone who didn’t even code until LLMs made it trivially easy, I probably got more relative value than even you.)
Note this 50% likely only holds if you are using a main stream language. For some non-main stream language I have gotten responses that where really unbelivably bad. Things like “the name of this variable wrong” which literally could never be the problem (it was a valid identifier).
And similarly, if you are trying to encode novel concepts, it’s very different from gluing together libraries, or implementing standard well known tasks, which I would guess is what habryka is mostly doing (not that this is a bad thing to do).
My guess is neither of you is very good at using them, and getting value out of them somewhat scales with skill.
Models can easily replace on the order of 50% of my coding work these days, and if I have any major task, my guess is I quite reliably get 20%-30% productivity improvements out of them. It does take time to figure out at which things they are good at, and how to prompt them.
I think you’re right, but I rarely hear this take. Probably because “good at both coding and LLMs” is a light tail end of the distribution, and most of the relative value of LLMs in code is located at the other, much heavier end of “not good at coding” or even “good at neither coding nor LLMs”.
(Speaking as someone who didn’t even code until LLMs made it trivially easy, I probably got more relative value than even you.)
Sounds plausible. Is that 50% of coding work that the LLMs replace of a particular sort, and the other 50% a distinctly different sort?
Note this 50% likely only holds if you are using a main stream language. For some non-main stream language I have gotten responses that where really unbelivably bad. Things like “the name of this variable wrong” which literally could never be the problem (it was a valid identifier).
And similarly, if you are trying to encode novel concepts, it’s very different from gluing together libraries, or implementing standard well known tasks, which I would guess is what habryka is mostly doing (not that this is a bad thing to do).