I think neither one year ago nor now nor anytime soon will “LLMs improve productivity” have a clear binary answer. It’s a jagged landscape that depends on things like
The domain
The concrete task
The human’s experience
How they’re using the LLM
How conscientious they are in the process
...and 50 other things
And with every year, a larger part of this landscape comes up positive for LLMs.
Ironically, I can imagine that part of what leads to the relative improvement of LLM performance is that (some) humans, due to extended LLM use, have seen their skills atrophy, and hence the bar was lowered. I think it’s very difficult to use such technology without “moving backward”, as John has put it.
I think neither one year ago nor now nor anytime soon will “LLMs improve productivity” have a clear binary answer. It’s a jagged landscape that depends on things like
The domain
The concrete task
The human’s experience
How they’re using the LLM
How conscientious they are in the process
...and 50 other things
And with every year, a larger part of this landscape comes up positive for LLMs.
Ironically, I can imagine that part of what leads to the relative improvement of LLM performance is that (some) humans, due to extended LLM use, have seen their skills atrophy, and hence the bar was lowered. I think it’s very difficult to use such technology without “moving backward”, as John has put it.