To me “Insight vs engineering” seems like a relatively weak factor, because I don’t see much of a principled argument for a continued differential. (Whereas, there are very principled arguments for relevance of feedback quality and data availability.)
In a lot of contexts it seems true that current LLMs tend to be better at “mindless implementation”. But mindless engineering is also easier in some sense. E.g., in coding the relatively mindless part seems easier than coming up with great UI design ideas or model architectures or whatever. Also, in some contexts the models do seem somewhat creative to me. (Perhaps a bit hard to assess, because they also know so much more than humans...)
In verifiable domains, you might even think they’re effectively more creative because they can cheaply pursue lots of approaches in parallel, including very long shot approaches. For instance, I’d expect LLMs to come up with lots of “creative” mathematical results/proofs/counterexamples soon because they can pursue approaches that have a one in a million chance of working and would take a human a week to pursue. (I assume human mathematicians mostly wouldn’t pursue one-in-a-million chances of proving even P!=NP if it takes them a week to check whether the approach works.)
To me “Insight vs engineering” seems like a relatively weak factor, because I don’t see much of a principled argument for a continued differential. (Whereas, there are very principled arguments for relevance of feedback quality and data availability.)
In a lot of contexts it seems true that current LLMs tend to be better at “mindless implementation”. But mindless engineering is also easier in some sense. E.g., in coding the relatively mindless part seems easier than coming up with great UI design ideas or model architectures or whatever. Also, in some contexts the models do seem somewhat creative to me. (Perhaps a bit hard to assess, because they also know so much more than humans...)
In verifiable domains, you might even think they’re effectively more creative because they can cheaply pursue lots of approaches in parallel, including very long shot approaches. For instance, I’d expect LLMs to come up with lots of “creative” mathematical results/proofs/counterexamples soon because they can pursue approaches that have a one in a million chance of working and would take a human a week to pursue. (I assume human mathematicians mostly wouldn’t pursue one-in-a-million chances of proving even P!=NP if it takes them a week to check whether the approach works.)