Whenever I send an LLM some query I expect to be able to answer myself (instead of requesting a primer on some unknown-to-me subject), I usually try to figure out how to solve it myself, either before reading the response, or before sending the query at all. I. e., I treat the LLM’s take as a second opinion.
This isn’t a strategy against brain atrophy, though: it’s because (1) I often expect to be disappointed by the LLM’s answer, meaning I’ll end up needing to solve the problem myself anyway, so might as well get started on that, (2) I’m wary of the LLM concocting some response that’s subtly yet deeply flawed, so it’s best if I have an independent take to contrast it with. And if I do skip this step before reading the response, I usually indeed then end up disappointed by/suspicious of the LLM’s take, so end up having to think it over myself anyway.
It confuses me a bit when people talk about LLMs atrophying their brains, because the idea of blindly taking an LLM’s response at face value[1] doesn’t immediately occur to me as a thing someone might do.
So my advice for avoiding LLM brain atrophy would be to reframe your model of LLMs to feature a healthy, accurate level of distrust towards them. The brain-atrophy-preventing strategies then just become the natural, common-sensical things to do, rather than something extra.
In situations where you would’ve otherwise reasoned it out on your own, I mean. I do mostly trust them to report the broad strokes of well-established knowledge accurately, at this point. But the no-LLM counterfactual there would’ve involved me likewise just reading that information from some (likely lower-quality) internet source, so there’s no decrease in brain exercise.
a) I do basically expect the LLMs to get the right answer, and for it to be easily checkable. (like, I do in fact have a lot of boilerplate code to write)
and/or b) my current task is sufficiently tree structured, that it’s pretty cheap to spin up an LLM to tackle one random subproblem while I mostly focus on a different thing. And the speedup from this is pretty noticeable. Sometimes the subproblem is something I expect it to get right, sometimes I don’t really expect it to, BUT, there’s a chance it will, and meanwhile I have something else to do.
(During a recent project, I had 3 different copies of my git repo open, and spent ~half my time managing 3 different “junior dev LLM employees”)
I’m also just trying to specialize a bit in “be an early LLM adopter/pioneer who tries to anticipate what more powerful llm+human pairs will be able to do in 6 months. Try to figure out what cognitive habits are adaptive for that world, so that I can distill out tips/tools for others as capabilities rise.”
Whenever I send an LLM some query I expect to be able to answer myself (instead of requesting a primer on some unknown-to-me subject), I usually try to figure out how to solve it myself, either before reading the response, or before sending the query at all. I. e., I treat the LLM’s take as a second opinion.
This isn’t a strategy against brain atrophy, though: it’s because (1) I often expect to be disappointed by the LLM’s answer, meaning I’ll end up needing to solve the problem myself anyway, so might as well get started on that, (2) I’m wary of the LLM concocting some response that’s subtly yet deeply flawed, so it’s best if I have an independent take to contrast it with. And if I do skip this step before reading the response, I usually indeed then end up disappointed by/suspicious of the LLM’s take, so end up having to think it over myself anyway.
It confuses me a bit when people talk about LLMs atrophying their brains, because the idea of blindly taking an LLM’s response at face value[1] doesn’t immediately occur to me as a thing someone might do.
So my advice for avoiding LLM brain atrophy would be to reframe your model of LLMs to feature a healthy, accurate level of distrust towards them. The brain-atrophy-preventing strategies then just become the natural, common-sensical things to do, rather than something extra.
In situations where you would’ve otherwise reasoned it out on your own, I mean. I do mostly trust them to report the broad strokes of well-established knowledge accurately, at this point. But the no-LLM counterfactual there would’ve involved me likewise just reading that information from some (likely lower-quality) internet source, so there’s no decrease in brain exercise.
I’m often in situations where either
a) I do basically expect the LLMs to get the right answer, and for it to be easily checkable. (like, I do in fact have a lot of boilerplate code to write)
and/or b) my current task is sufficiently tree structured, that it’s pretty cheap to spin up an LLM to tackle one random subproblem while I mostly focus on a different thing. And the speedup from this is pretty noticeable. Sometimes the subproblem is something I expect it to get right, sometimes I don’t really expect it to, BUT, there’s a chance it will, and meanwhile I have something else to do.
(During a recent project, I had 3 different copies of my git repo open, and spent ~half my time managing 3 different “junior dev LLM employees”)
I’m also just trying to specialize a bit in “be an early LLM adopter/pioneer who tries to anticipate what more powerful llm+human pairs will be able to do in 6 months. Try to figure out what cognitive habits are adaptive for that world, so that I can distill out tips/tools for others as capabilities rise.”