I frequently rubber-duck at them when thinking through some novel research threads. I find them useful for (a) providing “pseudosocial” intellectual stimulation, (b) bringing up relevant concepts known at large but not to me, and sometimes for (c) sanity-checking me.
The bulk of the value in (c) is in the act of my preparing my argument to be presented to something-that-feels-like-an-intelligent-entity, however. This, by itself, activates different instincts/heuristics in me, making me spot holes in the argument or clarify it in ways I wouldn’t have otherwise. The LLM’s actual response afterwards is mostly irrelevant; I often only skim it or even dismiss the supposed holes it identifies as red herrings.
What I don’t find them useful for is (d) developing the new research thread in new directions. Intuitively, their default behavior there is to look for safety: “pulling back” from speculative directions, “rounding the idea off” to something known, introducing a bunch of caveats that muddle things instead of committing to a strong simplifying assumption, etc. You can of course get away from this default behavior by being sufficiently confident and using big words arranged in clever-seeming arguments… but I’m pretty sure I’d be able to talk them into anything this way, it’s the “AI psychosis” basin.
I admittedly haven’t tried to see if they could do novel conceptual research fully autonomously, not since a while ago. Trying this again is a low-priority task on my to-do list, but I don’t expect much.
I frequently rubber-duck at them when thinking through some novel research threads. I find them useful for (a) providing “pseudosocial” intellectual stimulation, (b) bringing up relevant concepts known at large but not to me, and sometimes for (c) sanity-checking me.
The bulk of the value in (c) is in the act of my preparing my argument to be presented to something-that-feels-like-an-intelligent-entity, however. This, by itself, activates different instincts/heuristics in me, making me spot holes in the argument or clarify it in ways I wouldn’t have otherwise. The LLM’s actual response afterwards is mostly irrelevant; I often only skim it or even dismiss the supposed holes it identifies as red herrings.
What I don’t find them useful for is (d) developing the new research thread in new directions. Intuitively, their default behavior there is to look for safety: “pulling back” from speculative directions, “rounding the idea off” to something known, introducing a bunch of caveats that muddle things instead of committing to a strong simplifying assumption, etc. You can of course get away from this default behavior by being sufficiently confident and using big words arranged in clever-seeming arguments… but I’m pretty sure I’d be able to talk them into anything this way, it’s the “AI psychosis” basin.
I admittedly haven’t tried to see if they could do novel conceptual research fully autonomously, not since a while ago. Trying this again is a low-priority task on my to-do list, but I don’t expect much.