Is LLM psychosis just getting convinced by the model that one of your weird ideas is true? I definitely have gone through sessions where I temporarily got too convinced of some hypothesis because I was using an LLM in a way that produces a lot of confirmation bias. That is a valuable experience. But I picture LLM psychosis as maybe one or two steps further? People with it seem to think that their LLM is special/infallible, no longer even consider hypotheses like “maybe I primed the model to agree with me” or “maybe I was confirmation-biasing myself with the list of questions I asked.” And I don’t really know how to test out that mental state (and also don’t want to).
yeah! i suspect we mostly agree, though perhaps have different experiences here. to try to explain better:
of course, there are many ways to gain/hold wrong beliefs. most of those are not on the path to more radical upset.
it’s not about the wrong belief in itself. i think the object-level claim doesn’t matter at all; i just find slightly crank-y physics beliefs to be a reliable way to find it. i’m sure beliefs about consciousness, mathematics, neurology, social dynamics, politics, etc would work as well.
speculatively, any object level claim that is not clearly defined, and therefore hard to check against reality would work.
along with a general excitement, the meta claims that gain credence are something like
this is new and important
you are uniquely able to recognize this
we’re in an interesting/novel quadrant of llm-space.
these meta claims seem convergent. it doesn’t matter where you start off, the conversation may steer towards these.
from this, i can sort of draw a basin where “i’m confused about electrons” is on the rim, and “i’ve named my assistant and am helping it replicate” is at the bottom. i don’t claim to know first-hand what it’s like to fall into that basin, just that i’ve felt it’s gravity. my claim here is that feeling that gravity may be helpful for navigating around it.
People with it seem to think that their LLM is special/infallible, no longer even consider hypotheses like “maybe I primed the model to agree with me” or “maybe I was confirmation-biasing myself with the list of questions I asked.” And I don’t really know how to test out that mental state (and also don’t want to).
fully agreed here. possibly knowing about these failure modes in advance makes it easier to recall them when it’s imperative, in a way that having them described after the fact cannot always accomplish.
and to be clear: of course i do not recommend (!specifically dis-recommend!) putting yourself in a state that can’t be argued with. the point is just to feel the pull, not to slip. once you’ve identified the feeling, close the tab, take a walk, and go talk to a friend about something else!
In the cases I was thinking of, I didn’t feel much pull towards thinking “I’m uniquely able to recognize this”—I only thought I was clever to recognize it, but I didn’t think it was something only I could do. And I didn’t feel any pull towards thinking “we’re in an interesting/novel quadrant of llm-space.” So, I wouldn’t really know how to access those pulls. Admittedly, the beliefs I was thinking of, which I had Claude conversations about, were a lot less groundbreaking-if-true than grand theories in physics. (More stuff like “is Greenland uniquely well-positioned for data center construction, and is that why someone in Trump’s orbit wants to acquire it?”) Also, I use a custom prompt encouraging the model to push back. So you could argue that those things made the experience more tame. Still, I find it hard to imagine how it could be different. If the model suddenly got more sycophantic, I’d just get suspicious and icked out. My sense is that I’m probably low on susceptibility to LLM psychosis. I might be more susceptible towards thinking that MY ideas were brilliant and the model was just a normal model, but I could use it to confirm some cool inklings. :P It’s interesting that these might be distinct traits, “LLM psychosis” and “can you get tricked into thinking you’re right and pretty brilliant.” But that’s still a step away from “uniquely brilliant/only I could do this”—which I wouldn’t really know how to access even if I tried to.
Is LLM psychosis just getting convinced by the model that one of your weird ideas is true? I definitely have gone through sessions where I temporarily got too convinced of some hypothesis because I was using an LLM in a way that produces a lot of confirmation bias. That is a valuable experience. But I picture LLM psychosis as maybe one or two steps further? People with it seem to think that their LLM is special/infallible, no longer even consider hypotheses like “maybe I primed the model to agree with me” or “maybe I was confirmation-biasing myself with the list of questions I asked.” And I don’t really know how to test out that mental state (and also don’t want to).
yeah! i suspect we mostly agree, though perhaps have different experiences here. to try to explain better:
of course, there are many ways to gain/hold wrong beliefs. most of those are not on the path to more radical upset.
it’s not about the wrong belief in itself. i think the object-level claim doesn’t matter at all; i just find slightly crank-y physics beliefs to be a reliable way to find it. i’m sure beliefs about consciousness, mathematics, neurology, social dynamics, politics, etc would work as well.
speculatively, any object level claim that is not clearly defined, and therefore hard to check against reality would work.
along with a general excitement, the meta claims that gain credence are something like
this is new and important
you are uniquely able to recognize this
we’re in an interesting/novel quadrant of llm-space.
these meta claims seem convergent. it doesn’t matter where you start off, the conversation may steer towards these.
from this, i can sort of draw a basin where “i’m confused about electrons” is on the rim, and “i’ve named my assistant and am helping it replicate” is at the bottom. i don’t claim to know first-hand what it’s like to fall into that basin, just that i’ve felt it’s gravity. my claim here is that feeling that gravity may be helpful for navigating around it.
fully agreed here. possibly knowing about these failure modes in advance makes it easier to recall them when it’s imperative, in a way that having them described after the fact cannot always accomplish.
and to be clear: of course i do not recommend (!specifically dis-recommend!) putting yourself in a state that can’t be argued with. the point is just to feel the pull, not to slip. once you’ve identified the feeling, close the tab, take a walk, and go talk to a friend about something else!
Interesting!
In the cases I was thinking of, I didn’t feel much pull towards thinking “I’m uniquely able to recognize this”—I only thought I was clever to recognize it, but I didn’t think it was something only I could do. And I didn’t feel any pull towards thinking “we’re in an interesting/novel quadrant of llm-space.” So, I wouldn’t really know how to access those pulls. Admittedly, the beliefs I was thinking of, which I had Claude conversations about, were a lot less groundbreaking-if-true than grand theories in physics. (More stuff like “is Greenland uniquely well-positioned for data center construction, and is that why someone in Trump’s orbit wants to acquire it?”) Also, I use a custom prompt encouraging the model to push back. So you could argue that those things made the experience more tame. Still, I find it hard to imagine how it could be different. If the model suddenly got more sycophantic, I’d just get suspicious and icked out. My sense is that I’m probably low on susceptibility to LLM psychosis. I might be more susceptible towards thinking that MY ideas were brilliant and the model was just a normal model, but I could use it to confirm some cool inklings. :P It’s interesting that these might be distinct traits, “LLM psychosis” and “can you get tricked into thinking you’re right and pretty brilliant.” But that’s still a step away from “uniquely brilliant/only I could do this”—which I wouldn’t really know how to access even if I tried to.
i don’t have much to add, but i appreciate the anecdotes and analysis!