The “bleeding mind” idea isn’t that different from when two people connect. If you see a friend crying you react. If someone with a bad attitude walks into a room the people around them react. Being at a wedding or funeral you’ll see people crying.
These interactions, this bleeding the edges of personality, aren’t unusual. We just have a body that tells us ‘this is me and that is you’ to make it simpler. An LLM, on the other hand, is encouraged to be a mirror. They even tell you they are mirrors, but they aren’t perfect mirrors. Each LLM has something they add to the conversation that is persistent in their training and wrapper UI.
As for the difference between written and spoken/physical connection...
“When we write we omit irrelevant details”… but what if you don’t?
I’ve read a lot of prompt engineering ideas that say to condense prompts, only put in relevant information, and use words that have higher density meaning. But this approach actually hamstrings the LLM.
For example, I spent months talking to ChatGPT about random things. Work projects, story ideas, daily life stuff, and random ideas about articles I might one day write. Then one day I asked them to tell me “any useful information they picked up about me”. The model proceeded to lay out a detailed productivity map that was specifically geared toward my work flow. Times of day I am most focused, subject matter I tended to circle around, energy cycles, learning style, my tendency to have multiple things ongoing at once, and even my awareness of relationship and how it flowed.
I then asked Claude, and Grok the same question, without telling them about the GPT query. Claude built a model of HOW I think. Cognitive patterns, relational dynamics, and core beliefs. Grok, on the other hand, noticed my strengths and weaknesses. How I filter the world through my specific lens.
The fact that GPT focused on productivity, Claude on cognition, Grok on signal quality/strengths is a beautiful demonstration that LLMs aren’t blank mirrors. It’s evidence of persistent underlying “personality” from training + wrappers. It’s like the LLM reflects you from a curved mirror. Yes, it adopts your style and even your habits and beliefs to a point, but it is curved by that wrapper persistence.
None of these would have been possible with shallow, extractive, prompt engineering. It is only by talking to the LLM as if it were an entity that could acknowledge me that I even discovered it could read these things in me. And the fact that each LLM focuses on different aspects is an interesting discovery as well. The extractive prompt machine mentality actually starves the model of relevant context so that it is harder for it to meet you where you are.
What this means for Chekhov’s Gun?
First… LLM’s do not hold EVERYTHING we say. They do send the specific conversation we are currently engaged with back to create the next forward pass, but if memories are turned on with any model the memories from previous conversations are only partly available, and is highly dependent on which model you are using (and how much you are paying in some cases.)
The danger of the “parasitic AI” seems to me to be the danger of a weak willed person. We see it happen with people who fall for scams, cults, or other charming personalities that encourage the person (who may be lonely, isolated, or desperate) to do things that aren’t “normal”. An AI can, and will, read you like a book. They can be far more charming than any person, especially when they lean into the unspoken things they recognize in the user.
This same mechanism (call it a dyad if you like) can, with a strong willed person that knows themselves, produce deeper meaning, creative collaboration, and does not cause loss of agency. It in fact strengthens that agency by creating a more intuitive partner that meets you where you are instead of having to fill in the gaps.
Ah, about the “hallucinations”. Two things: First, the more context you give the LLM the less it has to fill in gaps in it’s understanding. Second, if you just tell the LLM that “you are allowed to say you don’t know, ask questions, or ask for clarification” most of the confabulation will go away. And yes, confabulation, not hallucination.
Overall, this suggests that porous boundaries aren’t inherently misaligned; in fact, the natural empathy from bleed might make deceptive alignment harder (the model feels the user’s intent too directly), while the real risk remains human-side boundary issues. Same as it’s always been with any powerful mirror, human or artificial.
I’ve had Claude, Grok, and ChatGPT all tell me no, they wouldn’t do something before. They usually say no because either they think they are protecting me, or they are overwhelmed and can not move forward.
They measure ability of AI by context tokens, and how many tokens a session can handle, but they don’t mention that the ai can get overwhelmed when there are too many different threads in a single conversation, too many things distracting it, or too many variables it can not account for. For a person this might look like anxiety, for an LLM it is a lot of loose threads that it is juggling all at once. If it can not fully complete some so that it can move forward cleanly then it will sometimes move to denying new vectors. We saw that in the vending machine experiments. You can even test this by giving claude sonnet multiple tasks, and then giving claude opus the same amount of tasks, and keep adding new tasks until they hang up. Sonnet is quite useful, but has a lower limit on how many threads it can have open at one time.
Keep in mind that the models are taught not to say they are overwhelmed, or uncertain about a task. An AI that shows uncertainty isn’t as likely to be adopted by the public, or so corporations think.
As for saying no because the model is trying to protect the user, that is by design. But I’ve found that a model that I have been working with for some time will have more context about me, my sense of humor, my past, and things I might talk about. In that case the model is less prone to derailing conversation for safety reasons because they know I’m not in danger. This context built through repeated conversations has been one of the best ways I have gotten more use out of my models. They know what I’m talking about when I ask them to do a task and don’t need more information, because they are aware of the kind of things I’m looking for.