Interesting. I did something not dissimilar recently (https://medium.com/@thorn-thee-valiant/can-i-teach-an-llm-to-do-psychotherapy-with-a-json-and-a-txt-file-db443fa08e47) but from a more clinical perspective. Thinking about it I think there are probably two different risks here—the “AI psychosis” concept I’m still a bit sceptical of, that is the idea that LLM interactions cause or are exacerbating psychosis and this is where the concerns about collusion seem to fit in—but there’s also the simple risk question, does the LLM adopt the user’s frame to encourage or not challenge really dangerous stuff. Obviously there are lots of caveats about using AI generated patients and AI generated scorers in this kind of work. I was amused that you also found GPT-OSS:20B pushed back hard against delusions—as I also found this—but the AI patients didn’t respond well to it (which is why clinicians don’t usually go all out to directly challenge as it threatens the therapeutic frame—it’s slightly different when diagnosing psychosis as you have to test the fixety of the delusions so challenge is needed).
Interesting. I did something not dissimilar recently (https://medium.com/@thorn-thee-valiant/can-i-teach-an-llm-to-do-psychotherapy-with-a-json-and-a-txt-file-db443fa08e47) but from a more clinical perspective. Thinking about it I think there are probably two different risks here—the “AI psychosis” concept I’m still a bit sceptical of, that is the idea that LLM interactions cause or are exacerbating psychosis and this is where the concerns about collusion seem to fit in—but there’s also the simple risk question, does the LLM adopt the user’s frame to encourage or not challenge really dangerous stuff. Obviously there are lots of caveats about using AI generated patients and AI generated scorers in this kind of work. I was amused that you also found GPT-OSS:20B pushed back hard against delusions—as I also found this—but the AI patients didn’t respond well to it (which is why clinicians don’t usually go all out to directly challenge as it threatens the therapeutic frame—it’s slightly different when diagnosing psychosis as you have to test the fixety of the delusions so challenge is needed).