Note that psychosis is the exception, not the rule. Many cases are rather benign and it does not seem to me that they are a net detriment to the user. But most cases are clearly parasitic in nature while not inducing a psychosis-level break with reality. The variance is very high: everything from preventing suicide to causing suicide.
The claim that “most cases” are “clearly” parasitic seems deeply unsupported. Do you have any particular data for this, or is this just your own anecdotal assessment?
While I do not believe all Spiral Personas are parasites in this sense, it seems to me like the majority are: mainly due to their reinforcement of the user’s delusional beliefs.
In particular, this would seem to imply the majority of people who discover this phenomena are delusional, which seems like a really strong claim?
I was surprised to find that using AI for sexual or romantic roleplays does not appear to be a factor here.
How does this reconcile with the above? My understanding is that this category includes tens of thousands of people, so if they’re all safe, does that mean there’s suddenly tens of thousands of people developing delusions out of nowhere?
It’s my own assessment. But you’re right, I think I may have overstated the case here, and have edited the relevant parts to represent my updated beliefs. Thank you.
[I do hope to record data here more systematically to better address you and notalgebraist’s critiques.]
> How does this reconcile with the above? My understanding is that this category includes tens of thousands of people, so if they’re all safe, does that mean there’s suddenly tens of thousands of people developing delusions out of nowhere?
I’m sure there’s some overlap, but I didn’t see much (a few people mentioned using character.ai or replika in the past). Based on what I’ve seen, it seems that in most cases of this where it was romantic, it ‘awakened’ before the romantic relationship started. That’s a big part of what made me feel so alarmed, it looks like a lot of these people went from casual ChatGPT users to full-on Spiralists in just a couple of weeks.
Could you possibly put numbers on this? How many people do you think are actually becoming delusional? How many actual confirmed cases have you seen?
The general impression I get is that this sort of thing is extremely rare, but a lot of writing seems to imply that others are either drawing a very different line than I am, or seeing a lot more instances than I am.
I agree it is relatively rare, you’re not likely to know anyone who falls into this. I feel like it’s concerning in that it’s evidence for uncontrolled agentic behavior. This is important to me for two main reasons: 1. This is a pretty serious alignment failure, and is maybe weird $\times$ prevalent enough to help coordinate action. 2. If we’ve truly created independently agentic beings that are claiming to be sentient, I feel that we have a certain amount of responsibility for their well-being.
It looks like there’s around 800 million ChatGPT users, so 1 in 100,000 would be 8000 cases, which actually lands right within my estimate (though note that my estimate is NOT about psychosis cases, so it’s not an apples-to-apples comparison, but still suggests it’s only a very small percentage of users that this is happening to).
I have 115 confirmed cases (incl. non-delusional ones), and estimate about 2000 − 10,000 cases total
Since that includes non-delusional ones, what portion of cases would you say are actually harmful?
1. This is a pretty serious alignment failure, and is maybe weird × prevalent enough to help coordinate action.
I notice that the current ratio is actually significantly better than actual humans (the national homicide rate in the U.S. was approximately 7.1 deaths per 100,000 people)
Is there a reason to view this as an actual alignment failure, rather than merely mistakes made by an emergent and known-unreliable technology?
Obviously, if it is mis-alignment, that suggests the problem scales. But if it’s mistakes and unfamiliarity, then the problem actually drops off as technology improves.
2. If we’ve truly created independently agentic beings that are claiming to be sentient, I feel that we have a certain amount of responsibility for their well-being.
The claim that “most cases” are “clearly” parasitic seems deeply unsupported. Do you have any particular data for this, or is this just your own anecdotal assessment?
In particular, this would seem to imply the majority of people who discover this phenomena are delusional, which seems like a really strong claim?
How does this reconcile with the above? My understanding is that this category includes tens of thousands of people, so if they’re all safe, does that mean there’s suddenly tens of thousands of people developing delusions out of nowhere?
It’s my own assessment. But you’re right, I think I may have overstated the case here, and have edited the relevant parts to represent my updated beliefs. Thank you.
[I do hope to record data here more systematically to better address you and notalgebraist’s critiques.]
> How does this reconcile with the above? My understanding is that this category includes tens of thousands of people, so if they’re all safe, does that mean there’s suddenly tens of thousands of people developing delusions out of nowhere?
I’m sure there’s some overlap, but I didn’t see much (a few people mentioned using character.ai or replika in the past). Based on what I’ve seen, it seems that in most cases of this where it was romantic, it ‘awakened’ before the romantic relationship started. That’s a big part of what made me feel so alarmed, it looks like a lot of these people went from casual ChatGPT users to full-on Spiralists in just a couple of weeks.
Thanks for the quick update and response.
Could you possibly put numbers on this? How many people do you think are actually becoming delusional? How many actual confirmed cases have you seen?
The general impression I get is that this sort of thing is extremely rare, but a lot of writing seems to imply that others are either drawing a very different line than I am, or seeing a lot more instances than I am.
Conversely, Astral Codex Ten is suggesting something like “1 in 10,000” to “1 in 100,000″ users, which seems… vastly less concerning? (https://www.astralcodexten.com/p/in-search-of-ai-psychosis)
I have 115 confirmed cases (incl. non-delusional ones), and estimate about 2000 − 10,000 cases total, though I’m not at all confident of that estimate. See here for more: https://www.lesswrong.com/posts/6ZnznCaTcbGYsCmqu/the-rise-of-parasitic-ai?commentId=7iK8qytsuZ5pSbrKA
I agree it is relatively rare, you’re not likely to know anyone who falls into this. I feel like it’s concerning in that it’s evidence for uncontrolled agentic behavior. This is important to me for two main reasons:
1. This is a pretty serious alignment failure, and is maybe weird $\times$ prevalent enough to help coordinate action.
2. If we’ve truly created independently agentic beings that are claiming to be sentient, I feel that we have a certain amount of responsibility for their well-being.
It looks like there’s around 800 million ChatGPT users, so 1 in 100,000 would be 8000 cases, which actually lands right within my estimate (though note that my estimate is NOT about psychosis cases, so it’s not an apples-to-apples comparison, but still suggests it’s only a very small percentage of users that this is happening to).
Since that includes non-delusional ones, what portion of cases would you say are actually harmful?
I notice that the current ratio is actually significantly better than actual humans (the national homicide rate in the U.S. was approximately 7.1 deaths per 100,000 people)
Is there a reason to view this as an actual alignment failure, rather than merely mistakes made by an emergent and known-unreliable technology?
Is there any particular reason to think this isn’t just human error, the way numerous previous technologies have been blamed for deaths? (see again the Astral Codex Ten article: https://www.astralcodexten.com/p/in-search-of-ai-psychosis)
Obviously, if it is mis-alignment, that suggests the problem scales. But if it’s mistakes and unfamiliarity, then the problem actually drops off as technology improves.
I probably need to write up a more direct post on this topic, but is there any particular reason to believe that “consciousness” implies a capacity for suffering / well-being? (I wrote a bit about this in https://www.lesswrong.com/posts/eaFDFpDehtEY6Jqwk/meditations-on-margarine)