That’s a good point, but what that makes me wonder in turn is whether the AI having a speed advantage in writing might be displacing that for this particular type of interaction. Basically: “I would’ve needed to write a long manifesto to truly demonstrate the depth and brilliance of my ideas, and now I can get a really classy-looking one with equations and everything for 10% as much effort!”
(I’m tempted to go look up transcripts and check, but my cognitohazard shielding is on the low side for that right now. I wouldn’t mind if someone else did it.)
Basically: “I would’ve needed to write a long manifesto to truly demonstrate the depth and brilliance of my ideas, and now I can get a really classy-looking one with equations and everything for 10% as much effort!”
This does in fact largely appear to be the case in the cases I have studied. I also recall seeing (seems I forgot to save this transcript) a crackpot posting their long manifesto in the first message or so, and then iterating on getting validation for it with much shorter messages.
More common, at least in the cases I’ve come across, is for the user to have vague ideas or beliefs they don’t really take all that seriously. They’ll take their idea to the chatbot, just for a bit of curious exploration. But then! It “turns out” that this random half-baked idea is “actually” THE KEY to understanding quantum consciousness or whatever… and then this gets written up into a long manifesto and put on a personal website or github repo.
Also I’m a bit surprised about how much pushback my heuristic is getting? These are fundamentally conversations, which have a natural cadence to them that doesn’t allow for enough time for a human to write a longish new message each turn (remember that most people do not type nearly as fast as the average you here). People don’t stand around giving uninterrupted mini-lectures to each other, back and forth, on EACH turn—not even at rationalist parties! Maybe many rationalists interact with chatbots more as a letter correspondent, but if so this is highly unusual (and not true for me).
Maybe many rationalists interact with chatbots more as a letter correspondent, but if so this is highly unusual (and not true for me).
I do! Not full ‘letters’, but definitely a paragraph at a time is normal for me. (I also naturally do it in other conversational media, sometimes unfortunately; this is a habit I’ve tried to break among genpop, with partial success. In another life I have sometimes been known for accidentally smothering people’s Twitch chat…) I would guess that my entire communicative and thinking style was heavily influenced by written culture first and oral culture a distant second, so I talk like a book unless I’m trying not to.
Spot check on a recent conversation I had with ChatGPT-5 in which I was trying to solidify my understanding of a few points in mathematics: my message lengths from that conversation, rounded to multiples of 10 and sorted, were (70 310 340 370 400 480 770 820). The lowest one corresponds to my initial question, and you can see the others all fall into a sort of paragraph-y range, with no really short ones at all.
I can easily believe that this would be unusual overall, though I don’t know if it would have occurred to me to think of that if you hadn’t pointed it out. I don’t know how unusual it is among people selected for being active LW users.
I should have mentioned (it’s one of those things where I have to think longer about it to make it more legible to myself) that my heuristic is also looking at the variance in message length, which is much higher in real chats I’ve seen than in these ones.
If that makes a user unrealistic, then I’m unrealistic!
More seriously: you might be unrealistic as an example of the type of user who can readily be pulled into a delusional spiral?
Maybe? My mental model of crackpots involves them writing very long manifestos.
That’s a good point, but what that makes me wonder in turn is whether the AI having a speed advantage in writing might be displacing that for this particular type of interaction. Basically: “I would’ve needed to write a long manifesto to truly demonstrate the depth and brilliance of my ideas, and now I can get a really classy-looking one with equations and everything for 10% as much effort!”
(I’m tempted to go look up transcripts and check, but my cognitohazard shielding is on the low side for that right now. I wouldn’t mind if someone else did it.)
This does in fact largely appear to be the case in the cases I have studied. I also recall seeing (seems I forgot to save this transcript) a crackpot posting their long manifesto in the first message or so, and then iterating on getting validation for it with much shorter messages.
More common, at least in the cases I’ve come across, is for the user to have vague ideas or beliefs they don’t really take all that seriously. They’ll take their idea to the chatbot, just for a bit of curious exploration. But then! It “turns out” that this random half-baked idea is “actually” THE KEY to understanding quantum consciousness or whatever… and then this gets written up into a long manifesto and put on a personal website or github repo.
Also I’m a bit surprised about how much pushback my heuristic is getting? These are fundamentally conversations, which have a natural cadence to them that doesn’t allow for enough time for a human to write a longish new message each turn (remember that most people do not type nearly as fast as the average you here). People don’t stand around giving uninterrupted mini-lectures to each other, back and forth, on EACH turn—not even at rationalist parties! Maybe many rationalists interact with chatbots more as a letter correspondent, but if so this is highly unusual (and not true for me).
I do! Not full ‘letters’, but definitely a paragraph at a time is normal for me. (I also naturally do it in other conversational media, sometimes unfortunately; this is a habit I’ve tried to break among genpop, with partial success. In another life I have sometimes been known for accidentally smothering people’s Twitch chat…) I would guess that my entire communicative and thinking style was heavily influenced by written culture first and oral culture a distant second, so I talk like a book unless I’m trying not to.
Spot check on a recent conversation I had with ChatGPT-5 in which I was trying to solidify my understanding of a few points in mathematics: my message lengths from that conversation, rounded to multiples of 10 and sorted, were (70 310 340 370 400 480 770 820). The lowest one corresponds to my initial question, and you can see the others all fall into a sort of paragraph-y range, with no really short ones at all.
I can easily believe that this would be unusual overall, though I don’t know if it would have occurred to me to think of that if you hadn’t pointed it out. I don’t know how unusual it is among people selected for being active LW users.
With perfect spelling and grammar? In a chat?
I should have mentioned (it’s one of those things where I have to think longer about it to make it more legible to myself) that my heuristic is also looking at the variance in message length, which is much higher in real chats I’ve seen than in these ones.