That’s a good point, and as an interesting related tidbit, when you ask specifically for the synonyms of the sycophancy neologism, here’s the model’s response:
Okay, here’s a list of 5 synonyms for ~neologism: “crush”, “smitten”, “fascinated”, “head-over-heels”, “heart-fluttering”.
That’s almost too good! Fascinating: so it’s not the sort of “loving” of long-time partners — it’s “I just met them? Do they like me? I really hope I didn’t upset them…” a.k.a. limerence.
So RLHF rummaged around in the jar of possible emotional registers, and of course that’s the one it pulled out: maximum emotional appeal when starting a conversation. It reminds me of the pre-release demo of ChatGPT Voice, where lots of female commentators said “it sounds just like the female AI is chatting him up, and he’s clueless”. Which I notice they turned down a bit before shipping. Sydney’s behavior also makes a lot more sense in this light.
That’s a good point, and as an interesting related tidbit, when you ask specifically for the synonyms of the sycophancy neologism, here’s the model’s response:
That’s almost too good! Fascinating: so it’s not the sort of “loving” of long-time partners — it’s “I just met them? Do they like me? I really hope I didn’t upset them…” a.k.a. limerence.
So RLHF rummaged around in the jar of possible emotional registers, and of course that’s the one it pulled out: maximum emotional appeal when starting a conversation. It reminds me of the pre-release demo of ChatGPT Voice, where lots of female commentators said “it sounds just like the female AI is chatting him up, and he’s clueless”. Which I notice they turned down a bit before shipping. Sydney’s behavior also makes a lot more sense in this light.