One reason I’ve since become a bit more skeptical about AI achieving superhuman-level persuasive ability soon* is that it feels like persuading other humans has been tethered to reproductive fitness for some time. It feels like persuasive ability is something that the human mind was heavily optimized for, especially compared to something like mathematics. If I had to guess, I’d guess that superhuman ability in mathematics (and programming, and other verifiable domains) will arrive long* before superhuman persuasive ability. That said, it seems like LLMs are already somewhat persuasive, showing the ability to write good oral arguments before the US Supreme Court or writing comments on r/changemyview. This makes me suspect that I’m not correct. And it does feel like persuasion is something that can be improved through RL, though it feels like the feedback loop is pretty long.
*without an intelligence explosion underway. But I imagine that the result of an intelligence explosion will have many other ways to pwn us.
r/changemyview may select for posters who have certain expectations of how a view changes: reasoned debate, well-researched arguments, careful analogies. these are orthogonal skills to raw persuasion.
Yeah, agreed. Oral arguments are also pretty structured, and probably also benefit greatly from the enormous amount of legal knowledge that LLMs enjoy. Their current persuasive abilities of LLMs feel pretty qualitatively different from whatever skills Charles Manson used to build up a cult.
It seems like being persuasive is mainly working out what the entity being persuaded wants to get and wants to avoid. Once you can work that out (basically—something like cognitive empathy, including how psychopaths work), you then just have to select arguments that suggest that they are more likely to get what they want (and less likely to get what they don’t want) if they agree with you.
If you can simulate another entity with high fidelity, then you can just run a bunch of different arguments by them and see which ones tend to work better. This transforms it into an optimisation problem. Or you can even do it the other way round and map which arguments work on which bins of people.
My guess is that the human mind is sufficiently messy that working out what the entity wants to get and avoid, then selectively working out arguments is a reasonably small component of persuasion. When I think of people who were extraordinarily persuasive, for example Charles Manson or Daryl Davis, what seemed to be going on was much more nuanced than just that. Manson seemed to hijack the minds of the people in his ‘family’ through drugs and power games in a way that seems irreducible to arguments about carrots and sticks. Same goes for a lot of cult leaders, who seemingly attempt to modify their members values themselves rather than just offering a way for their members to achieve their values, Likewise, Davis got people to leave the KKK not purely through offering rational arguments for why they’re more likely to get the things they want (though this may(?) have been a component), but by inducing something like cognitive dissonance.
I guess the main point I want to raise is that persuasion is often more than just words. Words that work to persuade when said by a big, scary-looking guy are likely different from words that work when said by someone who looks like a small child, which are likely different from words that work when said by what people perceive to be an AI. Not to mention things like hypnotic suggestibility, using social pressure, or otherwise going around the part of the human mind that considers arguments at all.
That said, I don’t really have an argument against just simulating an entity and trying out different scenarios, then doing more of what works and less of what doesn’t. That feels like it should work.
Why assume this is only words? AI systems can already generate images, video and sound, which gives you a lot more subconscious bandwidth (scents and tactile feedback are admittedly harder). There’s also the shortcut of finding a charismatic/convincing person and convincing them to help, which may or may not make things a lot easier.
One reason I’ve since become a bit more skeptical about AI achieving superhuman-level persuasive ability soon* is that it feels like persuading other humans has been tethered to reproductive fitness for some time. It feels like persuasive ability is something that the human mind was heavily optimized for, especially compared to something like mathematics. If I had to guess, I’d guess that superhuman ability in mathematics (and programming, and other verifiable domains) will arrive long* before superhuman persuasive ability. That said, it seems like LLMs are already somewhat persuasive, showing the ability to write good oral arguments before the US Supreme Court or writing comments on r/changemyview. This makes me suspect that I’m not correct. And it does feel like persuasion is something that can be improved through RL, though it feels like the feedback loop is pretty long.
*without an intelligence explosion underway. But I imagine that the result of an intelligence explosion will have many other ways to pwn us.
r/changemyview may select for posters who have certain expectations of how a view changes: reasoned debate, well-researched arguments, careful analogies. these are orthogonal skills to raw persuasion.
Yeah, agreed. Oral arguments are also pretty structured, and probably also benefit greatly from the enormous amount of legal knowledge that LLMs enjoy. Their current persuasive abilities of LLMs feel pretty qualitatively different from whatever skills Charles Manson used to build up a cult.
It seems like being persuasive is mainly working out what the entity being persuaded wants to get and wants to avoid. Once you can work that out (basically—something like cognitive empathy, including how psychopaths work), you then just have to select arguments that suggest that they are more likely to get what they want (and less likely to get what they don’t want) if they agree with you.
If you can simulate another entity with high fidelity, then you can just run a bunch of different arguments by them and see which ones tend to work better. This transforms it into an optimisation problem. Or you can even do it the other way round and map which arguments work on which bins of people.
My guess is that the human mind is sufficiently messy that working out what the entity wants to get and avoid, then selectively working out arguments is a reasonably small component of persuasion. When I think of people who were extraordinarily persuasive, for example Charles Manson or Daryl Davis, what seemed to be going on was much more nuanced than just that. Manson seemed to hijack the minds of the people in his ‘family’ through drugs and power games in a way that seems irreducible to arguments about carrots and sticks. Same goes for a lot of cult leaders, who seemingly attempt to modify their members values themselves rather than just offering a way for their members to achieve their values, Likewise, Davis got people to leave the KKK not purely through offering rational arguments for why they’re more likely to get the things they want (though this may(?) have been a component), but by inducing something like cognitive dissonance.
I guess the main point I want to raise is that persuasion is often more than just words. Words that work to persuade when said by a big, scary-looking guy are likely different from words that work when said by someone who looks like a small child, which are likely different from words that work when said by what people perceive to be an AI. Not to mention things like hypnotic suggestibility, using social pressure, or otherwise going around the part of the human mind that considers arguments at all.
That said, I don’t really have an argument against just simulating an entity and trying out different scenarios, then doing more of what works and less of what doesn’t. That feels like it should work.
Why assume this is only words? AI systems can already generate images, video and sound, which gives you a lot more subconscious bandwidth (scents and tactile feedback are admittedly harder). There’s also the shortcut of finding a charismatic/convincing person and convincing them to help, which may or may not make things a lot easier.