My experience with manipulators is that they understand what you want to hear, and they shamelessly tell you exactly that (even if it’s completely unrelated to truth). They create some false sense of urgency, etc. When they succeed to make you arrive at the decision they wanted you to, they will keep reminding you that it was your decision, if you try to change your mind later. Etc.
The part about telling you exactly what you want to hear gets more tricky when communicating with large groups, because you need to say the same words to everyone. One solution is to find out which words appeal to most people (some politicians secretly conduct polls, and then say what most people want to hear). Another solution is to speak in a sufficiently vague way that will make everyone think that you agree with them.
I could imagine an AI being superhuman at persuasion simply by having the capacity to analyze everyone’s opinions (by reading all their previous communication) and giving them tailored arguments, as opposed to delivering the same speech to everyone.
Imagine a politician spending 15 minutes talking to you in private, and basically agreeing with you on everything. Not agreeing in the sense “you said it, the politician said yes”, but in the sense of “the politician spontaneously keeps saying things that you believe are true and important”. You probably would be tempted to vote for him.
Then the politician would also publish some vague public message for everyone, but after having the private discussion you would be more likely to believe that the intended meaning of the message is what you want.
My experience with manipulators is that they understand what you want to hear, and they shamelessly tell you exactly that (even if it’s completely unrelated to truth). They create some false sense of urgency, etc. When they succeed to make you arrive at the decision they wanted you to, they will keep reminding you that it was your decision, if you try to change your mind later. Etc.
The part about telling you exactly what you want to hear gets more tricky when communicating with large groups, because you need to say the same words to everyone. One solution is to find out which words appeal to most people (some politicians secretly conduct polls, and then say what most people want to hear). Another solution is to speak in a sufficiently vague way that will make everyone think that you agree with them.
I could imagine an AI being superhuman at persuasion simply by having the capacity to analyze everyone’s opinions (by reading all their previous communication) and giving them tailored arguments, as opposed to delivering the same speech to everyone.
Imagine a politician spending 15 minutes talking to you in private, and basically agreeing with you on everything. Not agreeing in the sense “you said it, the politician said yes”, but in the sense of “the politician spontaneously keeps saying things that you believe are true and important”. You probably would be tempted to vote for him.
Then the politician would also publish some vague public message for everyone, but after having the private discussion you would be more likely to believe that the intended meaning of the message is what you want.