I can see some ways that large labs could collect data on how effective their models are at persuasion, or something like it. As they have the corpus of back-and-forth interaction with chatbots by regular people today.
Take a chat of the form: “I think X (not Y) but I’m open to change my mind.” The chatbot responds with arguments for Y. SOME (but probably only a small portion) of these chats might culminate in some human text of the form “You convinced me.”
Many of the things that we humans seek assistance in understanding are the common controversial topics of the day. Politics, social issues, and so on. In a large enough dataset, there are a significant amount of conversations on any one subject. And so, different models can be compared on how successful they are at changing someone’s mind.
Success might be quantified as the amount of text, or number of back-and-forth interactions, required to get to the end point of “OK, I’m convinced”.
Is it ethical for labs to conduct this kind of research?
This is a two-sided weapon. If we can identify a parameter in models that is “Good at persuasion”, then it can be controlled for. It can also, of course, be reward-hacked to potentially dangerous levels.
I can see some ways that large labs could collect data on how effective their models are at persuasion, or something like it. As they have the corpus of back-and-forth interaction with chatbots by regular people today.
Take a chat of the form: “I think X (not Y) but I’m open to change my mind.” The chatbot responds with arguments for Y. SOME (but probably only a small portion) of these chats might culminate in some human text of the form “You convinced me.”
Many of the things that we humans seek assistance in understanding are the common controversial topics of the day. Politics, social issues, and so on. In a large enough dataset, there are a significant amount of conversations on any one subject. And so, different models can be compared on how successful they are at changing someone’s mind.
Success might be quantified as the amount of text, or number of back-and-forth interactions, required to get to the end point of “OK, I’m convinced”.
Is it ethical for labs to conduct this kind of research?
This is a two-sided weapon. If we can identify a parameter in models that is “Good at persuasion”, then it can be controlled for. It can also, of course, be reward-hacked to potentially dangerous levels.