As a slight tangent, I’m interested in your thoughts on the nature of the “conversation space” being traversed. You seemed to imply that conversations can have goals, i.e. destinations that participants in the conversation can try to steer it towards. For this, I’m imagining that the interlocutors have certain mental maps (e.g., a “who”, “what”, “when”, “where”, “why”, and “how”, along with their relations) that they either want filled in for themselves or that they’re trying to fill in for the other person.
The conversation would then involve looking for holes in each other’s mental maps (regions of high uncertainty) and cooperating to fill them in. Interruptions would then entail one participant trying to shift the other person’s attention to a (slightly?) different mental map that’s of greater interest to them (kind of like what I’m doing with this comment?).
As a further aside, I think this is one area where large language models like GPT-3 fall short of how humans actually use language. They can simulate conversations, but they can’t really participate in genuine conversation-space traversals in the sense of deliberately looking for gaps in understanding and for ways to fill those gaps.
By the way, how would your model handle other types of conversation that have purposes other than conveying or seeking information, such as witty banter, small talk, or giving/receiving orders? Would such conversations still involve traversals in the same space, or would it look qualitatively different? Would there still be goal states or just open-ended evolution?
The conversation would then involve looking for holes in each other’s mental maps (regions of high uncertainty) and cooperating to fill them in.
Absolutely! Although I’m not sure how well this particular path traversal analogy fits that idea. I like the one I used in Debugging the student more. I think the differences between the two are subtle but there.
You seemed to imply that conversations can have goals, i.e. destinations that participants in the conversation can try to steer it towards.
Yeah, I think so. An example that comes to my mind is that recently I was texting with a friend. We were talking about how there are so many covid cases in the NBA. I said how it feels weird to me given how disproportionate it is vs the general population. My friend said stuff about how athletes travel a lot and often do risky things like going out clubbing. I said that’s probably true but it doesn’t seem strong enough to explain it. Then he said how they are constantly getting covid tests. That lead to a lightbulb going off in my head. “Of course! That’s it!” We both felt happy with that as the explanation for the phenomenon, and using this analogy, we reached the destination.
But to address what you bring up later in your comment, I don’t think conversations always do or always should have this sort of singular destination as the goal. Witty banter is a good example of that. I think that like most things, it’s a spectrum. Sometimes there is a very clear and singular destination that everyone knows they are after, but that is an extreme. Other times the conversation participants know they are headed in some general direction, but aren’t sure exactly where the destination. Ie. there is an element of babbling.
They can simulate conversations, but they can’t really participate in genuine conversation-space traversals in the sense of deliberately looking for gaps in understanding and for ways to fill those gaps.
I actually know almost nothing about how language models like GPT-3 work, but it at least seems like it should be possible for them to do this, no?
By the way, how would your model handle other types of conversation that have purposes other than conveying or seeking information, such as witty banter, small talk, or giving/receiving orders? Would such conversations still involve traversals in the same space, or would it look qualitatively different? Would there still be goal states or just open-ended evolution?
To address this more explicitly, I think the model still fits.
In the witty banter case, the participants continue to take tangents, never pursuing one particular destination too hard. They do so because they enjoy the exploring and/or the novelty of going in new directions.
In the giving/receiving orders case, the authority figure has a lot of control over where the conversation goes. And they restrict the paths that the subordinate can take. Eg. by requiring a yes or no answer. Or often times only giving the subordinate one choice for an answer, eg. “Yes sir!”.
In small talk, it is taboo to go down certain paths. Eg. this clip makes fun of the fact that moving from small talk to “medium talk” is such a big taboo. In the context of small talk, the other paths to things like medium talk (“How’s your marriage?”) still exist, you can still go down them, it’s just that doing so is taboo.
As a slight tangent, I’m interested in your thoughts on the nature of the “conversation space” being traversed. You seemed to imply that conversations can have goals, i.e. destinations that participants in the conversation can try to steer it towards. For this, I’m imagining that the interlocutors have certain mental maps (e.g., a “who”, “what”, “when”, “where”, “why”, and “how”, along with their relations) that they either want filled in for themselves or that they’re trying to fill in for the other person.
The conversation would then involve looking for holes in each other’s mental maps (regions of high uncertainty) and cooperating to fill them in. Interruptions would then entail one participant trying to shift the other person’s attention to a (slightly?) different mental map that’s of greater interest to them (kind of like what I’m doing with this comment?).
As a further aside, I think this is one area where large language models like GPT-3 fall short of how humans actually use language. They can simulate conversations, but they can’t really participate in genuine conversation-space traversals in the sense of deliberately looking for gaps in understanding and for ways to fill those gaps.
By the way, how would your model handle other types of conversation that have purposes other than conveying or seeking information, such as witty banter, small talk, or giving/receiving orders? Would such conversations still involve traversals in the same space, or would it look qualitatively different? Would there still be goal states or just open-ended evolution?
Absolutely! Although I’m not sure how well this particular path traversal analogy fits that idea. I like the one I used in Debugging the student more. I think the differences between the two are subtle but there.
Yeah, I think so. An example that comes to my mind is that recently I was texting with a friend. We were talking about how there are so many covid cases in the NBA. I said how it feels weird to me given how disproportionate it is vs the general population. My friend said stuff about how athletes travel a lot and often do risky things like going out clubbing. I said that’s probably true but it doesn’t seem strong enough to explain it. Then he said how they are constantly getting covid tests. That lead to a lightbulb going off in my head. “Of course! That’s it!” We both felt happy with that as the explanation for the phenomenon, and using this analogy, we reached the destination.
But to address what you bring up later in your comment, I don’t think conversations always do or always should have this sort of singular destination as the goal. Witty banter is a good example of that. I think that like most things, it’s a spectrum. Sometimes there is a very clear and singular destination that everyone knows they are after, but that is an extreme. Other times the conversation participants know they are headed in some general direction, but aren’t sure exactly where the destination. Ie. there is an element of babbling.
I actually know almost nothing about how language models like GPT-3 work, but it at least seems like it should be possible for them to do this, no?
To address this more explicitly, I think the model still fits.
In the witty banter case, the participants continue to take tangents, never pursuing one particular destination too hard. They do so because they enjoy the exploring and/or the novelty of going in new directions.
In the giving/receiving orders case, the authority figure has a lot of control over where the conversation goes. And they restrict the paths that the subordinate can take. Eg. by requiring a yes or no answer. Or often times only giving the subordinate one choice for an answer, eg. “Yes sir!”.
In small talk, it is taboo to go down certain paths. Eg. this clip makes fun of the fact that moving from small talk to “medium talk” is such a big taboo. In the context of small talk, the other paths to things like medium talk (“How’s your marriage?”) still exist, you can still go down them, it’s just that doing so is taboo.