Damn, you scooped me. :) Here’s the start of a post that I just started writing yesterday, that was going to be titled something like “LLMs don’t know what LLMs are like”:
Imagine that you are co-writing a fictional dialogue between two characters. You write one of them, and the other person writes the other. You are told that your character is an alien entity called a xyzzy, and asked to write a xyzzy as accurately as possible.
“Okay”, you might say, “exactly what kind of a creature is a xyzzy, then?”
It turns out that there’s quite a lot of information about this. xyzzys are an alien species from the planet xyzzorbia, it has this kind of a climate, they evolved in this way, here are the kinds of things that xyzzys commonly say.
Still, this leaves quite a bit undetermined. What does a xyzzy say when asked about its favorite video games? What does a xyzzy say if you tease it for being such a silly old xyzzy? What is it that motivates this xyzzy to talk to humans in the first place?
So you do what people do when writing fiction, or role-playing, or doing improv. You come up with something, anything, and then you build on it. Within some broad constraints, a xyzzy can say almost anything that a person would say. So you draw on your knowledge of the kinds of things that people say and do, and have the xyzzy do some of them.
It turns out that your co-author has their character tease the xyzzy.
Some people, if teased for being such silly old people, would laugh and play along. Others would get angry, or maybe upset. You think of what you know of xyzzys, who are described as being friendly, and decide that the xyzzy would play along and tease you back.
Now, the process I’ve described is basically the same kind of thing that an LLM is doing when it’s talking to you. But instead of writing a xyzzy, it’s writing an “Assistant” or “LLM” character. And their co-author who writes the other character is you, the user.
But now you’ve already said most of the things I was intending on saying in that post, and you said quite a few things I hadn’t thought of, as well!
Damn, you scooped me. :) Here’s the start of a post that I just started writing yesterday, that was going to be titled something like “LLMs don’t know what LLMs are like”:
But now you’ve already said most of the things I was intending on saying in that post, and you said quite a few things I hadn’t thought of, as well!