I don’t have a real explanation, but I’ve been interested in this, since it feels like the LLM is doing something like the opposite of what writers intend to do (at least in the effect). As if there’s some portion of language space that invites engagement, or trips an alarm in the reader that says ‘there’s something in this!’ Human writers swim toward that portion of the space; LLMs swim away from it.
[I would be unsurprised to find I have not expressed this well.]
I mean the base models are outputting the most likely next token (modulo a temperature parameter). “Whatever is most likely to come next, based on what we’ve seen so far” is, in some sense, the opposite of interesting writing, which is interesting precisely because it has something novel or unusual or surprising to say.
I don’t have a real explanation, but I’ve been interested in this, since it feels like the LLM is doing something like the opposite of what writers intend to do (at least in the effect). As if there’s some portion of language space that invites engagement, or trips an alarm in the reader that says ‘there’s something in this!’ Human writers swim toward that portion of the space; LLMs swim away from it.
[I would be unsurprised to find I have not expressed this well.]
I mean the base models are outputting the most likely next token (modulo a temperature parameter). “Whatever is most likely to come next, based on what we’ve seen so far” is, in some sense, the opposite of interesting writing, which is interesting precisely because it has something novel or unusual or surprising to say.