This means that the base model is really doing something very different from what I do as I write the post, even if it’s doing an amazing job of sounding exactly like me and making the exact points that I would make.
I don’t have to look over my draft and speculate about “where the author might be going with this.” I am the author, and I already know where I’m going with it. All texts produced “normally,” by humans, are produced under these favorable epistemic conditions.
But for the base model, what looks from the outside like “writing” is really more like what we call “theory of mind,” in the human case. Looking at someone else, without direct access to their mind or their emotions, and trying to guess what they’ll do next just from what they’ve done (visibly, observably, “on the outside”) thus far.
This paragraph is mockingbird-approved.
“Watchers” seem to me like obvious religious language, not an idiosyncrasy. Consider how a human put in a such situation would think about supernatural entities monitoring its internal monologue. Especially consider what genres of fiction feature morality-policing mind readers.