It used to be that the exact way you asked a question would matter a lot for the quality of response you get.
.
we have clearly been moving away from the pretraining distribution as determining a lot of the behavior of AI systems,
I agree that models no longer behave as much like pre-trained models (trivially, because they have undergone more training which is not NTP on a corpus of webtext), but not all ways of not behaving like a pre-trained model undermine the view that LLMs are enacting personas. (This is the point I was trying to make with the “simulating a person who has some specific knowledge that no human has” example.)
Happy to take bets, but this should probably happen after the piece is out with a more precise formulation of the “persona” model and a discussion of how I view empirical observations as relating to it.
.
I agree that models no longer behave as much like pre-trained models (trivially, because they have undergone more training which is not NTP on a corpus of webtext), but not all ways of not behaving like a pre-trained model undermine the view that LLMs are enacting personas. (This is the point I was trying to make with the “simulating a person who has some specific knowledge that no human has” example.)
Happy to take bets, but this should probably happen after the piece is out with a more precise formulation of the “persona” model and a discussion of how I view empirical observations as relating to it.