The predictor has to model the boss and the assistant somehow. The model of the boss learns something about the boss’ intent from the prompt. The model of the assistant may find this a piece of useful processing to have and so shares the same submodel containing the boss’ intent.
Now when the boss becomes a real user, the predictor does the same thing with the user. So it has a model of the user with their intent, and this model of the user’s intent is also used directly by the assistant. The correct thing would have been to maintain the user model’s model of the user’s intent, and the assistant’s model of the user and their intent, as separate entities. This would allow for the assistant to explicitly model the possibility that they are mistaken about the user’s intent.
In anthropomorphized terms: it feels like it can directly feel the user’s own intent. Hopefully that makes things more clear?
Thanks, I’m very glad to get some feedback.
The predictor has to model the boss and the assistant somehow. The model of the boss learns something about the boss’ intent from the prompt. The model of the assistant may find this a piece of useful processing to have and so shares the same submodel containing the boss’ intent.
Now when the boss becomes a real user, the predictor does the same thing with the user. So it has a model of the user with their intent, and this model of the user’s intent is also used directly by the assistant. The correct thing would have been to maintain the user model’s model of the user’s intent, and the assistant’s model of the user and their intent, as separate entities. This would allow for the assistant to explicitly model the possibility that they are mistaken about the user’s intent.
In anthropomorphized terms: it feels like it can directly feel the user’s own intent. Hopefully that makes things more clear?