This would have the added advantage that it would be possible to programmatically determine at all positions whether the model is generating tokens, and also at every position whether the model thinks it’s generating tokens (almost certainly just a linear feature), and where this’d diverge be more vigilant. Similar for system instructions, text from user, etc.
This would have the added advantage that it would be possible to programmatically determine at all positions whether the model is generating tokens, and also at every position whether the model thinks it’s generating tokens (almost certainly just a linear feature), and where this’d diverge be more vigilant. Similar for system instructions, text from user, etc.