Along the line of other comments, we should: a) have the harness running the model impose correct tag parsing logic: open tags and close tags must be correctly paired in the correct order and cannot be nested. This requires imposing this on both text that gets inserted into the context from the system/user/tools, and also imposing it on token generation from the model: if a tag is open, the only tag token that can be generated is to close that open tag. and then open the valid next one. b) Now that the tags are definitely correct, have the harness parse them mechanically and directly inject a specific, fixed signal into the residual stream along with the tokens that correctly identifies the current role (via another mechanism comparable to but simpler than the ROPE signals). Thus the model doesn’t have to learn to recognize the correct role, because the signal was already provided for it. c) actively train the model NOT to react to stylistic cues that contradict of the role signals we’re providing, by occasionally providing training data with the text mixed up, and training the model to refuse if that happens
Along the line of other comments, we should:
a) have the harness running the model impose correct tag parsing logic: open tags and close tags must be correctly paired in the correct order and cannot be nested. This requires imposing this on both text that gets inserted into the context from the system/user/tools, and also imposing it on token generation from the model: if a tag is open, the only tag token that can be generated is to close that open tag. and then open the valid next one.
b) Now that the tags are definitely correct, have the harness parse them mechanically and directly inject a specific, fixed signal into the residual stream along with the tokens that correctly identifies the current role (via another mechanism comparable to but simpler than the ROPE signals). Thus the model doesn’t have to learn to recognize the correct role, because the signal was already provided for it.
c) actively train the model NOT to react to stylistic cues that contradict of the role signals we’re providing, by occasionally providing training data with the text mixed up, and training the model to refuse if that happens