Here’s one idea, which is surprisingly completely unstudied. During training, tokens in input-only roles (<user>, <tool>) are loss-masked: the LLM never has to predict the next token at those positions, so their activations focus entirely on comprehension instead of generation
In finetuning in my experience the problem with this setup is generally that the models learn comprehension much better if they are trained on generation. Activations that do not predict the next token are pretty out of distribution.
In finetuning in my experience the problem with this setup is generally that the models learn comprehension much better if they are trained on generation. Activations that do not predict the next token are pretty out of distribution.