Clément Dumas comments on What We Learned Trying to Diff Base and Chat Models (And Why It Matters)

[Clément Dumas]

Yeah we’ve thought about it but didn’t run any experiment yet. An easy trick would be to add a $L_{diff}$ to the crosscoder reconstruction loss:

$$L_{diff}=\text{MSE}\left(\right(\text{chat}-\text{base})-(\text{chat\_recon}-\text{base\_recon}\left)\right)=\text{MSE}\left(e_{\text{chat}}\right)+\text{MSE}\left(e_{\text{base}}\right)-2e_{\text{chat}}\cdot e_{\text{base}}$$

with

$$\begin{array}{cc}{e}_{\text{chat}}& =\text{chat}-\text{chat\_recon}\\ e_{\text{base}}& =\text{base}-\text{base\_recon}\end{array}$$

So basically a generalization is to change the crosscoder loss to:

$$L=\text{MSE}\left(e_{\text{chat}}\right)+\text{MSE}\left(e_{\text{base}}\right)+\lambda \cdot (2e_{\text{chat}}\cdot e_{\text{base}}),\lambda \in [-1,0]$$

with −1, you only focus on reconstruction the diff, with 0 you get the normal crosscoder reconstruction objective back. −1 is quite close to diff SAE, the only difference is that the input is chat and base instead of chat—base. Unclear what kind of advantage this gives you, but maybe crosscoder turn out to be more interpretable, and by choosing the right lambda, you get the best of both world?

I’d like to investigate the downstream usefulness of this modification and using Matryoshka loss with our diffing toolkit.