Very nice! What does color represent in the heatmaps? Cycle loss/accuracy?
Each is scored by how well the encoder reconstructs the original activation (cosine similarity).
If the base model uses non-normalized activations, shouldn’t the dot product be your training signal? Otherwise information in the magnitude of the activations would be ignored. I wonder if that might account for some of the inaccuracy (0.8).
The heatmaps represent two separate things: on the original model’s tokens, it represents cosine similarity of the reconstruction. On the decoder’s output tokens, the heatmap shows the weight the encoder’s pooling head puts on that token’s activation.
Yeah cosine sim does throw away magnitude information. Dot product wouldn’t work because you can maximise it by predicting a very large vector, but MSE would be a reasonable choice.
Very nice! What does color represent in the heatmaps? Cycle loss/accuracy?
If the base model uses non-normalized activations, shouldn’t the dot product be your training signal? Otherwise information in the magnitude of the activations would be ignored. I wonder if that might account for some of the inaccuracy (0.8).
The heatmaps represent two separate things: on the original model’s tokens, it represents cosine similarity of the reconstruction. On the decoder’s output tokens, the heatmap shows the weight the encoder’s pooling head puts on that token’s activation.
Yeah cosine sim does throw away magnitude information. Dot product wouldn’t work because you can maximise it by predicting a very large vector, but MSE would be a reasonable choice.