So, what insights did you glean from this? Attention values are pretty intuitive, but did you notice anything surprising in what the various probes produced in a given situation?
Plenty of insights. Rather than probes, I played with the app’s logit lens and was inspired to quantitatively look into compositionality. One insight is that it appears certain tactics like forks are compositional features rather than a new feature in and of itself. You can check the logits of the fork for a piece move and compare it to the sums of the logits for the two individual threats the move creates, and the values are roughly equal.
Fix the board state but remove one of the forked pieces and look at the value of the logit lens for the same fork move without that piece through time.
So, what insights did you glean from this? Attention values are pretty intuitive, but did you notice anything surprising in what the various probes produced in a given situation?
Plenty of insights. Rather than probes, I played with the app’s logit lens and was inspired to quantitatively look into compositionality. One insight is that it appears certain tactics like forks are compositional features rather than a new feature in and of itself. You can check the logits of the fork for a piece move and compare it to the sums of the logits for the two individual threats the move creates, and the values are roughly equal.
Interesting. I’m not sure I entirely follow you on “the logits for the two individual threats the move creates”, though.
Fix the board state but remove one of the forked pieces and look at the value of the logit lens for the same fork move without that piece through time.