In post 4, we also studied a toy model mapping pairs of integers to arbitrary labels where we knew all the data and could generate as much as we liked, and didn’t find the toy model any easier to interpret, in terms of finding internal sparsity or meaningful intermediate states.
I just read all of post 4
https://www.lesswrong.com/s/hpWHhjvjn67LJ4xXX/p/JRcNNGJQ3xNfsxPj4
There is nothing there about a toy model mapping pairs of integers.
What am I missing?