RGRGRG comments on Finding Sparse Linear Connections between Features in LLMs

[RGRGRG]

To confirm—the weights you share, such as 0.26 and 0.23 are each individual entries in the W matrix for:
y=Wx ?

[Logan Riggs]

Correct. So they’re connecting a feature in F2 to a feature in F1.