I think any theory of impact that relied on model architectures remaining static was likely doomed. I hope that in studying models without a residual stream, interpretability researchers find techniques that will generalise further.
I think any theory of impact that relied on model architectures remaining static was likely doomed. I hope that in studying models without a residual stream, interpretability researchers find techniques that will generalise further.