Thanks for sharing this, very cool to see active research in this direction! It’s exciting to see the field starting to move this way because there is so much potential here.
Getting the geometry right isn’t an easy jump to land, but it’s clear that they’re on to something given their steering successes. The transition of the atomic unit from direction to its own subspace is a rock solid way of handling the linear/nonlinear complexities of navigating manifold routing.
I’m fully in agreement with the authors that SAEs leave so much on the table by ignoring features’ internal geometry. That said, I think they don’t move far enough as they’re still using lossy tools reading concepts off of fit. (To their credit they are doing a very clean implementation, but a clean implementation of a lossy method is still lossy.)
The logical upgrade here is a method that allows for reading the exact manifold off the weights instead of using a dictionary to rank approximations. That’s where things really start to unlock from what I’m seeing.
Thanks for sharing this, very cool to see active research in this direction! It’s exciting to see the field starting to move this way because there is so much potential here.
Getting the geometry right isn’t an easy jump to land, but it’s clear that they’re on to something given their steering successes. The transition of the atomic unit from direction to its own subspace is a rock solid way of handling the linear/nonlinear complexities of navigating manifold routing.
I’m fully in agreement with the authors that SAEs leave so much on the table by ignoring features’ internal geometry. That said, I think they don’t move far enough as they’re still using lossy tools reading concepts off of fit. (To their credit they are doing a very clean implementation, but a clean implementation of a lossy method is still lossy.)
The logical upgrade here is a method that allows for reading the exact manifold off the weights instead of using a dictionary to rank approximations. That’s where things really start to unlock from what I’m seeing.