Seonglae Cho comments on Showing SAE Latents Are Not Atomic Using Meta-SAEs

[Seonglae Cho]

What happens when we learn Meta-SAE’s decoder weights again? Meta-Meta-SAE? 🤔

I can only expect greater lossy decomposition