Arthur Conmy comments on Activation space interpretability may be doomed

Arthur Conmy 12 Jan 2025 18:49 UTC
7 points
0
the best vector for probing is not the best vector for steering

AKA the predict/control discrepancy, from Section 3.3.1 of Wattenberg and Viegas, 2024
- bilalchughtai 12 Jan 2025 19:24 UTC
  5 points
  4
  Parent
  Also related to the idea that the best linear SAE encoder is not the transpose of the decoder.