The results seem a bit mixed regarding “probes trained on SAE features perform better than probes”. I see 3 times SAEs are significantly better and 2 times the raw probe is better.
The results seem a bit mixed regarding “probes trained on SAE features perform better than probes”. I see 3 times SAEs are significantly better and 2 times the raw probe is better.