Related idea: If we can show that models themselves learn featyres of different granularity, we could then test whether SAEs reflect this difference. (I expect they do not.) This would imply that SAEs capture properties of the data rather than the model.
Related idea: If we can show that models themselves learn featyres of different granularity, we could then test whether SAEs reflect this difference. (I expect they do not.) This would imply that SAEs capture properties of the data rather than the model.