Peter Chatain comments on Negative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (GDM Mech Interp Team Progress Update #2)

[Peter Chatain]

a L1 penalty that penalizes large latent activations, JumpReLU (middle) and TopK (bottom) SAEs …

This should say :
JumpReLU (top)