Global Threshold—Let’s treat all features the same. Set all feature activations less than [0.1] to 0 (this is equivalent to adding a constant to the encoder bias).
The bolded part seems false? This maps 0.2 original act → 0.2 new act while adding 0.1 to the encoder bias maps 0.2 original act → 0.1 new act. Ie, changing the encoder bias changes the value of all activations, while thresholding only affects small ones
The bolded part seems false? This maps 0.2 original act → 0.2 new act while adding 0.1 to the encoder bias maps 0.2 original act → 0.1 new act. Ie, changing the encoder bias changes the value of all activations, while thresholding only affects small ones
Ah, you’re right. I’ve updated it.