Maxime Riché answers Are Mixture-of-Experts Transformers More Interpretable Than Dense Transformers?

[Maxime Riché]

Relevant paper: MONET: MIXTURE OF MONOSEMANTIC EXPERTS FOR TRANSFORMERS

LessWrong: Monet: Mixture of Monosemantic Experts for Transformers Explained