Distilling Singular Learning Theory

This sequence distills Sumio Watanabe’s Singular Learning Theory (SLT) by explaining the essence of its main theorem—Watanabe’s Free Energy Formula for Singular Models—and illustrating its implications with intuition-building examples. I show why neural networks are singular models, and demonstrate how SLT provides a framework for understanding phases and phase transitions in neural networks.