Distilling Singular Learning Theory

This sequence distills Sumio Watanabe’s Singular Learning Theory (SLT) by explaining the essence of its main theorem—Watanabe’s Free Energy Formula for Singular Models—and illustrating its implications with intuition-building examples. I show why neural networks are singular models, and demonstrate how SLT provides a framework for understanding phases and phase transitions in neural networks.

DSLT 0. Distill­ing Sin­gu­lar Learn­ing Theory

DSLT 1. The RLCT Mea­sures the Effec­tive Di­men­sion of Neu­ral Networks

DSLT 2. Why Neu­ral Net­works obey Oc­cam’s Razor

DSLT 3. Neu­ral Net­works are Singular

DSLT 4. Phase Tran­si­tions in Neu­ral Networks