RSS

Sin­gu­lar Learn­ing Theory

TagLast edit: 20 Jun 2023 23:25 UTC by DanielFilan

Singluar learning theory is a theory that applies algebraic geometry to statistical learning theory, developed by Sumio Watanabe. Reference textbooks are “the grey book”, Algebraic Geometry and Statistical Learning Theory, and “the green book”, Mathematical Theory of Bayesian Statistics.

DSLT 1. The RLCT Mea­sures the Effec­tive Di­men­sion of Neu­ral Networks

Liam Carroll16 Jun 2023 9:50 UTC
47 points
8 comments13 min readLW link

DSLT 0. Distill­ing Sin­gu­lar Learn­ing Theory

Liam Carroll16 Jun 2023 9:50 UTC
76 points
6 comments5 min readLW link

DSLT 2. Why Neu­ral Net­works obey Oc­cam’s Razor

Liam Carroll18 Jun 2023 0:23 UTC
20 points
14 comments17 min readLW link

DSLT 3. Neu­ral Net­works are Singular

Liam Carroll20 Jun 2023 8:20 UTC
24 points
5 comments19 min readLW link

Spooky ac­tion at a dis­tance in the loss landscape

28 Jan 2023 0:22 UTC
61 points
4 comments7 min readLW link
(www.jessehoogland.com)

An­nounc­ing Timaeus

22 Oct 2023 11:59 UTC
186 points
15 comments4 min readLW link

Gra­di­ent sur­fing: the hid­den role of regularization

Jesse Hoogland6 Feb 2023 3:50 UTC
35 points
6 comments14 min readLW link
(www.jessehoogland.com)

Neu­ral net­works gen­er­al­ize be­cause of this one weird trick

Jesse Hoogland18 Jan 2023 0:10 UTC
166 points
28 comments53 min readLW link
(www.jessehoogland.com)

Ti­maeus’s First Four Months

28 Feb 2024 17:01 UTC
166 points
6 comments6 min readLW link

Growth and Form in a Toy Model of Superposition

8 Nov 2023 11:08 UTC
87 points
5 comments14 min readLW link

In­ves­ti­gat­ing the learn­ing co­effi­cient of mod­u­lar ad­di­tion: hackathon project

17 Oct 2023 19:51 UTC
85 points
4 comments12 min readLW link

DSLT 4. Phase Tran­si­tions in Neu­ral Networks

Liam Carroll24 Jun 2023 17:22 UTC
30 points
3 comments16 min readLW link

Epoch wise crit­i­cal pe­ri­ods, and sin­gu­lar learn­ing theory

Garrett Baker14 Dec 2023 20:55 UTC
9 points
1 comment5 min readLW link

Stage­wise Devel­op­ment in Neu­ral Networks

20 Mar 2024 19:54 UTC
80 points
1 comment11 min readLW link

Sim­ple ver­sus Short: Higher-or­der de­gen­er­acy and er­ror-correction

Daniel Murfet11 Mar 2024 7:52 UTC
89 points
5 comments12 min readLW link

Gen­er­al­iza­tion, from ther­mo­dy­nam­ics to statis­ti­cal physics

Jesse Hoogland30 Nov 2023 21:28 UTC
63 points
9 comments28 min readLW link

Towards Devel­op­men­tal Interpretability

12 Jul 2023 19:33 UTC
172 points
8 comments9 min readLW link

Ap­ply for the 2023 Devel­op­men­tal In­ter­pretabil­ity Con­fer­ence!

25 Aug 2023 7:12 UTC
33 points
0 comments2 min readLW link

You’re Mea­sur­ing Model Com­plex­ity Wrong

11 Oct 2023 11:46 UTC
82 points
15 comments13 min readLW link

[Question] A few Align­ment ques­tions: util­ity op­ti­miz­ers, SLT, sharp left turn and identifiability

Igor Timofeev26 Sep 2023 0:27 UTC
5 points
1 comment2 min readLW link

Sin­gu­lar learn­ing the­ory and bridg­ing from ML to brain emulations

1 Nov 2023 21:31 UTC
26 points
16 comments29 min readLW link

My hopes for al­ign­ment: Sin­gu­lar learn­ing the­ory and whole brain emulation

Garrett Baker25 Oct 2023 18:31 UTC
57 points
5 comments12 min readLW link

My im­pres­sion of sin­gu­lar learn­ing theory

Ege Erdil18 Jun 2023 15:34 UTC
45 points
30 comments2 min readLW link

A short ‘deriva­tion’ of Watan­abe’s Free En­ergy Formula

Wuschel Schulz29 Jan 2024 23:41 UTC
13 points
6 comments7 min readLW link

Sin­gu­lar­i­ties against the Sin­gu­lar­ity: An­nounc­ing Work­shop on Sin­gu­lar Learn­ing The­ory and Alignment

1 Apr 2023 9:58 UTC
87 points
0 comments1 min readLW link
(singularlearningtheory.com)

Jesse Hoogland on Devel­op­men­tal In­ter­pretabil­ity and Sin­gu­lar Learn­ing Theory

Michaël Trazzi6 Jul 2023 15:46 UTC
42 points
2 comments4 min readLW link
(theinsideview.ai)

Open Call for Re­search As­sis­tants in Devel­op­men­tal Interpretability

30 Aug 2023 9:02 UTC
54 points
11 comments4 min readLW link

My Crit­i­cism of Sin­gu­lar Learn­ing Theory

Joar Skalse19 Nov 2023 15:19 UTC
77 points
56 comments12 min readLW link

Learn­ing co­effi­cient es­ti­ma­tion: the details

Zach Furman16 Nov 2023 3:19 UTC
36 points
0 comments2 min readLW link
(colab.research.google.com)

Es­ti­mat­ing effec­tive di­men­sion­al­ity of MNIST models

Arjun Panickssery2 Nov 2023 14:13 UTC
40 points
3 comments1 min readLW link

In­ter­view Daniel Mur­fet on Univer­sal Phenom­ena in Learn­ing Machines

Alexander Gietelink Oldenziel6 Feb 2023 0:00 UTC
44 points
1 comment16 min readLW link
No comments.