Daily Insights

Do­cendo discimus

This se­quence records my progress learn­ing con­cepts re­lated to ma­chine learn­ing and AI al­ign­ment. With some ex­cep­tions, I in­tend to post an ex­pla­na­tion of an im­por­tant con­cept ev­ery day.

Walk­through: The Trans­former Ar­chi­tec­ture [Part 1/​2]

Walk­through: The Trans­former Ar­chi­tec­ture [Part 2/​2]

Un­der­stand­ing Batch Normalization

Re­think­ing Batch Normalization

A Sur­vey of Early Im­pact Measures

Un­der­stand­ing Re­cent Im­pact Measures

Four Ways An Im­pact Mea­sure Could Help Alignment

Why Gra­di­ents Van­ish and Explode

A Primer on Ma­trix Calcu­lus, Part 1: Ba­sic review

A Primer on Ma­trix Calcu­lus, Part 2: Ja­co­bi­ans and other fun

A Primer on Ma­trix Calcu­lus, Part 3: The Chain Rule