In­for­ma­tion Theory

En­tropy, and Short Codes

Con­di­tional In­de­pen­dence, and Naive Bayes

Mes­sage Length

The Tele­phone The­o­rem: In­for­ma­tion At A Dis­tance Is Me­di­ated By Deter­minis­tic Constraints

[Question] Where can one learn deep in­tu­itions about in­for­ma­tion the­ory?

Utility Max­i­miza­tion = De­scrip­tion Length Minimization

The Se­cond Law of Ther­mo­dy­nam­ics, and Eng­ines of Cognition

Msg Len

Gen­er­al­ized Heat Engine

In­for­ma­tion Assets

Un­der­stand­ing the Merg­ing of Opinions with In­creas­ing In­for­ma­tion theorem

In­for­ma­tion the­ory and the sym­me­try of up­dat­ing beliefs

Search­ing for Bayes-Structure

Mu­tual In­for­ma­tion, and Den­sity in Thingspace

Spec­u­la­tions on in­for­ma­tion un­der log­i­cal uncertainty

Com­bin­ing causal­ity with al­gorith­mic in­for­ma­tion theory

How to Throw Away Information

Law with­out law: from ob­server states to physics via al­gorith­mic in­for­ma­tion theory

A quick sketch on how the Curry-Howard Iso­mor­phism kinda ap­pears to con­nect Al­gorith­mic In­for­ma­tion The­ory with or­di­nal logics

An Un­der­grad­u­ate Read­ing Of: Se­man­tic in­for­ma­tion, au­tonomous agency and non-equil­ibrium statis­ti­cal physics

[LINK] The Bayesian Se­cond Law of Thermodynamics

My (Mis)Ad­ven­tures With Al­gorith­mic Ma­chine Learning

Blind artifacts

KL Diver­gence as Code Patch­ing Efficiency

A Toy Model for Me­dia Sharing

Min­i­mal Maps, Semi-De­ci­sions, and Neu­ral Representations

[Question] Are the Born prob­a­bil­ities re­ally that mys­te­ri­ous?

Our com­pressed perception

Mind­ful­ness as debugging

Does but­terfly af­fect?

The Black­well or­der as a for­mal­iza­tion of knowledge

En­tropy isn’t suffi­cient to mea­sure pass­word strength

De­con­fus­ing Lan­dauer’s Principle

A physi­cist’s ap­proach to Ori­gins of Life