RSS

In­for­ma­tion Theory

Tag

En­tropy, and Short Codes

Eliezer Yudkowsky23 Feb 2008 3:16 UTC
78 points
27 comments4 min readLW link

Con­di­tional In­de­pen­dence, and Naive Bayes

Eliezer Yudkowsky1 Mar 2008 1:59 UTC
63 points
16 comments6 min readLW link

[Question] How does one rec­og­nize in­for­ma­tion and differ­en­ti­ate it from noise?

M. Y. Zuo3 Aug 2022 3:57 UTC
4 points
29 comments1 min readLW link

The Se­cond Law of Ther­mo­dy­nam­ics, and Eng­ines of Cognition

Eliezer Yudkowsky27 Feb 2008 0:48 UTC
176 points
74 comments9 min readLW link

[Question] New­bie ques­tions about in­for­ma­tion the­ory and transformers

Misaligned-Semi-intelligence21 Jun 2023 22:45 UTC
10 points
1 comment1 min readLW link

Mu­tual In­for­ma­tion, and Den­sity in Thingspace

Eliezer Yudkowsky23 Feb 2008 19:14 UTC
66 points
28 comments7 min readLW link

Six (and a half) in­tu­itions for KL divergence

CallumMcDougall12 Oct 2022 21:07 UTC
155 points
25 comments10 min readLW link1 review
(www.perfectlynormal.co.uk)

Search­ing for Bayes-Structure

Eliezer Yudkowsky28 Feb 2008 22:01 UTC
62 points
48 comments5 min readLW link

Mes­sage Length

Zack_M_Davis20 Oct 2020 5:52 UTC
134 points
25 comments12 min readLW link

In­tro­duc­tion to ab­stract entropy

Alex_Altair20 Oct 2022 21:03 UTC
229 points
78 comments18 min readLW link1 review

The Tele­phone The­o­rem: In­for­ma­tion At A Dis­tance Is Me­di­ated By Deter­minis­tic Constraints

johnswentworth31 Aug 2021 16:50 UTC
96 points
26 comments9 min readLW link2 reviews

[Question] Where can one learn deep in­tu­itions about in­for­ma­tion the­ory?

Valentine16 Dec 2021 15:47 UTC
67 points
27 comments2 min readLW link

Utility Max­i­miza­tion = De­scrip­tion Length Minimization

johnswentworth18 Feb 2021 18:04 UTC
209 points
44 comments5 min readLW link

Us­ing In­for­ma­tion The­ory to tackle AI Align­ment: A Prac­ti­cal Approach

Daniel Salami17 Dec 2022 1:37 UTC
10 points
4 comments7 min readLW link

You Can’t Pre­dict a Game of Pinball

Jeffrey Heninger30 Mar 2023 0:40 UTC
73 points
12 comments6 min readLW link
(aiimpacts.org)

Eaves­drop­ping on Aliens: A Data De­cod­ing Challenge

anonymousaisafety24 Jul 2022 4:35 UTC
44 points
9 comments4 min readLW link

Un­der­stand­ing the Merg­ing of Opinions with In­creas­ing In­for­ma­tion theorem

ViktoriaMalyasova21 Apr 2022 14:13 UTC
16 points
1 comment5 min readLW link

New­ton’s law of cool­ing from first principles

Nisan16 Jan 2024 4:21 UTC
23 points
13 comments2 min readLW link

On some first prin­ci­ples of intelligence

Macheng_Shen17 Mar 2023 17:42 UTC
−14 points
0 comments4 min readLW link

Gen­er­al­ized Heat Engine

johnswentworth5 Nov 2020 19:01 UTC
186 points
11 comments7 min readLW link

Msg Len

Zack_M_Davis12 Oct 2020 3:35 UTC
65 points
4 comments1 min readLW link

In­for­ma­tion Assets

ozziegooen24 Aug 2021 4:32 UTC
42 points
3 comments9 min readLW link

Knowl­edge Base 8: The truth as an at­trac­tor in the in­for­ma­tion space

iwis25 Apr 2024 15:28 UTC
−10 points
0 comments2 min readLW link

Re­vis­it­ing the Man­i­fold Hypothesis

Aidan Rocke1 Oct 2023 23:55 UTC
10 points
19 comments4 min readLW link

[Question] Are the Born prob­a­bil­ities re­ally that mys­te­ri­ous?

Adele Lopez2 Mar 2021 3:08 UTC
45 points
14 comments1 min readLW link

Our com­pressed perception

pchvykov6 Apr 2021 11:01 UTC
12 points
0 comments2 min readLW link

Mind­ful­ness as debugging

pchvykov30 Apr 2021 16:59 UTC
19 points
4 comments2 min readLW link

Does but­terfly af­fect?

pchvykov14 May 2021 4:20 UTC
14 points
21 comments2 min readLW link

The Black­well or­der as a for­mal­iza­tion of knowledge

Alex Flint10 Sep 2021 2:51 UTC
41 points
10 comments11 min readLW link

En­tropy isn’t suffi­cient to mea­sure pass­word strength

benwr17 Jan 2022 6:41 UTC
36 points
30 comments2 min readLW link
(www.benwr.net)

De­con­fus­ing Lan­dauer’s Principle

EuanMcLean27 May 2022 17:58 UTC
59 points
12 comments15 min readLW link

A physi­cist’s ap­proach to Ori­gins of Life

pchvykov28 Jun 2022 15:23 UTC
12 points
6 comments16 min readLW link

What Is The True Name of Mo­du­lar­ity?

1 Jul 2022 14:55 UTC
38 points
10 comments12 min readLW link

In­for­ma­tion the­o­retic model anal­y­sis may not lend much in­sight, but we may have been do­ing them wrong!

Garrett Baker24 Jul 2022 0:42 UTC
7 points
0 comments10 min readLW link

Steganog­ra­phy in Chain of Thought Reasoning

A Ray8 Aug 2022 3:47 UTC
61 points
13 comments6 min readLW link

[In­com­plete] What is Com­pu­ta­tion Any­way?

DragonGod14 Dec 2022 16:17 UTC
16 points
1 comment13 min readLW link
(arxiv.org)

Word-Dis­tance vs Idea-Dis­tance: The Case for Lanoitaring

Sable6 Nov 2022 5:25 UTC
7 points
7 comments7 min readLW link
(affablyevil.substack.com)

The har­ness­ing of complexity

geduardo10 Nov 2022 18:44 UTC
6 points
2 comments3 min readLW link

A hun­dredth of a bit of ex­tra entropy

Adam Scherlis24 Dec 2022 21:12 UTC
83 points
4 comments3 min readLW link

Erdős Prob­lems in Al­gorith­mic Probability

Aidan Rocke11 Sep 2023 16:44 UTC
13 points
4 comments2 min readLW link

“In­f­tox­i­c­ity” and other new words to de­scribe mal­i­cious in­for­ma­tion and com­mu­ni­ca­tion thereof

Jáchym Fibír23 Dec 2023 18:15 UTC
−1 points
6 comments3 min readLW link

LMSR sub­sidy pa­ram­e­ter is the price of information

Abhimanyu Pallavi Sudhir25 May 2024 18:05 UTC
4 points
0 comments1 min readLW link

The en­tropy maxim for bi­nary questions

dkl911 Feb 2024 17:17 UTC
2 points
1 comment1 min readLW link
(dkl9.net)

Knowl­edge Base 7: Long-tail knowl­edge and col­lec­tive intelligence

iwis18 Apr 2024 14:21 UTC
−8 points
0 comments1 min readLW link

In­for­ma­tion-The­o­retic Box­ing of Superintelligences

30 Nov 2023 14:31 UTC
30 points
0 comments7 min readLW link

Towards Mea­sures of Optimisation

12 May 2023 15:29 UTC
53 points
37 comments4 min readLW link

In­for­ma­tion the­ory and the sym­me­try of up­dat­ing beliefs

Academian20 Mar 2010 0:34 UTC
64 points
29 comments8 min readLW link

Spec­u­la­tions on in­for­ma­tion un­der log­i­cal uncertainty

TsviBT24 Feb 2016 21:58 UTC
1 point
0 comments12 min readLW link

Com­bin­ing causal­ity with al­gorith­mic in­for­ma­tion theory

[deleted]9 Jun 2012 1:31 UTC
21 points
3 comments1 min readLW link

How to Throw Away Information

johnswentworth5 Sep 2019 21:10 UTC
18 points
5 comments5 min readLW link

Law with­out law: from ob­server states to physics via al­gorith­mic in­for­ma­tion theory

avturchin28 Sep 2018 10:07 UTC
16 points
6 comments2 min readLW link
(arxiv.org)

A quick sketch on how the Curry-Howard Iso­mor­phism kinda ap­pears to con­nect Al­gorith­mic In­for­ma­tion The­ory with or­di­nal logics

[deleted]19 Apr 2015 19:35 UTC
26 points
27 comments3 min readLW link

An Un­der­grad­u­ate Read­ing Of: Se­man­tic in­for­ma­tion, au­tonomous agency and non-equil­ibrium statis­ti­cal physics

ryan_b30 Oct 2018 18:36 UTC
29 points
2 comments5 min readLW link

[LINK] The Bayesian Se­cond Law of Thermodynamics

shminux12 Aug 2015 16:52 UTC
13 points
16 comments1 min readLW link

My (Mis)Ad­ven­tures With Al­gorith­mic Ma­chine Learning

AHartNtkn20 Sep 2020 5:31 UTC
16 points
4 comments41 min readLW link

Kol­mogorov’s the­ory of Al­gorith­mic Probability

Aidan Rocke3 Aug 2023 0:58 UTC
5 points
2 comments2 min readLW link
(keplerlounge.com)

Blind artifacts

So8res7 Apr 2015 23:23 UTC
36 points
2 comments2 min readLW link
No comments.