Information Theory

Tag

Entropy, and Short Codes

Eliezer Yudkowsky23 Feb 2008 3:16 UTC

72 points

27 comments4 min readLW link

Conditional Independence, and Naive Bayes

Eliezer Yudkowsky1 Mar 2008 1:59 UTC

55 points

16 comments6 min readLW link

Message Length

Zack_M_Davis20 Oct 2020 5:52 UTC

131 points

25 comments12 min readLW link

Introduction to abstract entropy

Alex_Altair20 Oct 2022 21:03 UTC

203 points

73 comments18 min readLW link

The Telephone Theorem: Information At A Distance Is Mediated By Deterministic Constraints

johnswentworth31 Aug 2021 16:50 UTC

93 points

27 comments9 min readLW link 2 reviews

[Question] Where can one learn deep intuitions about information theory?

Valentine16 Dec 2021 15:47 UTC

66 points

25 comments2 min readLW link

Utility Maximization = Description Length Minimization

johnswentworth18 Feb 2021 18:04 UTC

191 points

41 comments5 min readLW link

Using Information Theory to tackle AI Alignment: A Practical Approach

Daniel Salami17 Dec 2022 1:37 UTC

6 points

4 comments7 min readLW link

The Second Law of Thermodynamics, and Engines of Cognition

Eliezer Yudkowsky27 Feb 2008 0:48 UTC

149 points

71 comments9 min readLW link

Msg Len

Zack_M_Davis12 Oct 2020 3:35 UTC

61 points

4 comments1 min readLW link

Generalized Heat Engine

johnswentworth5 Nov 2020 19:01 UTC

156 points

12 comments7 min readLW link

Information Assets

ozziegooen24 Aug 2021 4:32 UTC

42 points

3 comments9 min readLW link

Understanding the Merging of Opinions with Increasing Information theorem

ViktoriaMalyasova21 Apr 2022 14:13 UTC

13 points

1 comment5 min readLW link

Eavesdropping on Aliens: A Data Decoding Challenge

anonymousaisafety24 Jul 2022 4:35 UTC

44 points

9 comments4 min readLW link

[Question] How does one recognize information and differentiate it from noise?

M. Y. Zuo3 Aug 2022 3:57 UTC

4 points

29 comments1 min readLW link

On some first principles of intelligence

Macheng_Shen17 Mar 2023 17:42 UTC

−14 points

0 comments4 min readLW link

Information theory and the symmetry of updating beliefs

Academian20 Mar 2010 0:34 UTC

63 points

29 comments8 min readLW link

Searching for Bayes-Structure

Eliezer Yudkowsky28 Feb 2008 22:01 UTC

56 points

48 comments5 min readLW link

Mutual Information, and Density in Thingspace

Eliezer Yudkowsky23 Feb 2008 19:14 UTC

48 points

28 comments7 min readLW link

Speculations on information under logical uncertainty

TsviBT24 Feb 2016 21:58 UTC

1 point

0 comments12 min readLW link

Combining causality with algorithmic information theory

[deleted]9 Jun 2012 1:31 UTC

21 points

3 comments1 min readLW link

How to Throw Away Information

johnswentworth5 Sep 2019 21:10 UTC

18 points

5 comments5 min readLW link

Law without law: from observer states to physics via algorithmic information theory

avturchin28 Sep 2018 10:07 UTC

16 points

6 comments2 min readLW link

(arxiv.org)

A quick sketch on how the Curry-Howard Isomorphism kinda appears to connect Algorithmic Information Theory with ordinal logics

[deleted]19 Apr 2015 19:35 UTC

26 points

27 comments3 min readLW link

An Undergraduate Reading Of: Semantic information, autonomous agency and non-equilibrium statistical physics

ryan_b30 Oct 2018 18:36 UTC

28 points

2 comments5 min readLW link

[LINK] The Bayesian Second Law of Thermodynamics

shminux12 Aug 2015 16:52 UTC

13 points

16 comments1 min readLW link

My (Mis)Adventures With Algorithmic Machine Learning

AHartNtkn20 Sep 2020 5:31 UTC

16 points

4 comments41 min readLW link

Blind artifacts

So8res7 Apr 2015 23:23 UTC

36 points

2 comments2 min readLW link

KL Divergence as Code Patching Efficiency

Zachary Robertson27 Sep 2020 16:06 UTC

17 points

0 comments8 min readLW link

A Toy Model for Media Sharing

Zachary Robertson17 Oct 2020 15:59 UTC

12 points

0 comments3 min readLW link

Minimal Maps, Semi-Decisions, and Neural Representations

Zachary Robertson6 Dec 2020 15:15 UTC

30 points

2 comments4 min readLW link

[Question] Are the Born probabilities really that mysterious?

Adele Lopez2 Mar 2021 3:08 UTC

45 points

14 comments1 min readLW link

Our compressed perception

pchvykov6 Apr 2021 11:01 UTC

12 points

0 comments2 min readLW link

Mindfulness as debugging

pchvykov30 Apr 2021 16:59 UTC

19 points

4 comments2 min readLW link

Does butterfly affect?

pchvykov14 May 2021 4:20 UTC

14 points

22 comments2 min readLW link

The Blackwell order as a formalization of knowledge

Alex Flint10 Sep 2021 2:51 UTC

41 points

10 comments11 min readLW link

Entropy isn’t sufficient to measure password strength

benwr17 Jan 2022 6:41 UTC

36 points

30 comments2 min readLW link

(www.benwr.net)

Deconfusing Landauer’s Principle

EuanMcLean27 May 2022 17:58 UTC

44 points

12 comments15 min readLW link

A physicist’s approach to Origins of Life

pchvykov28 Jun 2022 15:23 UTC

11 points

6 comments16 min readLW link

What Is The True Name of Modularity?

TheMcDouglas, Lucius Bushnaq and Avery

1 Jul 2022 14:55 UTC

28 points

10 comments12 min readLW link

Information theoretic model analysis may not lend much insight, but we may have been doing them wrong!

Garrett Baker24 Jul 2022 0:42 UTC

7 points

0 comments10 min readLW link

Steganography in Chain of Thought Reasoning

A Ray8 Aug 2022 3:47 UTC

50 points

13 comments6 min readLW link

[Incomplete] What is Computation Anyway?

DragonGod14 Dec 2022 16:17 UTC

8 points

1 comment13 min readLW link

(arxiv.org)

Six (and a half) intuitions for KL divergence

TheMcDouglas12 Oct 2022 21:07 UTC

116 points

16 comments10 min readLW link

(www.perfectlynormal.co.uk)

Word-Distance vs Idea-Distance: The Case for Lanoitaring

Sable6 Nov 2022 5:25 UTC

7 points

7 comments7 min readLW link

(affablyevil.substack.com)

The harnessing of complexity

geduardo10 Nov 2022 18:44 UTC

6 points

2 comments3 min readLW link

A hundredth of a bit of extra entropy

Adam Scherlis24 Dec 2022 21:12 UTC

73 points

4 comments3 min readLW link

No comments.

In­for­ma­tion Theory

Information Theory