Don’t be afraid of the thousand-year-old vampire

[Ulisse Mini]

Background

I first started reading Jaynes Probability Theory around 2 years ago, I did not last long, I wasn’t able to follow the derivations in Chapter 2, so I gave up, assuming it would only get harder from there what was the point in continuing?

This was a big mistake! For several reasons

1. Trying to follow every detail is dumb, you’re there to learn not to pass a trivia test on solving the associativity equation. You’re here to learn probability not how to solve functional equations

2. It doesn’t get harder, it gets easier! You should always skim a textbook before reading it (and especially before giving up)

As I’m finally rectifying my mistake I figured I’d write an explanation for what Chapter 2 is actually about

Think “Alien” Not Robot

The reasoning in Chapter 2 shows that any rules of plausible-inference must match our own theory of probability after a suitable change of units. The convoluted functional-equation argument is needed to construct a function $p:R\to [0,1]$ which translates the alien’s plausibility $A|B$ into our probability $p\left(A|B\right)$.

This is very different to how $p$ is usually defined, for Jaynes $p$ is a translation between the plausibilities of the alien and our “nicer” probabilities that obey the sum and product rules. Contrast this with standard probability where $p$ is a function from subsets of the sample space to real numbers obeying certain axioms.

Any alien civilization’s concept of “probability the dice lands five” must be a monotonic function of our probability $1/6$. That’s amazing when you think about it, It shows that probability is discovered not invented. ^[1]

Furthermore, if you want the sum and product rule to take their natural forms you have to pick our units, this is analogous to how degrees Kelvin are defined to make the laws of thermodynamics look natural. ^[2]

Near the end of Chapter 2 (after we’ve shown uniqueness) Jaynes switches to a more traditional function $P:Prop\to [0,1]$ where $Prop$ is some logical proposition like “the dice lands five”.

Why Jaynes doesn’t mention the intuitive explanation until after dragging you through a hard-to-follow argument involving tons of functional equations and Calculus I have no idea, I guess Thousand-year old vampires are bad at pedagogy? ^[3]

Some study tips

Hopefully I’ve convinced you Jaynes isn’t that scary, and motivated you to start reading. A few quick tips

• Check out these lectures for higher level (and often better) explanations then in the book

• It’s fine not to be able to solve problems, you’re here to learn, not get your ego boosted from solving every exercise! Above all don’t use this as an excuse to give up.

• Actually follow other people’s advice. It’s all too easy to read something, go “yeah that’s obvious” then not to follow it, I’ve run into this failure mode and so have others.

---

1. ↩︎

   I feel a popularization of these ideas should be possible, they feel more fundamental (read: philosophically interesting) than most popularized science. Cantor’s theorem pales in comparison! Somebody get Numberphile to in on this!

2. ↩︎

   Or, so I’m told. I don’t know thermodynamics lol

3. ↩︎

   This has also made me think the world deserves a “Second edition” that corrects some of his egregious teaching, If you know of something like this (other books which take his approach to probability) please let me know

[philh]

I had difficulty with what I think was that chapter too. I asked about one part on math.stackexchange, it looks like someone answered my immediate question but judging by my followup there might have been more that I didn’t understand. (I no longer remember enough math to really understand the question or answer.)

[Ulisse Mini]

Yeah I read about 1/​3d of the proof of Cox’s theorem until I realized even if I followed every step I wouldn’t gain any intuition from it, then I skipped the rest

[Carlos Javier Gil Bellosta]

I believe that what Jaynes does is quite standard: start with a minimalistic set of axioms (or principles, or whatever) and work your way to the intiuitive results later on. Euclid geometry is just like that!

I just skimmed over the details of the proofs (and I am a mathematician by training!). I did not read Jaynes for such details. I just guess that if they were wrong, somebody would have already reported them. The meaty part is elsewhere.