A while back, I tried reading Jaynes carefully (i.e. workinglotsofderivations while reading). I’ll share my thoughts, but since I stopped after two and a half chapters, YMMV if you read further.
(1) I felt like I was reading a physics textbook. I’m a recovering physics major, and the experience gave me a serious case of Griffiths deja vu. For example, Jaynes does things like:
Play fast and loose with Taylor series expansions
Give arguments based on intuition and/or symmetry
Assume all functions are well-behaved
Use concepts / notation from calculus that I’ve completely forgotten (or never learned)
If you’ve taken university level physics before, then you should feel somewhat at home reading the first few chapters of Jaynes. If not, I would recommend putting in a bit of extra effort to make sure that you understand EVERY step of important arguments / derivations.
(2) After several days of effort, I got to the point where… you could show that if an urn has 3 red balls and 7 black balls, then the probability of drawing a red ball is 3⁄10. Yay!
Ok, fine, to put it another way, by making a few VERY basic assumptions about reasoning under uncertainty, you can show that the laws of probability are uniquely determined.
If you think this is the coolest revelation ever, then you should definitely read Jaynes. On the other hand, If you’d rather learn how to win at poker, or analyze randomized algorithms, or do calculations about 3d random walks in a cylinder, or something, then Jaynes is probably not the right textbook for you at this time.
So, onto your questions:
What math topics do I already need to understand to prepare myself for this?
Calculus, how to Taylor expand, how to carefully and patiently follow a long argument. I would recommend against going down a deep rabbit hole though (e.g. I would discourage trying to learn “all of multivariable calculus” before starting Jaynes).
Is there a better book to learn probability theory?
It depends; probability theory is a huge topic, and you can attack it from many different angles depending on your goals and interests (e.g. where you want to apply it, whether you’re learning it as a prerequisite for another topic). That said, if what you’re after is LessWrong / CFAR street cred, then I would probably stick with Jaynes ;-)
A while back, I tried reading Jaynes carefully (i.e. working lots of derivations while reading). I’ll share my thoughts, but since I stopped after two and a half chapters, YMMV if you read further.
(1) I felt like I was reading a physics textbook. I’m a recovering physics major, and the experience gave me a serious case of Griffiths deja vu. For example, Jaynes does things like:
Play fast and loose with Taylor series expansions
Give arguments based on intuition and/or symmetry
Assume all functions are well-behaved
Use concepts / notation from calculus that I’ve completely forgotten (or never learned)
If you’ve taken university level physics before, then you should feel somewhat at home reading the first few chapters of Jaynes. If not, I would recommend putting in a bit of extra effort to make sure that you understand EVERY step of important arguments / derivations.
(2) After several days of effort, I got to the point where… you could show that if an urn has 3 red balls and 7 black balls, then the probability of drawing a red ball is 3⁄10. Yay!
Ok, fine, to put it another way, by making a few VERY basic assumptions about reasoning under uncertainty, you can show that the laws of probability are uniquely determined.
If you think this is the coolest revelation ever, then you should definitely read Jaynes. On the other hand, If you’d rather learn how to win at poker, or analyze randomized algorithms, or do calculations about 3d random walks in a cylinder, or something, then Jaynes is probably not the right textbook for you at this time.
So, onto your questions:
Calculus, how to Taylor expand, how to carefully and patiently follow a long argument. I would recommend against going down a deep rabbit hole though (e.g. I would discourage trying to learn “all of multivariable calculus” before starting Jaynes).
It depends; probability theory is a huge topic, and you can attack it from many different angles depending on your goals and interests (e.g. where you want to apply it, whether you’re learning it as a prerequisite for another topic). That said, if what you’re after is LessWrong / CFAR street cred, then I would probably stick with Jaynes ;-)
Here are some alternatives:
Start with a problem book, maybe this one
Go for something that’s a bit more math-ey (and less physics-ey)
Richard Hamming is cool
Thanks!