focuses on helping you build models with gears about how information and related concepts works
has a bit of a bias towards talking about signals, like over a phone line, but this is actually really helpful because it’s the original application of most of the information theory metaphors
dives into thermodynamics without getting bogged down in a lot of calculations you’ll probably never do
I think you’ll also appreciate that it is self-aware that information theory is a big deal and a deep concept that explains a lot of stuff, and has some chapters on some of the far reaching stuff. There’s chapters at the end on cybernetics, psychology, and art.
This was one of the books that had the most impact on me and how I think and I basically can’t recommend it highly enough.
I came here to suggest the same book which I think of as “that green one that’s really great”.
One thing I liked about it was the way that it makes background temperature into a super important concept that can drive intuitions that are more “trigonometric/geometric” and amenable to visualization… with random waves always existing as a background relative to “the waves that have energy pumped into them in order to transmit a signal that is visible as a signal against this background of chaos”.
“Signal / noise ratio” is a useful phrase. Being able to see this concept in a perfectly quiet swimming pool (where the first disturbance that generates waves produces “lonely waves” from which an observer can reconstruct almost exactly where the first splash must have occurred) is a deeper thing, that I got from this book.
Okay, gotta punch up my recommendation a little bit.
About 10 years ago I moved houses and, thanks to the growing popularity of fancy ebooks, I decided to divest myself of most of my library. I donated 100s of books that weighted 100s of pounds and ate up 10s of boxes. I kept only a small set of books, small enough to fit in a single box and taking up only about half a shelf.
An Introduction to Information Theorymade the cut and I still have my copy today, happily sitting on a shelf next to me as I type. It’s that good and that important.
I highly recommend An Introduction to Information Theory: Symbols, Signals and Noise by John R. Pierce. Several things that are great about this book:
it’s short and concise, like many Dover books
focuses on helping you build models with gears about how information and related concepts works
has a bit of a bias towards talking about signals, like over a phone line, but this is actually really helpful because it’s the original application of most of the information theory metaphors
dives into thermodynamics without getting bogged down in a lot of calculations you’ll probably never do
I think you’ll also appreciate that it is self-aware that information theory is a big deal and a deep concept that explains a lot of stuff, and has some chapters on some of the far reaching stuff. There’s chapters at the end on cybernetics, psychology, and art.
This was one of the books that had the most impact on me and how I think and I basically can’t recommend it highly enough.
I came here to suggest the same book which I think of as “that green one that’s really great”.
One thing I liked about it was the way that it makes background temperature into a super important concept that can drive intuitions that are more “trigonometric/geometric” and amenable to visualization… with random waves always existing as a background relative to “the waves that have energy pumped into them in order to transmit a signal that is visible as a signal against this background of chaos”.
“Signal / noise ratio” is a useful phrase. Being able to see this concept in a perfectly quiet swimming pool (where the first disturbance that generates waves produces “lonely waves” from which an observer can reconstruct almost exactly where the first splash must have occurred) is a deeper thing, that I got from this book.
Okay, gotta punch up my recommendation a little bit.
About 10 years ago I moved houses and, thanks to the growing popularity of fancy ebooks, I decided to divest myself of most of my library. I donated 100s of books that weighted 100s of pounds and ate up 10s of boxes. I kept only a small set of books, small enough to fit in a single box and taking up only about half a shelf.
An Introduction to Information Theory made the cut and I still have my copy today, happily sitting on a shelf next to me as I type. It’s that good and that important.