Reconciling Shannon and Bayes.

Link post

If information is the resolution of uncertainty then we are all ignorant or dead. There can always be more data, things can always change, you can always find out later you missed something. If information is the resolution of uncertainty, then information is a choice. It’s the choice to decide the scope of a scientific study or the scope of a software release version. At some point you need to choose when enough is, well, enough, and decide.

Maybe information is the reduction of uncertainty. Maybe it’s both.

Maybe it’s Bayes.

Sometimes it would seem like rationalists treat Bayes as 01 switch, and Bayes would be dismayed. Probability is not a switch, it’s a gradient. Treating 90% probability as factI is the quickest way to be wrong. 0.1/​0.9 is more like it. Updating your assumptions continuously is the way to go.

This is to say, they are both correct. Shannon is right: at some point, you have to resolve. You trust the “sent” message in your app unless something goes wrong that makes you change your mind. And there’s a chance > 0 that something went wrong. But you resolve–not because it’s perfect, but because it’s good enough.

Shannon is correct, less uncertainty = more information.

Bayes is correct, nothing is ever certain, and everything may be updated.

Beyond Shannon and Bayes, lies agency: acting in the midst of uncertainty.


These paragraphs are part of the white paper/​manifesto for Wall Street Weather, a free, public dashboard recording market traded entities’ metrics, cryptographically signed and broadcast by economic agents.

Come read more, red team, give your opinion and let’s build it.

https://​​github.com/​​Laureana/​​wallstreetweather