This covers a really impressive range of material—well done! I just wanted to point out that if someone followed all of this and wanted more, Shannon’s 1948 paper is surprisingly readable even today and is probably a nice companion:
http://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf
This covers a really impressive range of material—well done! I just wanted to point out that if someone followed all of this and wanted more, Shannon’s 1948 paper is surprisingly readable even today and is probably a nice companion:
http://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf