Interested in AI, math, philosophy, psychology. Website: http://levifinkelstein.xyz/
Levi Finkelstein
Karma: 6
IMO the main concept to deeply understand when studying information theory is the notion of information content/self-information/Shannon-information. Most other things seems to be applications or expansions on this concept. For example entropy is just the expected information content when sampling from a distribution. Mutual information is the shared information content in two distributions. KL-divergence describes how much information content you’re getting relative to your choice of encoding. Information gain is the difference in information content after and before you drew a sample.
For this I would recommend this essay written by me. I would also recommend Terrence Tao’s post on internet anonymity. Or if you’ve seen Death Note, Gwern’s post on the mistakes of Light. Also this video on KL divergence. And this video by intelligent systems lab.
The biggest problem I’ve had with spaced repetition and mind mapping is that it’s very difficult and time consuming to represent non-trivial information in such a way that you won’t be fatigued over time (both in the creation and re-studying). In my experience they’re both skills you have to spend a lot of time on for them to be time/energy efficient, and often it’s a better use of your time to just read more and think more.
I think SRS especially is a crazy good learning hack, and it’s a curious question why the seemingly low-hanging fruit hasn’t been picked by more people. I think one large reason is because using SRS comes with a lot of resistance, both in the creation of cards and in how to use the rescheduling buttons/settings. If a newbie tried to use it to learn math most of the time they’ll just get fatigued and give up. In the opposite case where you use it successfully you can get really addicted to it and learn insane amounts. One of my most efficient learning experiences ever was when I read the Russell/Norvig AI book. I read 2 hours every day while making SRS cards of anything I wanted to remember, then reviewing the cards each morning. This made me experience a rush similar to doing a line of cocaine every time I studied the cards, since I felt like I was learning so much.
IMO, for SRS to become more mainstream they have to adopt some method of reducing resistance, for example by making it easier to create good cards with low effort, or making it more obvious to the user how they’re supposed to maneuver the rescheduling buttons. (I think the technique relating to the use of rescheduling buttons is very underappreciated, I’ve experienced enormous gains by changing how I use them. Which maybe seems obvious? Given that the entire point of SRS is that you’re supposed to revisit right before forgetting if you want optimal gains, so if you reschedule wrong it’ll fuck with your efficiency. Also I would generally air on scheduling a card earlier rather than later since if you already know a cards you can easily just click ahead to the next, but if it’s scheduled too late then you won’t be able to reap those long-term recall benefits, and you also don’t feel as good about you learning if you can’t remember the cards.)