I have made the case that with the advent of the internet went the need to memorize anything. Why worry about memorizing when I’ll never be tested for a grade and can access knowledge nearly instantaneously? As well, I reasoned, I have probably already memorized everything I need to. I focused my time instead on learning thinking techniques, such as Bayesian calculations, expected value calculations and various things for improving emotional control.
But after reading this a couple months back I decided to experiment with Anki, a digital flashcard program which exploits a cognitive phenomenon called the Spacing Effect by implementing a memorization technique called Spaced Repetition. The Spacing Effect is the widely observed tendency for people to recall information better when studied a few times over a long period than when studied many times over a short period. Balota et al (2007):
Spacing effects occur across domains (e.g., learning perceptual motor tasks vs. learning lists of words), across species (e.g., rats, pigeons, and humans), across age groups and individuals with different memory impairments, and across retention intervals of seconds to months.
You can think of the ‘forgetting curve’ as being like a chart of radioactive half-lives: each review bumps your memory up in strength 50% of the chart, say, but review doesn’t do very much in the early days because the memory simply hasn’t decayed very much! (Chart)
One consequence of the spacing effect is that cramming is useful for recalling things shortly after memorizing them; however, if those crammed memories are not eventually refreshed then they are likely to decay to nothing. From this observation came Spaced Repetition: a memorization technique using flashcards (usually) shown at increasing intervals of time to optimize the relationship between number of reviews and strength of memory. The PC explosion was a boon to Spaced Repetition since storing and showing flashcards as well as physically calculating their frequencies were delegated to the computer. The program Anki, for instance, permits the user to generate flashcard decks, specify study session length and frequency, specify how many new cards are introduced per session and specify the frequency of the cards based on the user’s input. Hard material is shown more often than easy material, with the ease or difficulty being determined directly by the user selecting buttons marked “again,” “easy,” “good,” and “hard.”
That sounds nifty, but how well does it work? As for myself, using the Anki default settings, I was able to thoroughly memorize a deck of 80 cognitive biases and related terms (160 cards total, name to definition and vice avers) in about three weeks using Anki ~15 minutes/day. Since the cards are pushed back further and further for review as I progressed, I have only five cards to review today. The first one, Endowment Effect, came instantly to me so I selected the “easy” button. Now, as a result of the Anki algorithm, I won’t see that card for 1.3 months. My low expectations for the Anki experiment were exceeded.
Piotr Wozniak, who designed the first SuperMemo algorithm in the early 80′s (of which later versions are still in use in SuperMemo as well as Anki), and devoted enormous energies to studying modern computer aided self-instruction systems promotes spaced repetition. He and two others developed a two-variable framework for memorization which they built upon to examine a way of optimizing interval spacing in Spaced Repetition. The first variable, memory retrieval (R), is the probability of recalling something and is approximated by an exponential decay function, while the second variable, memory stability (S), measures how long a memory lasts before it is forgotten entirely. Wozniak et al, expressed S as the inter-repetition interval time that produces R = 90% (the likelihood of recall being a 9 out of 10 chance) and concluded the following:
We express the changes in retrievability as:
(3.1) R=e-d*t
where:
t—time
R—probability of recall at time t (retrievability)
d—decay constant dependent on the stability
We can replace the constant d dependent on stability, with a constant k that is independent of stability:
(3.2) R=e-k*t/S
where:
t—time
R—probability of recall at time t
S—stability expressed by the inter-repetition interval that results in retrievability of 90% (i.e. R=0.9)
k—constant independent of stability
Drawing on analysis of large data sets cultivated from SuperMemo, Wozniak et al provide empirical evidence that memory decay matches their exponential decay approximation. The goal of Wozniak’s SuperMemo algorithm is to optimize inter-repetition spacing by (theoretically) refreshing a memory the moment before it decays totally, thus spiking that memory until it decays near totally again and gets spiked again. (Although, depending on the importance of being able to recall of a piece of information, it can be used, theoretically, to spike a memory every time it decays to likelihood of recall of 90%, 80%, 70%, etc.) Interestingly, in a meta-analysis by Balota et al (2007) , the authors conclude that while spaced repetition is certainly better than massed practice (studying all at once and then not reviewing again), spaced repetition shows no advantage over static spaced repetition (holding intervals constant)! Since most studies cited in the meta-analysis used a small number (usually three) retrieval attempts, the authors suggest that future research should expand this number to better reflect the way people can practically use spaced repetition. In my estimation, when it comes to memorization, given the ease of use of these digital flashcards programs, the specific algorithm design is a secondary concern to being personally disciplined to consistently review material until you think you’ve internalized it fully.
Another consideration beyond algorithm design is formulating a usable flashcard deck: simplifying the information and implementing techniques to enhance recall. Wozniak et al found evidence that it was harder to recall information the more complex it was. Hence, Wozniak recommends 20 rules for formatting knowledge to make flashcards more digestible during reviews. The first three rules are standard: understand before you learn, learn before you memorize and build on the basics. The remaining rules are specific to developing and maintaining flashcard decks, such as simplifying questions, using clozed deletion (a sentence missing a part replaced by three dots), including images, avoiding sets, etc. If you are planning on creating your own deck then familiarize yourself with these rules.
Additionally, I recommend including hyperlinks, if available, in your cards to sources with thorough explanations of the topics, and to be careful doing Wikipedia-based decks. Having small previous knowledge of cognitive biases when I started, it was essential to read expanded explanations on many of the terms to understand them completely, so I actually updated the deck, which someone else created, with hyperlinks on every card. I think it greatly enhanced the usability and effectiveness of the deck. Incidentally, when I went to Wikipedia to better grasp many of the terms, I found several entries there lacking in credibility. On at least two occasions, after being skeptical of an entry on a term, I Googled the term and found every other mention of it on the internet was either sourced to Wikipedia or directly copied from there.
This all still sounds nifty, but, I’ll repeat, why worry about memorizing when I’ll never be tested for a grade and can access knowledge nearly instantaneously? As for standard trivia type information (state capitals, etc), memorization is virtually a total waste of time (thank you, technology!). Instant recall of facts, except on Jeopardy or when using a foreign language, is generally not of value. Think of a time when your inability to instantly recall a fact resulted in a financial loss for you—I can’t. On the other hand, every damn day I am confronted with dynamic situations where I am forced to make quick decisions that vary in effectiveness based on how well I analyze what is happening and construct counter-strategies that maximize my utility. In these moments, when the necessary facts are right in front of us, what we usually don’t have is a comprehensive database of methodologies, heuristics and other decision theoretic knowledge to surf through and use for calculating useful outputs. You might be familiar with Bayesianism, Nonviolent Communication (NVC), PUA, logical fallacies and the like, but it is unlikely you have internalized the concepts to the point where even in the face of chaos or emotional turmoil (when it likely matters most) you can implement them to the best of your mental ability. Thus, I recommend using digital flashcards employing Spaced Repetition to memorize a relatively small set of widely applicable methods (and related knowledge) for use in dynamic situations that require instant or near-instant action.
NVC is the poster-child because it is an easy to remember step-by-step process which does not require complicated inputs for any of the steps; virtually anyone can observe a situation, dissect relevant information from it and then run it through the NVC process. While simple, NVC might be most valuable in chaotic or emotionally-charged social situation when minds are thrust into primate mode, making it that more important to ingrain thoroughly. Divia, who created an NVC deck and several other useful Anki decks, recounts successfully using NVC on a train when a drunk sports fan near her was acting belligerent (imagine that!). In my experience, having internalized a bunch of cognitive biases, I’m feel like I am vigilant about recognizing them in my thoughts and behaviors and in those of others, without devoting much conscious effort to doing so. I expect that databasing of logical fallacies and human behavioral cues will have the same effect. Please list other methods or knowledge that you think would be worth devoting time to memorize.
In sum, spaced repetition for memorization is superior to massed consumption without further review, although it is undetermined what inter-repetition algorithm is best. It seems that having discipline and consistency in review is more important than the inter-repetition spacing, as even static spacing works well. Also important is the design and maintenance of the flash card decks used for spaced repetition exercise, with an emphasis on simplifying the information presented. Lastly, be thoughtful about what things use spend time memorizing. Almost all information is just as useful to us wherever it currently is, especially if it is on the internet, than it would be if we had it memorized. Thus, I suggest using Anki or other spaced repetition software to memorize methods, concepts and knowledge can be deployed in dynamic situations where we are forced to make important decisions in an instant or near-instant.
REFERENCES
Balota, D.A., Duchek, J.M., & Logan, J.M. (2007). Is expanded retrieval practice a superior form of spaced retrieval? A critical review of the extant literature. In J. Nairne (Ed.), The Foundations of Remembering: Essays in Honor of Henry L. Roediger III, (pp. 83-106), Psychology Press, New York.
Memory, Spaced Repetition and Life
I have made the case that with the advent of the internet went the need to memorize anything. Why worry about memorizing when I’ll never be tested for a grade and can access knowledge nearly instantaneously? As well, I reasoned, I have probably already memorized everything I need to. I focused my time instead on learning thinking techniques, such as Bayesian calculations, expected value calculations and various things for improving emotional control.
But after reading this a couple months back I decided to experiment with Anki, a digital flashcard program which exploits a cognitive phenomenon called the Spacing Effect by implementing a memorization technique called Spaced Repetition. The Spacing Effect is the widely observed tendency for people to recall information better when studied a few times over a long period than when studied many times over a short period. Balota et al (2007):
Gwern analogizes the spacing effect with radioactive decay:
One consequence of the spacing effect is that cramming is useful for recalling things shortly after memorizing them; however, if those crammed memories are not eventually refreshed then they are likely to decay to nothing. From this observation came Spaced Repetition: a memorization technique using flashcards (usually) shown at increasing intervals of time to optimize the relationship between number of reviews and strength of memory. The PC explosion was a boon to Spaced Repetition since storing and showing flashcards as well as physically calculating their frequencies were delegated to the computer. The program Anki, for instance, permits the user to generate flashcard decks, specify study session length and frequency, specify how many new cards are introduced per session and specify the frequency of the cards based on the user’s input. Hard material is shown more often than easy material, with the ease or difficulty being determined directly by the user selecting buttons marked “again,” “easy,” “good,” and “hard.”
That sounds nifty, but how well does it work? As for myself, using the Anki default settings, I was able to thoroughly memorize a deck of 80 cognitive biases and related terms (160 cards total, name to definition and vice avers) in about three weeks using Anki ~15 minutes/day. Since the cards are pushed back further and further for review as I progressed, I have only five cards to review today. The first one, Endowment Effect, came instantly to me so I selected the “easy” button. Now, as a result of the Anki algorithm, I won’t see that card for 1.3 months. My low expectations for the Anki experiment were exceeded.
Piotr Wozniak, who designed the first SuperMemo algorithm in the early 80′s (of which later versions are still in use in SuperMemo as well as Anki), and devoted enormous energies to studying modern computer aided self-instruction systems promotes spaced repetition. He and two others developed a two-variable framework for memorization which they built upon to examine a way of optimizing interval spacing in Spaced Repetition. The first variable, memory retrieval (R), is the probability of recalling something and is approximated by an exponential decay function, while the second variable, memory stability (S), measures how long a memory lasts before it is forgotten entirely. Wozniak et al, expressed S as the inter-repetition interval time that produces R = 90% (the likelihood of recall being a 9 out of 10 chance) and concluded the following:
Drawing on analysis of large data sets cultivated from SuperMemo, Wozniak et al provide empirical evidence that memory decay matches their exponential decay approximation. The goal of Wozniak’s SuperMemo algorithm is to optimize inter-repetition spacing by (theoretically) refreshing a memory the moment before it decays totally, thus spiking that memory until it decays near totally again and gets spiked again. (Although, depending on the importance of being able to recall of a piece of information, it can be used, theoretically, to spike a memory every time it decays to likelihood of recall of 90%, 80%, 70%, etc.) Interestingly, in a meta-analysis by Balota et al (2007) , the authors conclude that while spaced repetition is certainly better than massed practice (studying all at once and then not reviewing again), spaced repetition shows no advantage over static spaced repetition (holding intervals constant)! Since most studies cited in the meta-analysis used a small number (usually three) retrieval attempts, the authors suggest that future research should expand this number to better reflect the way people can practically use spaced repetition. In my estimation, when it comes to memorization, given the ease of use of these digital flashcards programs, the specific algorithm design is a secondary concern to being personally disciplined to consistently review material until you think you’ve internalized it fully.
Another consideration beyond algorithm design is formulating a usable flashcard deck: simplifying the information and implementing techniques to enhance recall. Wozniak et al found evidence that it was harder to recall information the more complex it was. Hence, Wozniak recommends 20 rules for formatting knowledge to make flashcards more digestible during reviews. The first three rules are standard: understand before you learn, learn before you memorize and build on the basics. The remaining rules are specific to developing and maintaining flashcard decks, such as simplifying questions, using clozed deletion (a sentence missing a part replaced by three dots), including images, avoiding sets, etc. If you are planning on creating your own deck then familiarize yourself with these rules.
Additionally, I recommend including hyperlinks, if available, in your cards to sources with thorough explanations of the topics, and to be careful doing Wikipedia-based decks. Having small previous knowledge of cognitive biases when I started, it was essential to read expanded explanations on many of the terms to understand them completely, so I actually updated the deck, which someone else created, with hyperlinks on every card. I think it greatly enhanced the usability and effectiveness of the deck. Incidentally, when I went to Wikipedia to better grasp many of the terms, I found several entries there lacking in credibility. On at least two occasions, after being skeptical of an entry on a term, I Googled the term and found every other mention of it on the internet was either sourced to Wikipedia or directly copied from there.
This all still sounds nifty, but, I’ll repeat, why worry about memorizing when I’ll never be tested for a grade and can access knowledge nearly instantaneously? As for standard trivia type information (state capitals, etc), memorization is virtually a total waste of time (thank you, technology!). Instant recall of facts, except on Jeopardy or when using a foreign language, is generally not of value. Think of a time when your inability to instantly recall a fact resulted in a financial loss for you—I can’t. On the other hand, every damn day I am confronted with dynamic situations where I am forced to make quick decisions that vary in effectiveness based on how well I analyze what is happening and construct counter-strategies that maximize my utility. In these moments, when the necessary facts are right in front of us, what we usually don’t have is a comprehensive database of methodologies, heuristics and other decision theoretic knowledge to surf through and use for calculating useful outputs. You might be familiar with Bayesianism, Nonviolent Communication (NVC), PUA, logical fallacies and the like, but it is unlikely you have internalized the concepts to the point where even in the face of chaos or emotional turmoil (when it likely matters most) you can implement them to the best of your mental ability. Thus, I recommend using digital flashcards employing Spaced Repetition to memorize a relatively small set of widely applicable methods (and related knowledge) for use in dynamic situations that require instant or near-instant action.
NVC is the poster-child because it is an easy to remember step-by-step process which does not require complicated inputs for any of the steps; virtually anyone can observe a situation, dissect relevant information from it and then run it through the NVC process. While simple, NVC might be most valuable in chaotic or emotionally-charged social situation when minds are thrust into primate mode, making it that more important to ingrain thoroughly. Divia, who created an NVC deck and several other useful Anki decks, recounts successfully using NVC on a train when a drunk sports fan near her was acting belligerent (imagine that!). In my experience, having internalized a bunch of cognitive biases, I’m feel like I am vigilant about recognizing them in my thoughts and behaviors and in those of others, without devoting much conscious effort to doing so. I expect that databasing of logical fallacies and human behavioral cues will have the same effect. Please list other methods or knowledge that you think would be worth devoting time to memorize.
In sum, spaced repetition for memorization is superior to massed consumption without further review, although it is undetermined what inter-repetition algorithm is best. It seems that having discipline and consistency in review is more important than the inter-repetition spacing, as even static spacing works well. Also important is the design and maintenance of the flash card decks used for spaced repetition exercise, with an emphasis on simplifying the information presented. Lastly, be thoughtful about what things use spend time memorizing. Almost all information is just as useful to us wherever it currently is, especially if it is on the internet, than it would be if we had it memorized. Thus, I suggest using Anki or other spaced repetition software to memorize methods, concepts and knowledge can be deployed in dynamic situations where we are forced to make important decisions in an instant or near-instant.
REFERENCES
Balota, D.A., Duchek, J.M., & Logan, J.M. (2007). Is expanded retrieval practice a superior form of spaced retrieval? A critical review of the extant literature. In J. Nairne (Ed.), The Foundations of Remembering: Essays in Honor of Henry L. Roediger III, (pp. 83-106), Psychology Press, New York.