A vote against spaced repetition
LessWrong seems to be a big fan of spaced-repetition flashcard programs like Anki, Supermemo, or Mnemosyne. I used to be. After using them religiously for 3 years in medical school, I now categorically advise against using them for large volumes of memorization.
[A caveat before people get upset: I think they appropriate in certain situations, and I have not tried to use them to learn a language, which seems its most popular use. More at the bottom.]
A bit more history: I and 30 other students tried using Mnemosyne (and some used Anki) for multiple tests. At my school, we have a test approximately every 3 weeks, and each test covers about 75 pages of high-density outline-format notes. Many stopped after 5 or so such tests, citing that they simply did not get enough returns from their time. I stuck with it longer and used them more than anyone else, using them for 3 years.
Incidentally, I failed my first year and had to repeat.
By the end of that third year (and studying for my Step 1 boards, a several-month process), I lost faith in spaced-repetition cards as an effective tool for my memorization demands. I later met with a learning-skills specialist, who felt the same way, and had better reasons than my intuition/trial-and-error:
Flashcards are less useful to learning the “big picture”
Specifically, if you are memorizing a large amount of information, there is often a hierarchy, organization, etc that can make leaning the whole thing easier, and you loose the constant visual reminder of the larger context when using flashcards.
Flashcards do not take advantage of spatial, mapping, or visual memory, all of which the human mind is much better optimized for. It is not so well built to memorize pairs between seemingly arbitrary concepts with few to no intuitive links. My preferred methods are, in essence, hacks that use your visual and spatial memory rather than rote.
Here are examples of the typical kind of things I memorize every day and have found flashcards to be surprisingly worthless for:
The definition of Sjögren’s syndrome
The contraindications of Metronidazole
The significance of a rise in serum αFP
Here is what I now use in place of flashcards:
Ven diagrams/etc, to compare and contrast similar lists. (This is more specific to medical school, when you learn subtly different diseases.)
Mnemonic pictures. I have used this myself for years to great effect, and later learned it was taught by my study-skills expert, though I’m surprised I haven’t found them formally named and taught anywhere else. The basic concept is to make a large picture, where each detail on the picture corresponds to a detail you want to memorize.
Memory palaces. I recently learned how to properly use these, and I’m a true believer. When I only had the general idea to “pair things you want to memorize with places in your room” I found it worthless, but after I was taught a lot of do’s and don’ts, they’re now my favorite way to memorize any list of 5+ items. If there’s enough demand on LW I can write up a summary.
Spaced repetition is still good for knowledge you need to retrieve immediately, when a 2-second delay would make it useless. I would still consider spaced-repetition to memorize some of the more rarely-used notes on the treble and bass clef, if I ever decide to learn to sight-read music properly. I make no comment on it’s usefulness to learn a foreign language, as I haven’t tried it, but if I were to pick one up I personally would start with a rosetta-stone-esque program.
Your mileage may vary, but after seeing so many people try and reject them, I figured it was enough data to share. Mnemonic pictures and memory palaces are slightly time consuming when you’re learning them. However, if someone has the motivation and discipline to make a stack of flashcards and study them every day indefinitely, then I believe learning and using those skills is a far better use of time.
- 's comment on Learning Optimization by (29 Apr 2015 13:38 UTC; 14 points)
- ACX Montreal Meetup March 30th 2024 - Spaced Repetition by (22 Mar 2024 20:25 UTC; 6 points)
- 's comment on List a few posts in Main and/or Discussion which actually made you change your mind by (16 Jun 2014 22:37 UTC; 4 points)
- 's comment on Which areas of rationality are underexplored? - Discussion Thread by (6 Dec 2016 0:08 UTC; 3 points)
- 's comment on Managing one’s memory effectively by (7 Jun 2014 4:43 UTC; 2 points)
- 's comment on Less Wrong lacks direction by (27 May 2015 2:25 UTC; 0 points)
Please do.
Seconded. There are quite some comments referencing memory palaces here, but no post.
See also http://en.wikipedia.org/wiki/Method_of_loci
Note that Sherlock uses this method (with funny visualization in the series).
(your formatting is screwed up)
corrected
Now that this topic is buried on page 2, I don’t know if anyone will see this post. However, I’ve begun work on my tutorial. I intend to do a “demo”, constructing a memory palace. Is there a particular list (of about 5-9 items) that people might find universally useful? Memory palaces really need to be constructed by the individual, but for the demo, I’d prefer to to something at least mildly relevant.
Great for starting to work on this.
I don’t understand what you want us to say here. What kind of items?
Thirded. It’s nearly impossible to find a definitive guide on how to do memory palaces.
Possibly stupid question: didn’t Giordano Bruno write the go-to guide for that?
Yeah, in Latin. I don’t think it’s been translated to English, and it’s somewhat suspect now, considering it’s centuries old and was part of Bruno’s obsession with the occult.
This might do: https://en.wikipedia.org/wiki/The_Art_of_Memory
I’ve seen it already. As far as I know it was only history, do you know otherwise?
There’s this: https://en.wikipedia.org/wiki/Moonwalking_with_Einstein
Those are great books. The Art of Memory covers the history of memory techniques. Moonwalking with Einstein is more about the culture of the memorization scene than the techniques. I keep a list of practical books over here.
Again, not a guide.
No, it pretty much is. It’s also got other stuff too is all.
It’s encouraging to see so much interest! I’ll try to pull something together in the next few weeks.
Good information! This is really more “a vote against flashcards” than “a vote against spaced repetition”, though, at least given your concrete issues with flashcards. Spaced repetition is an algorithm for figuring out when to review material that you want to memorize; flashcards are one thing that spaced repetition is applied to, because it’s easy to stick flashcards in a computer. As far as I know, no matter what object-level mnemonic devices you’re using, spaced repetition is still strictly better than “when I feel like I’m forgetting” or “right before a test” or any of the other obvious review strategies, if you can deal with the cognitive load of scheduling things, or get a computer to do it for you.
Is there space for some sort of SRS that allows for input of the more helpful types of memorizations that you listed (pictures, venn diagrams, etc.)?
You are absolutely correct; this is a hair worth splitting. I meant “spaced repetition flashcards”, and I have only seen formal spaced repetition algorithms applied to flashcards. In my particular case, I end up with 30 or so “pages” of related information, as opposed to 500 flashcards. I agree that using spaced repetition algorithms to tell me when to study which page is likely better than alternative methods, though I haven’t found an algorithm optimized for that sort of thing, and at the moment my intuition of “when I’m forgetting” is sufficient for the low number of separate objects to study.
[For this comment, I will use the term “page” to mean any collection of related information, be it a list, table, memory palace, notes on a single topic, etc.]
To be explicit: I vote against using spaced repetition (of any sort) to identify specific facts within a “page” of information. When reviewing a page, of course you can go quickly over the parts you know well and dwell on the parts you don’t, but I would encourage the student to not completely ignore the other details “until it’s time.”
As an example: I have a collection of facts that can be represented as a large table or as individual facts. If I study it in a table, then I get the advantage of keeping the “big picture in mind”, plus I can activate spatial memory as well as rote. If I study it as separate facts: the Pro is I can use spaced repetition to greater effect, not reviewing the parts I know better, but the loss of the picture and the spatial memory makes it not worth the cost. (Note that the “big picture” isn’t a single sentence I can write down; it’s noticing trends in the data, how column A and B are similar except in in key areas, etc.)
As always, my experience is only in high volumes of information that can be organized vaguely hierarchically. (That said, I think if you look hard enough, you can find categories or hierarchies for any large volume of information, outside of truly random things like the sequence of a deck of cards.)
Caveat #2: “self testing” is really important! So if you are quitting flash cards, make sure you find some new way to quiz yourself, don’t just read passively.
Conclusion: -Spaced repetition algorithms might be viable, though I don’t know any suitable to my needs -I claim that spaced repetition flashcards are not useful for large volumes that have categories and/or relationships between facts. -benkuhn rightfully points out that my vendetta is mostly against flashcards, or any method focusing on “terminal facts” in random order without also studying closely related facts.
Life application: 1) If you see a table, don’t vaporize it into your flashcards. Rather, study the table. 2) If you see a mass of new data: ask yourself if you can organize it in a way meaningful to you, then study it in the structure you built.
Why not do both? The thing about SRS flashcards is that facts that you have strong recall of are nearly costless to add, because the review schedules get spaced into very large intervals (weeks, then months or even years), and it only takes a split second to see a card and realize it’s easy. So you could learn from tables if that really is more efficient for you, but having the data in SRS card form as well is a good insurance policy.
Frankly, because at the volume I was running, it was far too great an investment of time. When I stopped, I had about 75-100 scheduled (learned) flashcards per day if I added nothing the day before, though I usually added 60-some every day. The cards would take me 1-2 hours, and the amount kept building as I was adding to it faster than I was pushing them “out”.
Additionally, here our mileage may vary, but even with easy flashcards I occasionally find myself staring dumbly at it for ten or more seconds before I realize what it’s asking and smack my head. So I end up trimming out the stupid-easy ones, but that starts to defeat the purpose. Thus, for myself personally, I won’t duplicate in flashcards what I’m already memorizing elsewhere.
I know that everyone is different, so this is just my experience and what I have observed in other people. If others continue to have success with SRS, then far be it from me to insist they fix what isn’t broken.
To me, that suggests that the card is either too complex and should be split up further, or that you simply do not have solid recall of the relevant facts, so you should just flip to the answer and mark the card difficult. It’s quite normal to forget even some basic info over time; the point of SRS is to refresh these memories at the lowest viable cost.
I appreciate the input, truly, but I can confidently state that’s not the case in my situation. This happens even on the simplest questions that I know cold, and is a problem with mental fatigue, monotony, and reading. After the 100th card, I would expect similar results from “what color is the sky” occasionally. I highly doubt I am dyslexic, but I might be a little ADHD. Once again, I do not presume everyone has similar results, but when I did 150 cards per day (and lord help me if I missed a day), easy cards posed a significant drain on my time and mental energy.
Interesting. If you get that kind of mental fatigue from reading, maybe flashcards really are relatively inefficient for you. If it turns out that dyslexia is the problem, there is an open source font that can help with that issue. Some people have set their SRS up to read questions aloud using computer-generated speech. But yes, most of the time it’s a signal that you should take a break and perhaps switch to some other activty.
One thing that’s relevant to this discussion is that the latest SRS versions can actually cope quite well with missed reviews. Yes, you’ll still be presented with a backlog of suggestions, but the system gives you improved credit if you can recall a card easily even after the increased delay. Because that implies your memory of it was quite good in the first place, so it can be refreshed less often in the future.
Anki is very extensible. I think writing easy-to-use Anki plugins would be a great way to practice coding and get some useful stuff out there. In fact, I’m adding that to my list of things to look into.
This is an idea I had only toyed with but have yet to try in practice, but one can create meta-cards for non-data learning. Instead of creating cards that demand an answer, create cards that demand a drill, or a drill with a specific success outcome. I find it a bit hard to find “the best example” for this, perhaps because the spectrum of learnable-skills is so broad, but just for the sake of illustration: if you’re learning to paint, you can have “draw a still object”, “draw a portrait”, “practice color”, “practice right composition”, “practice perspective” &c, cards. After you finish your card-prompted drill, you move to the next card. Or if you’re practicing going pro at a game (with existing computer program AIs), you can have “Play AI X in a game situation S and achieve A”, “Practice game opening against AI until (able to reach a certain state)”, “practice a disadvantaged end-game situation against AI and bring the game to a draw”, and so on, cards. Of course reviewing the cards would take longer, but they are only meant to be used as scaffolding to harness the Anki spacing algorithm. The numeric parameters of the algorithm might need an adjustment (which is easy to do in Anki) for that, but I think that qualitatively it should work, at least for specific skills. Of course, this set-up, especially if it needs a major parametric-overhauling[1], is an investment, but every human breakthrough required its avant-gardians.
[1] Which is not granted: perhaps the algorithm is only problematic at the beginning of the “learning”, being too frequent, in which case you can just “cheat” carefully and “pass” every other review for a while, which is not a major disturbance. Or, on the contrary, perhaps “well learned cards” (interval > 3 months, or even 1 month, for example) should be discarded for more challenging ones (i.e, “beat the expert AI” replacing “beat beginner AI”, or “juggle 5 balls while riding a unicycle on a mid-air rope” replacing “juggle 4 balls”), which is even less of a problem, as you should immediately recognize well-learned skills (i.e. “practice counting up to 20”).
Unfortunately, Anki and other SRS software do not seem to support card dependencies (i.e. “only show card X if cards Y, Z… are firmly set in memory, as predicted by the spaced-repetition model”). If that was supported, using SRS to memorize heavily-structured data would simply be a matter of setting up appropriate dependencies. (A “memory palace” is really the same thing, except that the highest level of your hierarchy is a spatial model, i. e., your “palace” or “room”, containing the information you want to memorize.)
(One could generalize this by also supporting the option: “prioritize card X when cards Y, Z… are more easily recalled.” Then the deck could even include such things as mutual dependencies, loose associations etc. and the app could use them to “branch out” from what you know already, showing info that can most easily be committed to memory in a highly clustered way.)
The general idea of Anki is that you learn the knowledge first and then put it into Anki to avoid forgetting it.
Not necessarily. In some cases, the flashcard format is quite suited for learning new content as well—especially such things as vocabulary. Allowing inter-card dependencies could easily expand on these use cases.
It would also be directly useful in language learning: for instance, you could memorize some vocabulary words and then be prompted to learn related idioms, or collocations (i.e. words that are “often used together”). Despite its usefulness, this content is quite hard to memorize effectively in the absence of such specialized support.
This is not quite a “tech-tree” dependency structure, but you can use tags to stratify your cards and always review them in sequence from basic to dependent (i.e., first clear out the “basic” cards, then “intermediate”, then “expert”). Even if the grouping is arbitrary, I think you can go a long way with it. If your data is expected to be very large and/or have a predictable structure, you can always go for a “multiple-pyramid” structure, i.e, have “fruits basic” < “fruits advanced” < “fruits expert”, “veggies basics” < “veggies pro” tags &c, and perhaps even have an “edibles advanced” > veggies & fruits tag for very dependent cards.
On the assumption that the Anki algorithm works, just “reviewing down” to an empty deck every tag and proceeding thus sequentially from tag to tag, I think this would work too. Even if it so happened that by one Sunday you forgot “What is an American president” (basic) fact, it might still be profitable to rehearse that day the “Washington was the first president” card, despite the “20 rules” mentioned somewhere above. Presumably, if you had forgotten what a president is, the appropriate card is probably going to appear for review in the next few days, and so with a consistent (or even a semi-consistent) use of Anki, it would probably turn alright. This is more for the anecdotal sake, but this reminds me a time when I burst out laughing out loud while at the dictionary. I was reading at the time “Three Men in a Boat”, and there was one sentence in which I didn’t know 2-3 of the words; the punchline clicked as I read the definition of the last of them.
Either way, somewhere higher on this commenting thread, I have also thought about the possibility (or rather, lack of) of creating dependencies in Anki. I’m actually thinking of creating an add-on/plugin to enable that—I’m learning Python these days (on which Anki runs), and I’m just about to start grad school (if I get admitted), so it seems like just the right time to make this (possibly major) meta-learning investment.*
* Not to mention that, since I’m learning Python, it’s also a (non-meta) learning investment. Win-win.
It was to be expected-- Someone had already created a “hierarchy Tags” addon: https://ankiweb.net/shared/info/1089921461
I haven’t used it myself, but a comment there said “Simple, nice, and easy.”
How long have you been using these alternatives to flashcards? How likely is it that you are still in the honeymoon phase?
Great question. It’s been a long transition from flashcards. I developed the picture technique myself over a year and a half ago. If I learned a fact a year ago, if I studied it using the picture technique, I have about 70% recall if I reviewed it once 6 months ago. If I used flashcards for a month, I have 5% recall now. If I used flashcards continuously (~1-2 hrs per day) for 6 months then stopped using the deck 6 months ago, I have about 10% recall.
(I did a very cursory self-test, then approximated these numbers. It’s very far from perfect, but I didn’t pull them from a hat. For the memory palace numbers, this is pulled from what I’m currently studying, so is reliable.)
I am still in the honeymoon phase for memory palaces. If I reviewed them once a week after creating them, I have about 90% recall one week following that. I currently have 100% recall for the 4 lists ~10 items long that I made last month and have reviewed 3 times each, but haven’t reviewed in the last week.
I think that’s mainly an example where it’s not straightforward to make good cards.
Basically I would get a list that rise in serum αFP does X, Y and Z. Then I would make cards:
Does higher or lower serum αFP does X?
Does higher or lower serum αFP does Y?
Does higher or lower serum αFP does Z?
I personally formulate the cards a bit differently but that’s the core.
I do have Anki cards that contain Ven diagrams. At the beginning the Ven diagram is shown empty and the user is asked where a given item belongs on the Ven diagram.
As answer card the whole Ven diagram is shown and the item that the user had to place is highlited with a special color. I haven’t yet automated the production of such Ven diagrams but I think that’s part of the future of Spaced Repetition Learning.
If you want to learn information the day before a test and get a good score then Anki won’t help you for that purpose. That’s not what it’s made for. Anki exists to prevent you from forgetting information. There a good chance that you will forget your memory palace information in a year. Especially because memory palaces have a reuse problem. If you use the same palace for multiple lists you get conflicts.
I would guess most of those people violate various of Wozniaks rules. If I look at the most popular Anki deck about medicine that’s on Anki shared deck list I fully understand that learning that way isn’t optimal.
That’s a good point; I’m frequently frustrated by existing decks because they tend to include too much information in the answer, or not use as many cues as they could, or not repeat themselves as much, etc. A lot of people do seem to be using Anki sub-optimally, which may explain ancientcampus’s observations.
Can we please taboo “memorize” here? It seems to me the problem is conflating two different mental activities: 1) developing the ability to recall specific discrete responses to stimuli, and 2) gaining cohesive understanding of the component pieces of a “big picture”, and the connections between them. It seems not at all surprising that the best approach for each would differ, with spaced repetition being good for the former and things like venn diagrams, mnemonic pictures, and memory palaces being good for the latter.
Concrete examples for clarity: In the first category would be vocabulary learning, e.g. mapping the stimulus 国 to the response “country”. In the second category would be, say, abstract algebra: learning the group axioms and how they relate to the semigroup axioms in one direction and the abelian group axioms in the other direction.
What would be a good introduction to memory palaces? What did you use?
The video of Brienne’s presentation at the South Bay meetup is the most useful guide I’ve encountered.
Truthfully: I tried and failed miserably when I tried shorter articles (eHow, wikipedia, etc). My study skills coach taught me; his name is Ryan Orwig and teaches medical professionals around the country; he’s talked with memory champions and has ~10 years refining the technique specifically for medicine (but it works with any large body of facts, I think it would help with Law too). So, unfortunately there’s no resource I can point to. I can’t share his powerpoint, but I can make and share my own, which I will do when I have time.
That said, I just skimmed Brienne’s presentation in Ben_LandauTaylor’s link, and it seems to hit many of the points I like. I’ll listen to the whole thing later to see what I have to add.
Link to Ryan Orwig’s class, his travel schedule is on the right: thestatprogram.com
How good is your ability to internally visualize? I’ve been putting off investigating memory palaces for my own use because I find it very difficult to use a mental paintbrush.
My local LW group did some exercises with them and they worked rather well. I don’t have a very good ability to internally visualize but was still able to remember significantly more with a memory palace.
Find a friend to try it a couple of times and see how it works for you.
A great point. I can confidently say mine is at least “average”, likely above average. I consider myself a “visual learner,” with good “story memory” and I agree that as such memory palaces are a particularly good for me. However, when I use the technique, I’d say it’s mostly non-visual. I’d guess it’s 20% me “seeing” the room, 10% “everything else” (texure, sound, smell, emotion, all of which I find much harder but make deliberate effort to employ), and 70% conceptual “The spaceship is crashing through the door, sending shards of wood scattered across the bedroom”. That is one of the many “secrets” that make the technique so useful to me: most every object should perform an action that would in real life permanently damage or alter the room. With tricks like that, I think it is helpful for most all people, even those not visually inclined.
For what it’s worth: I have no data on this myself, but my study coach posits that everyone can do it, some have more trouble than others, but when done well it’s so effective that most everyone should benefit. He says most of his students are resistant, but almost all of them profess loving it once they develop the skillset to use it.
Anki is good for trigger → response sorts of memorization, but requires a bit of hacking for other things. Combining mnemonics with spaced repetition, I’ve heard, is ridiculously powerful. I’ve got a card with three sides, Trigger, Association, and Response, to try and strengthen the trigger → response bond. I’ve set it up so I’ve got Trigger → Response, Association → Response, Trigger → Association and Trigger → Association and Response cards. If anyone wants me to share this format, I’m happy to do so.
ETA: Combining this with habit-training techniques is, I predict, potentially powerful.
What do you mean specifically here by “habit-training techniques”? What you describe above, or something else? (like, beeminder or ksotala’s tips on habits etc.)
Stuff I learned at the Melbourne CFAR workshop. Class name was offline habit training, i.e. actually performing your habit multiple times in a row, in response to the trigger. Salient examples: Practicing getting out of bed in response to your alarm, practice walking in the door and putting your keys where they belong, practice putting your hands on your lap when about to bite nails, practice straightening your neck when you notice you’re hunched. These are all examples I’ve implemented, and I have had good results.
Adding associations is a key part, too. For these examples, I imagine the alarm as an air raid siren and my house getting bombed if I don’t get out of bed on time. I imagine Butch being shot by Vincent in an alternate version of Pulp Fiction if his father’s watch wasn’t on the little kangaroo and he had to hunt around for it. For biting my nails, I imagine Mia Wallace being stabbed in the heart . The connection here is biting nails can make you sick. The vividness and intensity makes up for how tenuous that is. For posture, I imagine Gandalf the Grey compared to Gandalf the White (plus triumphant LoTR music).
Since I made that comment, I got about a third of the way through Moonwalking With Einstein, and practiced the Memory Palace/method of loci a couple of times. I’ve lived in a bunch of different houses, so that works pretty well for me. Some of the stuff that was mentioned sounds a lot like spacing techniques. “”[...] if you revisit the journey through your memory palace later this evening, and again tomorrow afternoon, and perhaps again a week from now, this list will leave a truly lasting impression.”
This is another bit of evidence suggesting that spaced repetition would be powerful in combination with mnemonics. What Anki provides, which is far more important than the flashcard thing, is testing. I’ve been thinking about applying some of the ideas from test-driven development to self-programming, and Anki cards would be a core part of that.
Sorry, I realize most of that isn’t relevant, but I hope the parts that were are useful.
That’s exactly the kind of info I was looking for, thanks :)
Funny. I’ve used triumphant LoTR music once to overcome my terrible fear of heights. I was climbing mount Kathadin with friends (including passing along “Knife Edge ”), and the humming/singing out loud this music (+imagining a chopper-camera shooting from above) has completely effaced my fear. Possibly being called “Legolas” during middle-school and high-school helped, too.
This sounds like a false dilemma. Maybe on their own, flashcards are not very effective, and on their own, mnemonic devices work well enough (for you). But this is not enough to conclude that flashcards are not useful after you adopt mnemonic devices. Maybe the additional effort (of also using spaced repetition, in addition to mnemonic devices) doesn’t significantly improve long term retention, or maybe it does.
There’s an Anki Deck on “The 20 rules of formulating knowledge [in SRS]”. It’s highly recommended for frequent Anki users. Here’s some examples:
Start with the big picture
Refer to other memories
Use mnemonic techniques
Use imagery
Use graphic deletion (e.g. for diagrams, anatomy etc.)
Avoid sets (“contraindications of Metronidazole” would probably be a set) …
So it seems that many of the points you mention are addressed if you use Anki effectively. Your post makes sense though: In my impression 1) most people are not using it as effectively as they could 2) it’s not obvious how to use it effectively 3) effective use of SRS takes time and practice. There’s certainly also cases where it’s just not always the best technique.
Article: http://www.supermemo.com/articles/20rules.htm Deck: http://alexvermeer.com/download/How-to-Formulate-Knowledge.anki
PS: I second the post on memory palaces, sounds really interesting!
While recommending the article I wouldn’t recommend the deck because of cards like:
Front: redundancy can be performed by repeating information using...
Back: various methods
To me that card looks likes trouble.
True. In the deck they make a point about that it’s okay to have really simple cards since those will take a very small share of your time. But some are just useless. I keep a finger on the delete-button while doing my flash cards.
The problem with the card is not that it’s simple.
The problem is that even if I know that there method A, method B and method C for redundancy, my automatic answer to the question isn’t various methods.
I might think of method A, sometimes I might think of method C. Because there’s no clear answer the card doesn’t lend itself to SRS. If I haven’t seen the card in a year I won’t think that the answer to it is “various methods” but I will think of more specific methods.
For what it’s worth: Though I do not claim to be a perfect user of SRS flashcards, I used them intensively for 3 years of medical school, constantly refining my technique. Many people here have suggested ways to improve my strategies. I have not yet seen an idea that I have not already tried extensively. Though I’m far from perfect, I think it’s safe to say I have a better understanding than most beginners. There certainly is room for me to improve, but not much. If someone is considering using SRS long term for high volumes in medical school, here is my advice: it is possible a Perfect SRS User could use it more effectively than I did, but if you haven’t already used SRS for years, you aren’t such a person.
I never read that article, but I figured out many of those on my own. I agree with many of them, disagree with some. My input, for those that use it:
-Cloze deletion is simple, but to me, it is far too easy to “guess the teacher’s password” using that technique, and is of limited use. It’s great for high-school level fact regurgitation, but less useful for post-graduate stuff. You will quickly become good at the deck, but it does not strongly help your understanding of the material. That’s an important point: your skill at answering questions in the deck does not necessarily translate to your skill at answering questions in real life.
-Graphic deletion—I used to do this all the time, but it is really time consuming to set up. I consider myself fast with an image-editor, but it’s still a big drain. (Again, this is more of an issue in high volume) It also runs into the Cloze deletion problem.
-Use imagery: heck yes. I highly agree, in any situation (flashcards or no)
-Any technique splitting a larger whole into many smaller flashcards (the article lists several): This is possibly the WORST suggestion for high volume. While this is certainly very useful, again, when you use it in high volume I have found mental fatigue to become an issue. If you don’t include the entire whole, you miss out on the big picture in a situation where the big picture truly is important. If you DO include the whole, you run into the cloze deletion “guessing the teacher’s password” problem. That said, it has its uses in smaller volume, but I will never again use it in a high-volume deck.
To give an example: As the article suggests, I used to take a diagram, set up graphic deletion (make a series of images where a single element was blotted out), and run through the cards.
1) this takes a lot of startup time
2) Even ignoring the time to make the cards, I found reviewing the cards to be more time consuming than simply looking at the diagram, covering up the lables, and attempting to recall.
3) You get no practice recalling the diagram from memory
4) This technique is most effective if you will later see that exact diagram in real life/on the test, I argue it is a pitfall for guessing the teacher’s password and provides less intuitive understanding of the diagram.
The strength in SRS comes from not wasting time on the easy parts and only spending time on the hard parts of the diagram. The theory is, after the first two cycles, you’re only reviewing the “hard” parts of the diagram. On the other hand, you’ve spent more time making the cards, more time for the first and second card cycles, you’re taking a big hit to the “big picture” style, and have no practice conjuring the diagram itself from memory. Ignoring the big picture and general understanding elements: if SRS provides any time benefit for the rote memorization vs going without SRS cards, i would only expect the benefit to “catch up” in 3 weeks BARE minimum; for me (for fatigue reasons in high volume) I pin the crossing point at 3 to 6 months, assuming it’s an unintuitive diagram I use infrequently enough that I will forget it without review. I also argue that it provides a weaker general understanding of the diagram as a whole.
Just to comment on the last bit: It seems odd to me that you stress the “3 weeks BARE minimum” and the “crossing point at 3 to 6 months” as a con, while you have used SRS for three years. Given that SRS is used for retention, and assuming that 6 months is the “crossing point”, one would think that after three years of consistent SRS use you’d reap a very nice yield.
I know it’s a metaphoric language, but it seems additionally ironic that the “BARE minimum” you stress equals to your frequency of exams, while you disfavor the cloze deletion’s tendency to teach “guessing the teacher’s password”.
Is the advice perhaps against using SRS to learn/cram complex knowledge under a very limited time?
I have used Anki almost daily for the last six months.
Mostly, I use only images to represent concepts.
I think you postulate a dichotomy where none exists.
Yes, text only memory cards are, for many things, not good enough.
Then you say mnemonic pictures are what you use.
Guess what? That’s how I use Anki! I mostly use Anki with mnemonic pictures!
Another interesting case against SRS.
From a Psychology textbook I read (and other sources, including here): “Elaborative Rehearsal” is a kind of reviewing that improves retention: instead of just rereading atomic “facts”, it’s more effective to look for meanings and connotations, to ask “why?”, and to see how it fits in the bigger picture. Having a good understanding of principles and relationships makes transfer easier, i.e. it makes it more likely that you’ll be able to use what you learnt in different contexts (i.e. in daily life and not just on LessWrong / when studying psychology).
Anki seems pretty bad at all of those, so something that puts more emphasis on the global structure of knowledge (like a Memory Palace / Method of Loci) may be better.
On the other hand, I mostly remember the above because I put it in Anki and review my decks every day :)
I’m my experience psychology literature does not easily present atomic facts. Finding atomic facts in a article is a process that means that you have to understand what the article is about.
I took a book about learning theory and it told stories about how Aristotle did this and that and how some experiment turned out in a specific way. It names a bunch of facts but I didn’t found anything that looked like an atomic fact in the first two chapters.
I actually had this exact idea for learning the notes + saxophone fingerings for the treble clef. I was systematically going through Yamaha’s interactive chart, making screen shots and slowly putting them into anki cards during boring, otherwise low-attention demanding meetings.
I never finished the job—I just learned the notes and fingerings by directly practicing the saxophone. I think this is a bit of a parable of one of the challenges with flashcards: if you use the knowledge on the reg, you’ll retain it. If you don’t, you won’t, but if you’re not using it, why do you need it?
The intersection of stuff that’s rarely needed by very high value to know off the top of the head, seems generally small.
Have you tried Anki’s image occlusion and Cloze deletion feature. You can fit entire diagrams or texts that give you the “whole picture” all the while blanking out certain portions of it to test yourself. Anki is great. Admittedly, basic flashcards do have their limitations.
In what program can I create my own unique flashcard repetition algorithm? How?
Not sure about other people/situations, BUT I personally have found, in classroom settings relating to math and CS theory, that a 2-second delay can impede understanding. Especially when a definition relies on a combination of well-chunked previous concepts, which is especially the case when dealing with math.
An interesting comment to this article from the creators of spaced repetition
http://supermemopedia.com/wiki/A_vote_against_spaced_repetition
As spaced repetition and flashcards are a technique and tool respectively it is (to me) obvious that they are useful for certain kinds of circumstances. Flashcards really are useful only when you want to associate 2 things to each other (for example a word and its translation) and might not be the best way to build an organized knowledge of a subject. Because of that I wouldn’t use them for that purpose in any case.
Thank you for pointing out an area where they fail, that was useful information.
A question to the community: Do you really believe as much in spaced repetition/Anki as the post suggests?
I believe it’s really effective when used effectively.
I also think it’s really damned hard to use effectively. There’s a lot of cards I have queued to fix or delete because I didn’t formulate them well enough.
That like saying that computer are only useful when you want to add 0′s and 1′s. There’s Turing completion for computers. In a similar way you can break most knowledge down to atomic units. Often that means that you have to think about the knowledge in a more detailed way.
If I remember right something like 12% use it. On the other hand more than 12% did use it but don’t anymore. On the other hand those people who speak about Anki, are the people who believe in it.
However while I do believe in it I think there’s a huge room for improvement through things like automatic creation of Ven diagrams based cards.
Excellent question; I’d like to know too.
Dunno.
Love,
The Community.
Hm. But when we recall something, we refresh all the connected memories as well. It gives you the whole picture. You may be referring to some specific kinds of flashcards, the ones that only make you recall one particular fact, and that depend on some particular association. Here is an example of flashcards that I created to prepare for my uni course: http://www.memcode.com/courses/18. For example when I answer to this question: ‘How is one bel defined?’, I can’t help but recall:
dB formula (10log10(P1/P2))
that it only refers to power change, or variables proportional to power.
that decibel is 1⁄10 of bel
how are logarithms defined
Ans still it can be answered in about 3 seconds. I recall the big picture in a few seconds. I can’t imagine how any other technique may have a comparable effectiveness.
Is this against spaced repetition as such, or against flash cards?
For me the value of Anki (or my own custom program that I wrote a while back) is as a review-scheduler, not as a quizzer.
Anki became useful for me after I stopped making flashcards on the run, while I was learning the content. Now I make flashcards only from what I already know from memory, two or three days after I’ve learned it through other methods, without ever reading anything while I’m typing into a text file that I’ll then import to Anki ( their GUI is too slow and cumbersome for me ).
Spaced repetition came out of scientific studies on the forgetting curve; these weren’t studies on the learning curve.
A simple substitute strategy for using spaced repetition: Say fact usefulness has a power law distribution: some facts you are going to look up 10s or 100s of times, others not that frequently. Say it’s hard to predict which facts are going to be the ones you look up 100s of times. If that’s true then by using SR you’re going to create a lot of wasted cards for facts that you thought you’d look up 10s or 100s of times but in fact are pretty useless. Instead what you could do is every time you want to look up a fact, before looking it up, try to recall it from memory. Research shows that trying to recall facts solidifies their memories much better than looking them up, so over time you will come to have all of the facts you most frequently need at your mental fingertips using this strategy (a bit like microprocessor cache management).
One of my brothers is a physics undergrad at Caltech. He described the Caltech curriculum as having a “fire hose” feel where the professors throw one thing after another at you in rapid succession, trusting you to reconstruct that knowledge later as necessary. From what I’ve heard, MIT has a similar approach. This seems opposed to a spaced repetition approach where you make sure each chunk of knowledge is a solid, permanent block before proceeding.
One possibility is that the “fire hose” approach does get you spaced repetition for core concepts that you end up seeing in many different ways over the course of your study. It’s also possible that what’s best for elite engineering students doesn’t work well for everyone, or that elite engineering students are going to succeed no matter what approach you take, so curriculum design doesn’t matter much.
What i fail to understand is “Are you sane?” Using ANKI for three years! Mastery for the sake of Mastery is stupid, and not recommended. Mastery for purpose/reason is required. What is a swiss army knife? A swiss army knife is so powerful because it combines a lot of tools and pushes them into one cool gadget. Our mind is that swiss army knife, but you are trying to fill this swiss army knife 1000 times with only one type of weapon i.e. a thousand cork screws or a thousand simple knives, that defeats the purpose of the swiss army knife i.e. our brain. Spaced Repetition System is one of the 1000 systems out there that exist in the world to maximize your memory. The japanese have tons of techniques, which might be inferior than SRS for words but might be extremely efficient when you are trying to remember pictures! i.e. visual memory boost. Similarly Indias have a system to attack math and formulae, from their ancient texts called Vedic Mathematics. There is Benjamin Franklin’s method that he has developed to master writing. There is prousts method to master writing. If you believe that SRS is somehow this magic technique that you can use to remember every damn thing in this world forever! Sorry to say that you are extremely wrong! SRS works but only to some extent, i.e. to help you make associations quickly! So the question you have to ask is “Why do I need those associations?” Our brain does not remember things that you want it to remember, our brain remembers things which it perceives are useful, emotional, shocking, intriguing. You brain remembers things using feelings and emotions. With ANKI you are only forcing it to remember after a certain time with an emotionless character, are you adding smell to that word? i.e smelling a flower are you adding taste to that word i.e. chewing a gum—does somebody slap you when you open up that specific word—using the sense of pain and touch to enhance your memory. Not only does you brain remember the moment when you are humiliated before an audience but it will replay it randomly on multiple occasions out of sudden 10 to 20 years into the future—WHY? because the amount of trauma that a specific incident has caused, made the brain remember it vividly, so that if anything similar to that situation were ever to arise in your life in the future, your brain will recreate this situation and make sure that you don’t repeat the same mistake and go through the same pain. So the people who fail at public speaking for the first time, will remember it for a long long time and will become frightened of the stage. So to imprint something upon the brian, the word must have an emotion/ a feeling attached to it. That is one other technique available. There is one more technique available, You remember the word better with application, if there are situations where you use the word, then with each usage, the word becomes your own, i.e. it moves into your personal inventory and soon you will find reasons to use that word. That is how the brain works. Thus my advice is use ANKI as a repository i.e. similar to a dictionary, don’t try to remember the dictionary. Load all the decks which are related to your area of study and when, only when you come across a word somewhere, you pick up that word from the existing decks and move them to a new deck along with the reason i.e. the context which forced you to look at the deck in the first place, into the DECK and that context is the link that binds you to the word! Without the context, the word merely become an orphan lost in your mind and you don’t have an anchor to recollect. These anchors of touch, context, smell, experiences, emotions, etc… are the key to memory. But, if at any point the brain feels “WHAT IS THE POINT” i.e. if the brain feels APATH attached with a word, this happens after a few months, when you see the word in ANKI so many times but it never gets used in real life ever. Then it is time to move the word into archive i.e. out of the active deck into APATHY DECK, a deck for useless words that take up your mind space but never get used and by now your mind cries about how useless that specific word has become. Thus ANKI is not a savior, it is one of the many tools in the market to help you. Take help from other tools too—Let me list a few tools for you—MINDMAPS - BLUMIND—EVERNOTE—Notetakers—Google Keep—Stickies—FLOWCHART TOOLS—EDRAW—Right tool for the right job and at the right time. Don’t assume that you can keep hitting all the nails with the same hammer, depending on the size of the nail, the hammer must be changed, sometimes, often you need to flip the hammer for a screwdriver—Read these books—Getting Things Done—Fluent Forever and other productivity books to find other tools and techniques used! Don’t just stick to one solution and assume that it is the only thing you will ever need ever again in your life. Do proper research about all the tools at you disposal! Sometimes you might find gems in the rough!
FYI, I had a lot of trouble reading this post due to being a single block of text. (It felt like it was supposed to about 10 paragraphs)
Again Raemon, emotion. The binding factor is emotion. I wrote this answer with a lot of emotion, in a single streak. I did not carefully plan the answer. This topic is very emotional to me, because I have researched about this topic for more than 10 years. I was trying to understand how to become a programmer, during the course of events, I thought, if I could figure out how to ace a natural language, english, german etc… I could use the theory of induction and use the same techniques to master computer languages. During the research, I read 100s of books written by writers and by programmers. I used tens of methods and techniques, SRS was one of them. Anyway, I did not carefully plan the answer or organize the reply, as the thoughts came to my mind, i put them. Thus, I think that you missed the important point that I was trying to say and focussed on the lack of proper structure and missing paragraphs. Well, that’s all I can say. May be there is a logical fallacy or bias for this, where in the subject loses all the important stuff and focusses on the trivial ones, may be grammar nazi ?