Learning strategies and the Pokemon league parable
I have recently noted a shift in my learning strategy, which I reflectively approve of. On hindsight, it feels obvious.
However, I can vividly recall many people I respect and admire recommending me to try a very similar thing in the past, and myself scoffing at them and trusting my gut over their advice.
Take this as a word of caution if you feel like the advice I am giving is obviously wrong: I have fallen in this pit, and it took me a while to climb out.
I claim there are two broad learning strategies one can follow.
The default strategy is what in the software engineering lingo is called a waterfall strategy. People attend college and different courses and read books, and gain knowledge on a broad collection of subjects. Afterwards, they move to a second phase where they try to apply what they have learned. If they cannot reach their goals with their current strategy, they back down to the first phase and start again.
I claim that this strategy has some glaring flaws, which I plan to expose via my experience in an area of great importance: Pokemon videogames.
When I got my first videoconsole, the first videogame I ever owned was Pokemon Gold. I absolutely loved that game, and I spent many hours absorbed trying to complete it.
For the most part, the level of challenge was adequate for an 8 year old, but there comes a point where the game suddenly spikes up in difficulty: the Pokemon league. In the league, you have to defeat five trainers with full teams of high level Pokemon in a row.
When I first confronted the Pokemon league, I was quite under leveled and I was utterly crushed.
My response to the problem was to back down and go to easier areas, where I could train with easier challenges.
After around 10h of training, I came back to the League and defeated the five trainers with relatively ease, and I won the title of Pokemon master, officially achieving my most ambitious 8 year old goal.
“Well Jaime”, you may say, “That does not look like your learning strategy had a problem. You successfully completed the challenge!”
And yes, I did. But it was awfully inefficient! I leveled at a very slow rate by fighting easier fights, I ended up training for too long and I picked up tactics to fight other trainers instead of learning the tactics that would have been optimal to fight through the league.
If instead I had kept fighting the trainers in the league, I would have leveled up much faster and I would have learned what tactics were good against each of the elite trainers I had to fight.
I claim that similar things happen to the people like myself two years ago who apply the waterfall strategy:
You end up wasting time learning stuff you do not need. Even worse, because of fade-out effects you will forget most of what you learned. Think about how many useful things did you learn in school that you still remember today.
You do not learn at an optimal rate, because it’s hard to calibrate for problems that are just above your current level.
There is no clear stop point of transitioning between learning mode and solving mode, which further results in overtraining.
Is there an alternative? Yes there is! I introduce to you project-based learning, aka the agile strategy. Instead of generally learning and then solving, run head first into the problem, and learn to fail effectively. When the brick wall in front of you refuses to move, think about what you need to learn to overcoming, and while you learn the techniques make an effort to apply them to the task of wall moving.
You will learn more about the problem and whether the techniques you are learning are actually useful this way.
When I introspect on why I thought waterfall strategy was a better fit for me than project based learning, I find the following reasons:
Waterfall is a good strategy when it is hard to determine what one needs to learn to solve a problem. It is easy to say “you never know if it is going to be useful” and just keep learning useless stuff. But there is virtue in precision, and I have learned to be willing to make more concrete predictions (“there is a low chance than learning French will pay off in the long run given my current trajectory”).
Humans like finding connections between disparate areas, and it feels really pleasant to connect two different things together (like for example playing Pokemon videogames and explaining the differences between strategies for learning). I am actually unsure of how much of this is true insight and how much is just confirmation bias (if I had not played Pokemon, maybe another example would have occurred to me).
Banging your head against a wall feels really bad, and just repeatedly trying to solve a problem through sheer willpower is not a strategy I would recommend. But that is not a very good excuse to skip the first try: sometimes the wall is made of paperboard, and you want to take advantage of that.
I am running out of time to write more, and I feel I have not given enough examples.
The reflection that initiated this post was me two years ago reading a new math textbook every month vs me now taking weeks off at a time trying to solve open research problems. I feel like the second strategy is proving much more useful to give me a feel of whether this is a good way of reaching my goals, and giving me a better map of what I need to learn to solve the research problems (turns out that it’s not so much about learning technical topics but rather about learning how to write better and be organized while pursuing research directions).
I keep seeing people wanting to contribute to AI alignment research or other important research areas and resorting to waterfall strategies instead of project based learning. This post is for them.
If you have an example you want to share or any thoughts, please contribute to the conversation in the comments section!
Darmani in the comments below points out that I am confusing what is normally called project-based learning (doing projects to learn generally useful skills) with what is normally called pull-based learning (learning skills to do a concrete project). Ooops!
This post meant to recommend the second one, and the agile vs waterfall analogy was meant to point out that if you have a concrete goal you are working towards, you can use it to constantly check whether you are already in a good position to make progress on the problem you really care about.
Following his recommendation, I also want to share a note on how I generated this post:
The generator of this post is a combination of the following observations:
1) I see a lot of people who keep waiting for a call to adventure
2) Most knowledge I have acquired through life has turned out to be useless, non transferable and/or fades out very quickly
3) It makes sense to think that people get a better grasp of what skills they need to solve a problem (such as producing high quality AI Alignment research) after they have grappled with the problem. This feels specially true when you are in the edge of a new field, because there is no one else you can turn to who would be able to compress their experience in a digestible format.
4) People (specially in mathematics) have a tendency to wander around aimlessly picking up topics, and then use very few of what they learn. Here I am standing on not very solid ground, because conventional wisdom is that you need to wander around to “see the connections”, but I feel like that might be just confirmation bias creeping in.
As someone who does a whole lot of pull-based learning, I’m going to chime in and say that using it as your main method of learning is probably not the best idea. tl;dr: Learning on the job is powerful, but it overfits by nature; while there’s probably more than a little confirmation bias from us ivory tower types, it’s almost certainly drowned out by “everything comes back to math and logic” and “the truth is all of a piece”.
There is a fairly natural divide, IMO, between “engineering fields” and “theoretical fields”—fields that are directly aimed at solving actual problems, and fields that are more about exploring what is possible and figuring out what is true in general. Pull-based learning is tempting in engineering fields for all the reasons you list—most of being able to solve a real world problem is precisely knowing about all the little fiddly bits of reality, and there is not yet a good way of predicting which fiddly bits you need to know while sitting in your armchair (metaphorically speaking.) In that regard you can get pretty damn far learning only what you “need to know”, to the point of finishing a number of fairly large projects...
… but your knowledge is fundamentally grounded in air, and it’s not always obvious how your experience generalizes. To generalize knowledge you need to build an abstract model, and the skill of building abstract models is very, very theoretical (almost by definition!). In particular, using pull-based learning as a primary tool reminds me of trying to learn physics without first learning math as a fundamental. Certainly, you can make use of what other people have done, and when they start pulling out integrals and Lagrangians and group symmetries you can go look that up and learn it then—but that won’t let you make your own generalizations, or (as Darmani says) contribute to pushing the boundaries on your own.
Personally, I find pull-based learning most useful as the first/last step in a loop. You do some project, and midway through you find there’s a whole bunch of stuff you wish you knew better and learn just enough to get it done. But then you go off and take a course in the dangling ends you discovered, and maybe explore a few branches off that tree too, before you come back to doing some new project challenging enough to force you to learn on the job again—and also enough to make you use your new skills, judge which of them are most important, and generally see them in a new light/in practice. To use the Pokemon metaphor, it’s like, after losing against the Elite Four, you boot up someone else’s save file right before the Elite Four in a totally different version, and try to pick up general “anti-Elite Four” tricks in general rather than “oh, this particular Elite Four has a Ghost specialist, Ice specialist, Fire specialist, and Steel/Psychic specialist, in that order”, which doesn’t generalize to other games.
As someone who might be seen as engaging in a waterfall approach, these are good points. One additional consideration: not everything you learn in completing a math textbook is knowledge; you also pick up skills, grooves in your mind and habits. In the context of AI alignment research, learning to write proofs is particularly relevant to having a security mindset: one begins to appreciate the rhythm by which one finds all relevant edge cases. This kind of approach is required if you want to say things that hold in all contexts; in my project-based research work, I have found that this groove seriously contributes to reasoning soundly about what an alignment proposal will do, and how it breaks down.
I’m confused about whether you’re talking about “learning things specifically to solve a problem” (which I’ve seen called “pull-based learning”), or “learning things by doing projects” (i.e.: project-based learning). The former differs from the “waterfall method” (“push-based learning”) only in the sequence and selection: it’s just the difference between doing a Scala tutorial because you want to learn Scala, vs. because you just got put on a project that uses Scala (and hence you can skip parts of the tutorial the project doesn’t use).
For actual PBL: I am a PBL skeptic. I’ve seen so many people consider it self-evident that learning physics by building a catapult is superior to doing textbook problems that I wrote a blog post to highlight some of the major downsides: http://www.pathsensitive.com/2018/02/the-practice-is-not-performance-why.html . I’ve seen it become a fad, but I’ve not seen the After I wrote the blog post, I had a lot of people tell me about their negative experiences with PBL. One that stands out is a guy who took a PBL MOOC on driverless cars, and didn’t like it because they spent too much time learning about how to use some special pieces of software rather than anything fundamental or transferable.
Quick summary:
Advantages of PBL:
More motivating to some
Includes all aspects of practice needed in performance (e.g.: does not omit the skill of integrating many smaller skills together)
Disadvantages:
Does not naturally lead to correct sequencing of knowledge
Not optimized for rapid learning; does not teach subskills independently
May omit skills which are useful for compressing knowledge, but not directly useful in practice (e.g.: learning chord structure makes it easier to memorize songs, but is not directly used in performing music)
May include overly-specific, non-reusable knowledge
I don’t think PBL works very efficiently. I think it can produce a lot of successful practitioners, but have trouble seeing how it could produce someone able to push the boundaries of a field. I will gladly pay $10 to anyone who can give me an example of someone well-regarded in mathematics (e.g.: multiple publications in top journals in the past decade, where this person was the primary contributor) who acquired their mathematics chiefly by PBL (i.e.: not studying mathematics except for what is needed to work on a specific problem, concurrently with working on the problem).
I actually got directed to your article by another person before this! Congrats on creating something that people actually reference!
On hindsight, yeah, project based learning is nor what I meant nor a good alternative to traditional learning; if you can use cheat codes to speed up your learning using the experience from somebody else you should do so without a doubt.
The generator of this post is a combination of the following observations:
1) I see a lot of people who keep waiting for a call to adventure
2) Most knowledge I have acquired through life has turned out to be useless, non transferable and/or fades out very quickly
3) It makes sense to think that people get a better grasp of what skills they need to solve a problem (such as producing high quality AI Alignment research) after they have grappled with the problem. This feels specially true when you are in the edge of a new field, because there is no one else you can turn to who would be able to compress their experience in a digestible format.
4) People (specially in mathematics) have a tendency to wander around aimlessly picking up topics, and then use very few of what they learn. Here I am standing on not very solid ground, because conventional wisdom is that you need to wander around to “see the connections”, but I feel like that might be just confirmation bias creeping in.
Put that in your post! I got what you’re saying way better after reading that.