there are two different modes of learning i’ve noticed.
top down: first you learn to use something very complex and abstract. over time, you run into weird cases where things don’t behave how you’d expect, or you feel like you’re not able to apply the abstraction to new situations as well as you’d like. so you crack open the box and look at the innards and see a bunch of gears and smaller simpler boxes, and it suddenly becomes clear to you why some of those weird behaviors happened—clearly it was box X interacting with gear Y! satisfied, you use your newfound knowledge to build something even more impressive than you could before. eventually, the cycle repeats, and you crack open the smaller boxes to find even smaller boxes, etc.
bottom up: you learn about the 7 Fundamental Atoms of Thingism. you construct the simplest non-atomic thing, and then the second simplest non atomic thing. after many painstaking steps of work, you finally construct something that might be useful. then you repeat the process anew for every other thing you might ever find useful. and then you actually use those things to do something
generally, i’m a big fan of top down learning, because everything you do comes with a source of motivation for why you want to do the thing; bottom up learning often doesn’t give you enough motivation to care about the atoms. but also, bottom up learning gives you a much more complete understanding.
there are two different modes of learning i’ve noticed.
top down: first you learn to use something very complex and abstract. over time, you run into weird cases where things don’t behave how you’d expect, or you feel like you’re not able to apply the abstraction to new situations as well as you’d like. so you crack open the box and look at the innards and see a bunch of gears and smaller simpler boxes, and it suddenly becomes clear to you why some of those weird behaviors happened—clearly it was box X interacting with gear Y! satisfied, you use your newfound knowledge to build something even more impressive than you could before. eventually, the cycle repeats, and you crack open the smaller boxes to find even smaller boxes, etc.
bottom up: you learn about the 7 Fundamental Atoms of Thingism. you construct the simplest non-atomic thing, and then the second simplest non atomic thing. after many painstaking steps of work, you finally construct something that might be useful. then you repeat the process anew for every other thing you might ever find useful. and then you actually use those things to do something
generally, i’m a big fan of top down learning, because everything you do comes with a source of motivation for why you want to do the thing; bottom up learning often doesn’t give you enough motivation to care about the atoms. but also, bottom up learning gives you a much more complete understanding.
Seems to strongly echo Karpathy, in that top-down learning is most effective for building expertise
https://x.com/karpathy/status/1325154823856033793?s=46&t=iz509DJpCAibJadbMh4TvQ