# Learning by Doing

Most peo­ple be­lieve very strongly that the best way to learn is to learn by do­ing. Par­tic­u­larly in the field of pro­gram­ming.

I have a differ­ent per­spec­tive. I see learn­ing as very de­pen­dency based. Ie. there are a bunch of con­cepts you have to know. Think of them as nodes. Th­ese nodes have de­pen­den­cies. As in you have to know A, B and C be­fore you can learn D.

And so I’m always think­ing about how to most effi­ciently tra­verse this graph of nodes. The effi­cient way to do it is to learn things in the proper or­der. For ex­am­ple, if you try to learn D with­out first un­der­stand­ing, say A and C, you’ll strug­gle. I think that it’d be more effi­cient to iden­tify your what your holes are (A and C) and ad­dress them first be­fore try­ing to learn D.

I don’t think that the “dive into a pro­ject” ap­proach leads to an effi­cient traver­sal of this con­cept graph. That’s not to say that it doesn’t have its ad­van­tages. Here are some:

1. Peo­ple tend to find the act of build­ing some­thing fun, and thus mo­ti­vat­ing (even if it has no* use other than as a means to the end of learn­ing).

2. It’s of­ten hard to con­struct a cur­ricu­lum that is com­pre­hen­sive enough. Do­ing real world pro­jects of­ten forces you to do things that are hard to oth­er­wise ad­dress.

3. It’s of­ten hard to con­struct a cur­ricu­lum that is or­dered prop­erly. Do­ing real world pro­jects of­ten is a rea­son­ably effi­cient way of travers­ing the graph of nodes.

Per­son­ally I think that as far as 2) and 3) go, pro­jects some­times have their place, but they should be com­bined heav­ily with some sort of more for­mal cur­ricu­lum, and that the pro­jects should be fo­cused pro­jects. My real point is that the trade­offs should always be taken into ac­count, and that an effi­cient traver­sal of the con­cept graph is largely what you’re af­ter.
I should note that I feel most strongly about pro­jects be­ing over­rated in the field of pro­gram­ming. I also feel rather strongly about it for quan­ti­ta­tive fields in gen­eral. But in my limited ex­pe­rience with non-quan­ti­ta­tive fields, I sense that 2) and 3) are too difficult to do for­mally and that pro­jects are prob­a­bly the best ap­prox­i­ma­tions (in 2015; in the fu­ture I an­ti­ci­pate smart tu­tors be­ing way more effec­tive than any pro­ject ever was or can be). For ex­am­ple, I’ve spent some time try­ing to learn de­sign by read­ing books and stuff on the in­ter­net, but I sense that I’m re­ally miss­ing some­thing that is hard to get with­out do­ing pro­jects un­der the guidance of a good in­struc­tor.
What do you guys think about all of this?

Side Notes:

*Some peo­ple think, “pro­jects are also good be­cause when you’re done, you’ve pro­duced some­thing cool!”. I don’t buy this ar­gu­ment.
• In­for­mal re­sponse: “C’mon, how many of the pro­jects that you do as you’re learn­ing ever end up be­ing used, let alone pro­duce real util­ity for peo­ple?”.

• More for­mal re­sponse: I re­ally be­lieve in the idea that pro­duc­tivity of pro­gram­mers differs by or­ders of mag­ni­tude. Ie. some­one who’s 30% more knowl­edge­able might be 100x more pro­duc­tive (as in faster and able to solve more difficult prob­lems). And so… if you want to be pro­duc­tive, you’d be bet­ter off in­vest­ing in learn­ing un­til you’re re­ally good, and then start to “cash in” by pro­duc­ing.

1. Another thing I hate: when peo­ple say, “you just have to prac­tice”. I’ve asked peo­ple, “how can I get good at X?” and they’ve re­sponded, “you just have to prac­tice”. And they say it with that con­de­scend­ing so­phis­ti­cated cyn­i­cism. And even af­ter I prod, they re­main firm in their af­fir­ma­tion that you “just have to prac­tice”. It feels to me like they’re say­ing, “I’m sorry, there’s no way to effi­ciently tra­verse the graph. It’s all the same. You just have to keep prac­tic­ing.” Sorry for the rant-y tone :/​. I think that my Sys­tem I is suffer­ing from the illu­sion of trans­parency. I know that they don’t see learn­ing as travers­ing a graph like I do, and that they’re prob­a­bly just try­ing to give me good ad­vice based on what they know.
2. My thoughts on learn­ing to learn.
• You can’t de­ter­mine what the most effec­tive ways of learn­ing are by sit­ting and think­ing. What does the em­piri­cal ev­i­dence say?

Those “100x pro­gram­mers”: if you try to iden­tify some and look at their his­tory, are they dis­t­in­guished by hav­ing learned things in an un­usual or­der? By be­ing ex­traor­di­nar­ily clever? By work­ing very hard? By start­ing very young?

When you have ac­quired a skill and be­come demon­stra­bly good at it (I mean: there is ac­tual ex­ter­nal ev­i­dence, agreed by oth­ers, that you’re bet­ter than most), has it gen­er­ally been as a re­sult of care­fully or­dered the­o­ret­i­cal learn­ing or has there been an im­por­tant el­e­ment of prac­tice?

Are you sure you aren’t set­ting up a false di­chotomy, be­tween just div­ing into a pro­ject and care­fully or­dered the­o­ret­i­cal learn­ing? You do men­tion the pos­si­bil­ity of com­bin­ing the two but jump im­me­di­ately (with no ev­i­dence or ar­gu­ment I can see) to the as­ser­tion that the pro­jects should be tightly-fo­cused ones in the ser­vice of a care­fully de­signed for­mal cur­ricu­lum.

It seems to me—but this too is based on sit­ting and think­ing rather than on much em­piri­cal ev­i­dence—that we should ex­pect “do­ing” to be very im­por­tant in learn­ing. Your brain is a net­work of neu­rons, and ev­ery­thing we know about the ar­tifi­cial neu­ral net­works we’ve stud­ied sug­gests that the way to teach them to do a thing in­volves hav­ing them do that thing re­peat­edly and ad­just­ing to make them do it bet­ter. (Ob­vi­ous but im­por­tant caveat: real neu­rons do not be­have the ex­act same way as the ones in ANNs; real brains are wired in much more com­pli­cated ways than our ANNs; there is no guaran­tee that what’s true of one is true of the other; the above is in­tended more as in­tu­ition pump than as for­mal ar­gu­ment.) No one would ex­pect to be able to learn (say) to play the trum­pet well just by read­ing books; I don’t see why any­one should ex­pect to be able to learn to write parsers or prove the­o­rems in enu­mer­a­tive com­bi­na­torics or find effec­tive trad­ing strate­gies or de­sign am­plifiers whose out­put sounds good, just by read­ing books. (Or at­tend­ing lec­tures, or any­thing else that doesn’t in­volve a lot of “do­ing”.)

And that isn’t be­cause of gaps in the de­pen­dency tree; it’s be­cause “do­ing” is a very differ­ent ac­tivity from ex­plicit “learn­ing” and brings about differ­ent kinds of changes in the brain, and you need those changes as well as the ones brought about by ex­plicit “learn­ing” if you want to get good at things. (The way it feels from the in­side, to me, is: For­mal learn­ing can en­able you to do a thing by con­sciously work­ing out the steps, but it’s a much less effec­tive way of build­ing “in­tu­ition” and “taste” and “fluency” than ac­tual prac­tice; and if you want to be re­ally good at some­thing, you need those.)

• As gjm men­tioned, don’t bother think­ing through straight for­ward prob­lems that already have plenty of em­piri­cal data on them. Just look up the data. Two min­utes on google brought me to this study:

In-class ac­tivi­ties led to higher over­all scores than any other teach­ing method while lec­ture meth­ods led to the low­est over­all scores of any of the teach­ing meth­ods.

That’s just one ex­am­ple. I’m not go­ing to bother with a com­pre­hen­sive re­view of ex­ist­ing re­search. I’ve heard it from a wide enough va­ri­ety of sources I con­sider re­li­able that pro­ject-based learn­ing is ex­tremely effec­tive that I’ve always con­sid­ered it a low prob­a­bil­ity that care­ful re­search on the is­sue would re­veal any­thing es­pe­cially use­ful. Also, your pro­posed prob­lem of learn­ing in the right or­der has ab­solutely no re­la­tion­ship with pro­ject-based learn­ing. If you need to learn A be­fore learn­ing B, then choose a pro­ject that fo­cuses on A be­fore pick­ing a B-based pro­ject. Both ac­tive and pas­sive learn­ing can lead to A, B, C, and D be­ing taught out of or­der, and both can be used to teach A, B, C, and D in or­der as well.

• I agree with you. Pro­jects can be su­perfi­cial, showy, time-sinks, warm-fuzzy-feel-goods, or out-right mean­ingless soul drains. I have heard that pro­ject wor­ship is a ma­lig­nant dis­ease in the ed­u­ca­tion sys­tem and academia. Pro­fes­sors are as­sessed for tenure based on the quan­tity of pro­jects com­pleted, with­out a thought given to their abil­ity to teach and hardly a glance at the ac­tual merit of their pro­jects. In k-12 schools, end­less pro­jects can cover up the lack of mean­ingful con­tent in a cur­ricu­lum.

On the other hand, pro­jects seem wholly ap­pro­pri­ate for demon­strat­ing that you have a firm grasp of nodes A, B, C, and D. In fact, doesn’t know­ing that there is a pro­ject em­ploy­ing these con­cepts help many peo­ple pay closer at­ten­tion since they have to imag­ine a con­cept’s pos­si­ble ap­pli­ca­tions? How­ever, the knowl­edge, not the pro­ject, must be the goal. When we make pro­jects the goal, peo­ple bandy pro­jects around to rep­re­sent their alleged com­pe­tence.

Prac­tice doesn’t make your knowl­edge com­plete. It re­veals where your knowl­edge is lack­ing. There’s a differ­ence.

• I think the learn to pro­gram by pro­gram­ming adage came from a lack of places teach­ing the stuff that makes peo­ple good pro­gram­mers. I’ve never worked with some­one who has gone through one of the new pro­gram­ming schools, but I don’t think they pur­port to turn out se­nior-level pro­gram­mers, much less 99th per­centile pro­gram­mers. As far as I can tell, folks ei­ther learn ev­ery­thing be­yond the me­chan­ics and al­gorithms of pro­gram­ming from your se­niors in the work­place or dis­cover it for them­self.

So I’d say that there are nodes on the graph that I don’t have la­bels for, and are not taught for­mally as far as I know. The best way to learn them is to read lots of big well writ­ten code bases and try to figure out why ev­ery­thing was done one way and not some other. Se­cond best then maybe is to write a few huge code bases and figure out why things keep fal­ling apart?

• As far as I can tell, folks ei­ther learn ev­ery­thing be­yond the me­chan­ics and al­gorithms of pro­gram­ming from your se­niors in the work­place or dis­cover it for them­self.

… or from Stack Overflow /​ Wikipe­dia, no? When en­coun­ter­ing a difficult prob­lem, one can ei­ther ask some­one more knowl­edge­able, figure it out him­self, or look it up on the in­ter­net.

• I’m talk­ing about things on the level of se­lect­ing which con­cepts are nec­es­sary and use­ful to im­ple­ment in a sys­tem or higher. At the sim­plest that’s rec­og­niz­ing that you have three types of things that have ar­bi­trary at­tributes at­tached and im­ple­ment­ing an un­der­ly­ing thing-with-ar­bi­trary-at­tributes type in­stead of three spe­cial cases. You tend to get that kind of re­view from peo­ple with whom you share a pro­ject and a so­cial re­la­tion­ship such that they can tell you what you’re do­ing wrong with­out offense.

• I ap­prove of learn­ing-by-do­ing sim­ply be­cause the com­mu­ni­ca­ble is a sub­set of the learn­able or know­able.

And of­ten to com­mu­ni­cate a knowl­edge via words is not the fastest way to trans­mit it. Words are high-band­with com­mu­ni­ca­tion if and only if both par­ties know what ex­pe­riences those words mean i.e. there is shared ex­pe­rience. But to it may hard to de­scribe an or­ange to Eski­mos, it is eas­ier to hand one over and say “this”.

Caveat: this re­ally de­pends on the teach­ing meth­ods. For ex­am­ple, videos with ex­er­cises are bet­ter than just books, and even books with ex­er­cises are bet­ter than just books.

A “perfect e-book” would be an AI-men­tor, cor­rect­ing your mis­takes, at that level there is no differ­ence any­more.

Other rea­sons: we of­ten pay no at­ten­tion to a the­ory un­til we see why it is use­ful in prac­tice. For ex­am­ple, in school I they made me mem­o­rize the defi­ni­tion of OOP (in­her­i­tance, poly­mor­phism, en­cap­su­la­tion) and I just mem­o­rized and bar­fed it back with­out be­ing it­ner­ested in it. Many years later I’ve read it is all about avoid­ing com­pli­cated repet­i­tive case state­ments and I got en­light­ened. This was so much more use­ful than our hy­po­thet­i­cal OOP ex­am­ples of mod­el­ling a toaster. I gave no shits about mod­el­ling toast­ers. But when I was do­ing some­thing ac­tu­ally use­ful like a script that makes re­ports from a database into Ex­cel and emails it to a boss so that I don’t fuck­ing have to do them man­u­ally, and I got tired of repet­i­tively writ­ing case sales­re­port do this case pur­chasere­port do that, and the same case state­ment all over, then this de­scrip­tion just made sense: it leads to ac­tual bet­ter ex­pres­sivity.

It is of­ten use­ful to go through the mo­tions first, re­al­ize that HOLY GEE SHIT these mo­tions re­ally make stuff hap­pen OMG my code just drew a bouncy ball! Then be­ing very cu­ri­ous about the the­ory why and learn­ing it vo­ra­ciously. When we do it the other way around we get col­lege stu­dents who are boredly mem­o­rize the­ory as they have no idea what it is for.

• I see learn­ing as very de­pen­dency based. Ie. there are a bunch of con­cepts you have to know. Think of them as nodes. Th­ese nodes have de­pen­den­cies. As in you have to know A, B and C be­fore you can learn D.

Spot on. This is a big prob­lem is math­e­mat­ics ed­u­ca­tion; prior to uni­ver­sity a lot of teach­ing is done with­out pay­ing heed to the fun­da­men­tal con­cepts. For ex­am­ple—here in the UK—calcu­lus is taught well be­fore limits (in fact limits aren’t taught un­til stu­dents get to uni­ver­sity).

Teach­ing is all about cross­ing the in­fer­en­tial dis­tance be­tween the stu­dent’s cur­rent knowl­edge and the idea be­ing taught. It’s my im­pres­sion that most peo­ple who say “you just have to prac­tice,” say as such be­cause they don’t know how to cross that gap. You see this of­ten with pro­fes­sors who don’t know how to teach their own sub­jects be­cause they’ve for­got­ten what it was like not know­ing how to calcu­late the ex­pec­ta­tion of a per­turbed Hamil­to­nian. I sus­pect that in some cases the knowl­edge isn’t truly a part of them, so that they don’t know how to gen­er­ate it with­out already know­ing it.

Pro­jects are a good way to help stu­dents re­tain in­for­ma­tion (the test­ing effect) and also train ap­pro­pri­ate re­call. Ex­perts in a field are usu­ally ex­perts be­cause they can look at a prob­lem and see where they should be ap­ply­ing their knowl­edge—a skill that can only be effec­tively trained by ‘real world’ prob­lems. In my ex­pe­rience teach­ing A-level math stu­dents, the best stu­dents are usu­ally the ones that can ap­ply con­cepts they’ve learned in non-ob­vi­ous situ­a­tions.

You might find this ar­ti­cle I wrote on study­ing in­ter­est­ing.

• Teach­ing is all about cross­ing the in­fer­en­tial dis­tance be­tween the stu­dent’s cur­rent knowl­edge and the idea be­ing taught. It’s my im­pres­sion that most peo­ple who say “you just have to prac­tice,” say as such be­cause they don’t know how to cross that gap.

When the spe­cific in­fer­en­tial dis­tance is re­ally re­ally small, peo­ple can cross it by do­ing. This is how things were in­vented for the first time. And re­peat­ing this dis­cov­ery on your own can be a great feel­ing that gives you con­fi­dence and mo­ti­va­tion. So it could be a good teach­n­ing tech­nique to do this… as long as you have a suffi­ciently good model of your stu­dent, so you know what ex­actly is the “re­ally small dis­tance”, and if you later check whether the new con­cept was un­der­stood cor­rectly.

So I imag­ine that while some teach­ers may re­ally use this as an ex­cuse when they don’t know how to teach, I would be char­i­ta­ble and say that a lot of them prob­a­bly do not have cor­rect un­der­stand­ing of how ex­actly this works (that very small in­fer­en­tial dis­tances can be crossed eas­ily, but large ones can­not), so they just try copy­ing some­one else’s style and fail. Ac­tu­ally, some­times they ran­domly suc­ceed, be­cause once in a while they have a stu­dent who hap­pens to be re­ally close to the new con­cept, and this pre­vents them from giv­ing up their wrong ideas about teach­ing.

• Math ed­u­ca­tion is a spe­cial case as the stu­dents who choose it may not care so much about it s prac­ti­cal use. But in e.g. civil en­g­ineer­ing the stu­dents will be bored by a the­ory if they don’t have a hands-on ex­pe­rience on how this helps mak­ing brick-lay­ing bet­ter.

I went to a busi­ness school, our teach­ers prob­lem was we were bored and un­mo­ti­vated to learn, un­in­ter­ested in the ma­te­rial, we just wanted a pa­per. I think this does not hap­pen in math.

Ap­proach­ing the­ory through prac­ti­cal prob­lems was helpful in this. The smart busi­ness school teacher starts ex­plain­ing the­ory by “you know this guy who just lost a bunch of money?” that makes peo­ple listen

• There’s no need to just com­pare tu­tor­less pro­ject work with cur­ricu­lum based learn­ing.

Of course do­ing a pro­gram­ming pro­ject while hav­ing a men­tor who re­views your code and points out ar­eas of im­prove­ment will lead to bet­ter learn­ing than sim­ply do­ing your pro­ject alone.

• Be­ing 100x more pro­duc­tive is about not solv­ing hard prob­lems you don’t need to. Spend­ing time think­ing about ways to avoid the prob­lem of­ten pays off (fea­ture defi­ni­tion, code reuse, slow im­ple­men­ta­tions, ect). Much of the best prac­tices that you read about are solv­ing prob­lems you wish you had—I wish my prob­lem was poor doc­u­men­ta­tion be­cause that means some­one ac­tu­ally cares to use it. I was always sur­prised by how bad the code was out in the wild un­til I re­al­ized it was sur­vivor bias—the pre­vi­ous owner deferred solv­ing some prob­lem for a long time.

• When do­ing a pro­ject, you can learn new things… or you can keep re­peat­ing your old mis­takes.

As a pro­gram­mer, I am always asked at job in­ter­views how many years of prac­tice do I have; prefer­ably years spent us­ing ex­actly the same sys­tem as the com­pany does. Ob­vi­ously, the idea is that more years = bet­ter. One year makes you not a noob, three years make you a se­nior de­vel­oper, af­ter five or seven years you are con­sid­ered an ex­pert, es­pe­cially if you spent all those years us­ing the very same sys­tems as the com­pany that is con­sid­er­ing hiring you.

I also know a guy who spent more than 10 years as a database ap­pli­ca­tion de­vel­oper, who has never heard about the con­cept of database nor­mal­iza­tion, does not un­der­stand why pri­mary keys are sup­posed to be unique, never heard of re­fac­tor­ing, his code con­sists of thou­sand-lines long un­doc­u­mented func­tions, and… okay, I stop here, be­cause I could go on for­ever.

So, I guess it mat­ters a lot how speci­fi­cally you ap­proach do­ing your pro­jects.

• So, I guess it mat­ters a lot how speci­fi­cally you ap­proach do­ing your pro­jects.

And whether your cowork­ers let you get away with thou­sand-lines long un­doc­u­mented func­tions.

• I mostly taught my­self to pro­gram. Did an in­tro class fresh­man year of col­lege. Five years later my ad­viser had me tackle a cer­tain math prob­lem us­ing the com­puter and to do that, I had to learn a fair bit of pro­gram­ming. I had the dis­tinct im­pres­sion of be­ing thrown into the deep end and told to figure out how to swim.

I no­ticed that there’s a cer­tain un­der­stand­ing that comes from ac­tu­ally ap­ply­ing a con­cept that I just didn’t get from read­ing the book and work­ing through the ex­am­ples. I even­tu­ally picked up the habit of look­ing for a non­triv­ial ap­pli­ca­tion when­ever I came across new pro­gram­ming con­cepts. Of­ten­times the ap­pli­ca­tion I had in mind would re­quire filling in gaps in my knowl­edge. Through­out I’d use a lot of things I didn’t re­ally un­der­stand (by mod­ify­ing ex­ist­ing code) to get things done, and then when I’d come across that part of the the­ory ev­ery­thing would just seem to click.

I figure that most pro­gram­ming tech­niques were in­vented to deal with spe­cific prob­lems, and un­der­stand­ing those prob­lems gives a lot of in­tu­ition about the tech­niques.

Another thing I hate: when peo­ple say, “you just have to prac­tice”. I’ve asked peo­ple, “how can I get good at X?” and they’ve re­sponded, “you just have to prac­tice”. And they say it with that con­de­scend­ing so­phis­ti­cated cyn­i­cism. And even af­ter I prod, they re­main firm in their af­fir­ma­tion that you “just have to prac­tice”. It feels to me like they’re say­ing, “I’m sorry, there’s no way to effi­ciently tra­verse the graph. It’s all the same. You just have to keep prac­tic­ing.”

As Eu­clid once said, “there’s no royal road to ge­om­e­try”. Prac­tice means mak­ing mis­takes, figur­ing out your mis­con­cep­tions and gaps in un­der­stand­ing, then grap­pling with the ma­te­rial un­til it fi­nally makes sense.