The Truth About Mathematical Ability

There’s wide­spread con­fu­sion about the na­ture of math­e­mat­i­cal abil­ity, for a va­ri­ety of rea­sons:

  • Most peo­ple don’t know what math is.

  • Most peo­ple don’t know enough statis­tics to an­a­lyze the ques­tion prop­erly.

  • Most math­e­mat­i­ci­ans are not very metacog­ni­tive.

  • Very few peo­ple have more than a ca­sual in­ter­est in the sub­ject.

If the na­ture of math­e­mat­i­cal abil­ity were ex­clu­sively an ob­ject of in­tel­lec­tual in­ter­est, this would be rel­a­tively in­con­se­quen­tial. For ex­am­ple, many peo­ple are con­fused about Ein­stein’s the­ory of rel­a­tivity, but this doesn’t have much of an im­pact on their lives. But in prac­tice, peo­ple’s mis­con­cep­tions about the na­ture of math­e­mat­i­cal abil­ity se­ri­ously in­terfere with their own abil­ity to learn and do math, some­thing that hurts them both pro­fes­sion­ally and emo­tion­ally.

I have a long stand­ing in­ter­est in the sub­ject, and I’ve found my­self in the un­usual po­si­tion of be­ing an ex­pert. My ex­pe­riences in­clude:

  • Com­plet­ing a PhD in pure math at Univer­sity of Illinois.

  • Four years of teach­ing math at the high school and col­lege lev­els (pre­calcu­lus, calcu­lus, mul­ti­vari­able calcu­lus and lin­ear alge­bra)

  • Per­sonal en­coun­ters with some of the best math­e­mat­i­ci­ans in the world, and a study of great math­e­mat­i­ci­ans’ bi­ogra­phies.

  • A long his­tory of work­ing with math­e­mat­i­cally gifted chil­dren: as a coun­selor at MathPath for three sum­mers, through one-on-one tu­tor­ing, and as an in­struc­tor at Art of Prob­lem Solv­ing.

  • Study­ing the liter­a­ture on IQ and pa­pers from the Study of Ex­cep­tional Ta­lent as a part of my work for Cog­nito Men­tor­ing.

  • Train­ing as a full-stack web de­vel­oper at App Academy.

  • Do­ing a large scale data sci­ence pro­ject where I ap­plied statis­tics and ma­chine learn­ing to make new dis­cov­er­ies in so­cial psy­chol­ogy.

I’ve thought about writ­ing about the na­ture of math­e­mat­i­cal abil­ity for a long time, but there was a miss­ing el­e­ment: I my­self had never done gen­uinely origi­nal and high qual­ity math­e­mat­i­cal re­search. After com­plet­ing much of my data sci­ence pro­ject, I re­al­ized that this had changed. The ex­pe­rience sharp­ened my un­der­stand­ing of the is­sues.

This is a the first of a se­quence of posts where I try to clar­ify the situ­a­tion. My main point in this post is:

There are sev­eral differ­ent di­men­sions to math­e­mat­i­cal abil­ity. Com­mon mea­sures rarely as­sess all of these di­men­sions, and can paint a very in­com­plete pic­ture of what some­body is ca­pa­ble of.

What is up with Grothendieck?

I was sad­dened to learn of the death of Alexan­der Grothendieck sev­eral months ago. He’s the math­e­mat­i­cian who I iden­tify with the most on a per­sonal level, and I had hoped to have the chance to meet him. I hes­i­tated as I wrote the last sen­tence, be­cause some read­ers who are math­e­mat­i­ci­ans will roll their eyes as they read this, ow­ing to the con­no­ta­tion (even if very slight) that the qual­ity of my re­search might over­lap with his. The ma­te­rial be­low makes it clear why:

“His tech­ni­cal su­pe­ri­or­ity was crush­ing,” Thom wrote. “His sem­i­nar at­tracted the whole of Parisian math­e­mat­ics, whereas I had noth­ing new to offer. — Rene Thom, 1958 Fields Medalist

“When I was in in Paris as a stu­dent, I would go to Grothendieck’s sem­i­nar at IHES… I enoyed the at­mo­sphere around him very much … we did not care much about pri­or­ity be­cause Grothendieck had the ideas that we were work­ing on and pri­or­ity would have meant noth­ing. — Pierre Deligne, 1978 Fields Medalist

“[The IHES] is a re­mark­able place.. I knew about it be­fore I came there; it was a leg­endary place be­cause of Grothendieck. He was kind of a god in math­e­mat­ics.” — Mikhail Gro­mov, 2010 Abel Prize Winner

“On ar­riv­ing at the IHES, we or­di­nary math­e­mat­i­ci­ans share the same feel­ing that Mus­lims ex­pe­rience on a pil­gri­mage to Mecca. Here is the place were, for a dozen or so years, Grothendieck re­lentlessly ex­plained the holy word to his apos­tles. Of that saga, only the apoc­rypha reached us in the form of big, yel­low, bor­ing-look­ing books ed­ited by Springer. Th­ese dozens of vol­umes...are still our most pre­cious work­ing com­pan­ion.” — Ngo Bau Chau, 2010 Fields Medalist

Based on these re­marks alone, it seems hard to imag­ine how I could be any­thing like Grothen­de­ick. But when I read Grothendieck’s own de­scrip­tion of him­self, it’s haunt­ingly fa­mil­iar. He writes:

“I’ve had the chance...to meet quite a num­ber of peo­ple, both among my “el­ders” and among young peo­ple in my gen­eral age group, who were much more brilli­ant, much more “gifted” than I was. I ad­mired the fa­cil­ity with which they picked up, as if at play, new ideas, jug­gling them as if fa­mil­iar with them from the cra­dle—while for my­self I felt clumsy. even oafish, wan­der­ing painfully up a ar­du­ous track, like a dumb ox faced with an amor­phous moun­tain of things that I had to learn ( so I was as­sured), things I felt in­ca­pable of un­der­stand­ing the es­sen­tials or fol­low­ing through to the end. In­deed, there was lit­tle about me that iden­ti­fied the kind of bright stu­dent who wins at pres­ti­gious com­pe­ti­tions or as­similates, al­most by sleight of hand, the most for­bid­ding sub­jects.”

When I men­tioned this to pro­fes­sor at a top math de­part­ment who had taken a class with Grothendieck, he scoffed and said that he didn’t be­lieve it, ap­par­ently think­ing that Grothendieck was putting on airs in the above quo­ta­tion – en­gag­ing in a sort of brag­ging, along the lines of “I’m so awe­some that even though I’m not smart I was still one of the great­est math­e­mat­i­ci­ans ever.” It is hard to rec­on­cile Grothendieck’s self-de­scrip­tion with how his col­leagues de­scribe him. But I was stunned by the pro­fes­sor’s will­ing­ness to dis­miss the re­marks of some­body so great out of hand.

In fair­ness to the pro­fes­sor, I my­self am much bet­ter situ­ated to un­der­stand how Grothendieck’s re­marks could be sincere and faith­ful than most math­e­mat­i­ci­ans are, be­cause of my own un­usual situ­a­tion.

What is up with me?

I went to Low­ell High School in San Fran­cisco, an aca­demic mag­net school with ~650 stu­dents per year, who av­er­aged ~630 on the math SAT (81st per­centile rel­a­tive to all col­lege bound stu­dents). The math de­part­ment was very stringent with re­spect to al­low­ing stu­dents to take AP calcu­lus, ap­par­ently out of a self-in­ter­ested wish to keep their av­er­age AP scores as high as pos­si­ble. So de­spite the strength of the school’s stu­dents, Low­ell only al­lowed 10% of stu­dents to take AP Calcu­lus BC. I was one of them. The teach­ers made the ex­ams un­usu­ally difficult for an AP Calcu­lus BC course, so that stu­dents would be greatly over pre­pared for the AP exam . The re­sult was that a large ma­jor­ity of stu­dents got 5′s on the AP exam. By the end of the year, I had the 2nd high­est cu­mu­la­tive av­er­age out of all stu­dents en­rol­led in AP Calcu­lus BC. It would have been the high­est if the av­er­age had de­ter­mined ex­clu­sively by tests, rather than home­work that I didn’t do be­cause I already knew how to do ev­ery­thing.

From this, peo­ple un­der­stand­ably in­ferred that I’m un­usu­ally brilli­ant, and thought of me as one of the se­lect few who was a nat­u­ral math­e­mat­i­cian, hav­ing abil­ity per­haps pre­sent in only 1 in 1000 peo­ple. When I pointed out that things had not always been this way, and that I had in fact failed ge­om­e­try my fresh­man year and had to re­take the course, their re­ac­tions tended to be along the lines of Qiaochu’s re­sponse to my post How my math skills im­proved dra­mat­i­cally:

I find this post slightly dis­in­gen­u­ous. My ex­pe­rience has been that math­e­mat­ics is heav­ily g-loaded: it’s just not fea­si­ble to progress be­yond a cer­tain point if you don’t have the work­ing mem­ory or in­for­ma­tion pro­cess­ing ca­pac­ity or what­ever g fac­tor ac­tu­ally is to do so. The main con­clu­sion I draw from the fact that you even­tu­ally com­pleted a Ph.D. is that you always had the g for math; given that, what’s mys­te­ri­ous isn’t how you even­tu­ally performed well but why you started out perform­ing poorly.

It’s not at all mys­te­ri­ous to me why I started out perform­ing poorly. In fact, if Qiaochu had known only a lit­tle bit more, he would be less in­cre­d­u­lous.

Aside from tak­ing AP Calcu­lus BC dur­ing my se­nior year, I also took the SAT, and scored 720 on the math sec­tion (96th per­centile rel­a­tive to the pool of col­lege bound stu­dents). While there are many peo­ple who would be happy with this score, there were per­haps ~60 stu­dents at my high school who scored higher than me (in­clud­ing many of my class­mates who were in awe of me). Just look­ing at my math SAT score, peo­ple would think very un­likely that I would come close to be­ing the strongest calcu­lus stu­dent in my year.

As far re­moved my math­e­mat­i­cal abil­ity is from Grothendieck’s, we have at least one thing in com­mon: our re­spec­tive perfor­mances on some com­monly used mea­sures of math­e­mat­i­cal abil­ity are much lower than what most peo­ple would ex­pect based on our math­e­mat­i­cal ac­com­plish­ments.

Hope­fully these ex­am­ples suffice to make clear that what­ever math­e­mat­i­cal abil­ity is, it’s not “what the math SAT mea­sures.” What the math SAT mea­sures is highly rele­vant, but still not the most rele­vant thing.

What does the math SAT mea­sure?

Just for fun, let’s first look at what the Col­lege Board has to say on the sub­ject. Ac­cord­ing to The Offi­cial SAT Study Guide

The SAT does not test logic abil­ities or IQ. It tests your skills in read­ing, writ­ing and math­e­mat­ics – the same sub­jects you’re learn­ing in school. [...] If you take rigor­ous challeng­ing courses in high school, you’ll be ready for the test.

Some of you may be shocked by the Col­lege Board’s dis­in­gen­u­ous­ness with­out any fur­ther com­ment. How would they re­spond to my own situ­a­tion? Most hy­po­thet­i­cal re­sponses are ab­surd: They could say “Un­for­tu­nately, you were un­der­priv­ileged in hav­ing to go to the high school ranked 50th in the coun­try, where you didn’t have ac­cess to suffi­ciently rigor­ous challeng­ing courses” or “While you did take AP Calcu­lus BC, you didn’t take AP US His­tory, and that would have fur­ther de­vel­oped your math­e­mat­i­cal rea­son­ing skills” or “Our tests are re­ally badly cal­ibrated – we haven’t been able to get them to the point where some­body with 99.9 per­centile level sub­ject mat­ter knowl­edge re­li­ably scores at the 97th per­centile or higher.”

Their strongest re­sponse would be to say that the test has been re­vised since I took it in 2002 to make it more closely al­igned with the aca­demic cur­ricu­lum. This is true. But a care­ful ex­am­i­na­tion of the cur­rent ver­sion of the test makes it clear that it’s still not de­signed to test what’s learned in school. For ex­am­ple, con­sider ques­tions 16-18 in Sec­tion 2 of the sam­ple test:

SATItem

The grid above rep­re­sents equally spaced streets in a town that has no one-way streets. F marks the cor­ner where a fire­house is lo­cated. Points W, X, Y, and Z rep­re­sent the lo­ca­tions of some other build­ings. The fire com­pany defines a build­ing’s m-dis­tance as the min­i­mum num­ber of blocks that a fire truck must travel from the fire­house to reach the build­ing. For ex­am­ple, the build­ing at X is an m-dis­tance of 2, and the build­ing at Y is an m-dis­tance of 12 from the fire­house
  1. What is the m-dis­tance of the build­ing at W from the fire­house?

  2. What is the to­tal num­ber of differ­ent routes that a fire truck can travel the m-dis­tance from F to Z ?

  3. All of the build­ings in the town that are an m-dis­tance of 3 from the fire­house must lie on a...

I don’t think that rigor­ous, aca­demic challeng­ing courses build skills that en­able high school stu­dents to solve these ques­tions. They have some con­nec­tion with what peo­ple learn in school – in par­tic­u­lar, they in­volve num­bers and dis­tances. But the con­nec­tion is very ten­u­ous – they’re ex­tremely far re­moved from be­ing the best test of what stu­dents learn in school. They can be solved by a very smart 5th grader who hasn’t stud­ied alge­bra or ge­om­e­try.

The SAT Sub­ject Tests are much more closely con­nected with what stu­dents (are sup­posed to) learn in school. And they’re not merely tests of what stu­dents have mem­o­rized: some of the ques­tions re­quire deep con­cep­tual un­der­stand­ing and abil­ity to ap­ply the ma­te­rial in novel con­cepts. If the Col­lege Board wanted to make the SAT math sec­tion a test of what stu­dents are sup­posed to learn in school, they would do bet­ter to just swap it it with the Math­e­mat­ics Level 1 SAT Sub­ject Test.

If the SAT math sec­tion mea­sures some­thing other than the math skills that stu­dents are sup­posed to learn in school, what does it mea­sure? The situ­a­tion is ex­actly what the Col­lege Board ex­plic­itly dis­claims it to be: the SAT is an IQ test. This ac­counts for the in­clu­sion of ques­tions like the ones above, that a very smart 5th grader with no knowl­edge of alge­bra or ge­om­e­try could an­swer eas­ily, and that the av­er­age high school stu­dent who has taken alge­bra and ge­om­e­try might strug­gle with.

The SAT was origi­nally de­signed as a test of ap­ti­tude: not knowl­edge or learned skills. Though I haven’t seen an au­thor­i­ta­tive source, the con­sen­sus seems to be that the origi­nal pur­pose of the test was to help smart stu­dents from un­der­priv­ileged back­grounds have a chance to at­tend a high qual­ity col­lege – stu­dents who might not have had ac­cess to the ed­u­ca­tional re­sources to do well on tests of what stu­dents are sup­posed to learn in school. Frey and Det­ter­man found that as of 1979, the cor­re­la­tions be­tween SAT scores and IQ test scores were very high (0.7 to 0.85). The cor­re­la­tions have prob­a­bly dropped since then, as there have in fact been changes to make the SAT less like an IQ test, but to the ex­tent that the SAT differs from the SAT sub­ject tests, the differ­ence cor­re­sponds to the SAT be­ing more of a test of IQ.

The SAT may have served its in­tended pur­pose at the time, but since then there’s been mount­ing ev­i­dence that the SAT has be­come a harm­ful force in so­ciety. By 2007, things had reached a point that Charles Mur­ray wrote an ar­ti­cle ad­vo­cat­ing that the SAT be abol­ished in fa­vor of us­ing SAT sub­ject tests ex­clu­sively. This will have sig­nifi­cance to those of you who know Charles Mur­ray as the widely hated au­thor The Bell Curve, which em­pha­sizes the im­por­tance of IQ.

Twice ex­cep­tional gifted children

Let’s re­turn to the ques­tion of rec­on­cil­ing my very strong calcu­lus perfor­mance with my rel­a­tively low math SAT score. The differ­ence comes in sub­stan­tial part from my hav­ing a much greater love of learn­ing than is typ­i­cal of peo­ple of similar in­tel­li­gence. I think that the same was true of Grothendieck.

I could have re­sponded to Qiaochu’s sug­ges­tion that I had always had very high in­tel­li­gence and that that’s why I was able to learn math well by say­ing “No, you’re wrong, my SAT score shows that I don’t have very high in­tel­li­gence, the rea­son that I was able to learn math well is that I re­ally love the sub­ject.” But that would over­sim­plify things. In par­tic­u­lar, it leaves two ques­tions open:

  • A large part of why I failed ge­om­e­try my fresh­man year of high school is that I wasn’t in­ter­ested in the sub­ject at the time. I only got in­ter­ested in math af­ter get­ting in­ter­ested in chem­istry my sopho­more year. But al­most no­body at my high school was in­ter­ested in ge­om­e­try, and al­most ev­ery­body passed ge­om­e­try. What made me differ­ent?

  • Can a love of learn­ing re­ally boost one’s per­centile from 1 in 30 to 1 in 1000? The gap seems awfully large to be ac­counted for ex­clu­sively by love of learn­ing. And what of Grothendieck, for whom the gap may have been far larger?

Par­tial an­swers to these ques­tions come from the liter­a­ture on so-called “Twice Ex­cep­tional” (2e) chil­dren. The la­bel is used broadly, to re­fer to chil­dren who are in­tel­lec­tu­ally gifted and also have some sort of dis­abil­ity.

The cen­tral find­ing of the IQ liter­a­ture is that peo­ple who are good at one cog­ni­tive task tend to be good at any an­other cog­ni­tive task. For ex­am­ple, peo­ple who have bet­ter re­ac­tion time tend to also be bet­ter at ar­ith­metic, bet­ter at solv­ing logic puz­zles, bet­ter able to give co­her­ent ex­pla­na­tions of real world con­cepts, and bet­ter able to re­call a string of num­bers that are read to them. When I was a small child, my teach­ers no­ticed that I was an ex­cep­tion to the rule: I had a very easy time learn­ing some things and also found it very difficult to learn oth­ers. They referred me to a school psy­chol­o­gist, who found that I had ex­cep­tion­ally high rea­son­ing abil­ities, but only av­er­age short term mem­ory and pro­cess­ing speed: a 3 stan­dard de­vi­a­tion differ­ence.

There’s a sense in which my situ­a­tion is ac­tu­ally not so un­usual. The find­ing that peo­ple who are good at one cog­ni­tive task tend to be good at an­other is based on the study of peo­ple of av­er­age in­tel­li­gence. It be­comes less and less true as you look at peo­ple of pro­gres­sively higher in­tel­li­gence. Twice ex­cep­tional chil­dren are not very rare amongst in­tel­lec­tu­ally gifted chil­dren. Linda Silver­man writes

Gifted chil­dren may have hid­den learn­ing dis­abil­ities. Ap­prox­i­mately one-sixth of the gifted chil­dren who come to the Cen­ter for test­ing have some type of learn­ing dis­abil­ity—of­ten un­de­tected be­fore the as­sess­ment—such as cen­tral au­di­tory pro­cess­ing di­s­or­der (CAPD), difficul­ties with vi­sual pro­cess­ing, sen­sory pro­cess­ing di­s­or­der, spa­tial di­s­ori­en­ta­tion, dyslexia, and at­ten­tion defic­its. Gift­ed­ness masks dis­abil­ities and dis­abil­ities de­press IQ scores. Higher ab­stract rea­son­ing en­ables chil­dren to com­pen­sate to some ex­tent for these weak­nesses, mak­ing them harder to de­tect.

This starts to ex­plain why I failed ge­om­e­try dur­ing my fresh­man year of high school. The ma­te­rial was bor­ing and I wasn’t very fo­cused on grades. But I also gen­uinely found it difficult to an ex­tent that my class­mates didn’t. Learn­ing the ma­te­rial the way in which the course was taught re­quired a lot of mem­o­riza­tion – some­thing that I was markedly worse at than my class­mates at Low­ell, who had been se­lected for hav­ing high stan­dard­ized test scores.

It also ex­plains why I didn’t score higher than 720 on the math sec­tion of the SAT. It wasn’t be­cause I couldn’t an­swer ques­tions like the ones that I pasted above. It was be­cause some of the math SAT ques­tions are en­g­ineered to trip up stu­dents who for­get ex­actly what a prob­lem asked for, or who are prone to ar­ith­metic er­rors. Often a mul­ti­ple choice ques­tion will have one wrong an­swer for ev­ery such mis­take that a stu­dent might make. I used to think that this was a de­sign flaw, and that the test mak­ers didn’t know that they were pe­nal­iz­ing minor mis­takes very heav­ily. No – it wasn’t a de­sign flaw – they de­signed the test that way on pur­pose. The ques­tions test short-term mem­ory as a proxy to IQ. I tried to avoid mis­takes by be­ing re­ally sys­tem­atic about my work, and not take short­cuts. But it wasn’t enough given the time con­straints – mak­ing 3 minor mis­takes on any com­bi­na­tion of 54 ques­tions is enough to re­duce one’s score from 800 to 720.

It’s plau­si­ble that some­thing similar was true of Grothendieck.

It’s prob­a­bly in­tu­itively clear even to read­ers who are not math­e­mat­i­ci­ans that math is not about be­ing able to avoid mak­ing 3 minor mis­takes on 54 ques­tions. It’s very helpful to be quick and ac­cu­rate, and my math­e­mat­i­cal abil­ity is far lower than it would have been if my speed and ac­cu­racy were sub­stan­tially greater, but speed and ac­cu­racy are not the essence of math­e­mat­i­cal abil­ity.

What is the essence of math­e­mat­i­cal abil­ity?

I’ve only just scratched the sur­face of the sub­ject of math­e­mat­i­cal abil­ity in this post, largely fo­cus­ing on de­scribing what math­e­mat­i­cal abil­ity isn’t rather than what math­e­mat­i­cal abil­ity is. In sub­se­quent posts I’ll de­scribe math­e­mat­i­cal abil­ity in more de­tail, which will en­tail a dis­cus­sion of what math is. I’ll also ad­dress the ques­tion of how one can im­prove one’s math­e­mat­i­cal abil­ity.

In­tel­li­gence is highly rele­vant and largely ge­netic, but there are other fac­tors that are col­lec­tively roughly as im­por­tant, some of which are things that in­di­vi­d­u­als are in fact ca­pa­ble of de­vel­op­ing. For now, I’ll offer a teaser, which will be ob­scure to read­ers who lack sub­stan­tial ad­di­tional con­text, and which paints a very in­com­plete pic­ture even when un­der­stood deeply, but which should nev­er­the­less serve as food for thought. Grothendieck wrote:

In our ac­qui­si­tion of knowl­edge of the Uni­verse ( whether math­e­mat­i­cal or oth­er­wise) that which ren­o­vates the quest is noth­ing more nor less than com­plete in­no­cence. It is in this state of com­plete in­no­cence that we re­ceive ev­ery­thing from the mo­ment of our birth. Although so of­ten the ob­ject of our con­tempt and of our pri­vate fears, it is always in us. It alone can unite hu­mil­ity with bold­ness so as to al­low us to pen­e­trate to the heart of things, or al­low things to en­ter us and taken pos­ses­sion of us.

This unique power is in no way a priv­ilege given to “ex­cep­tional tal­ents”—per­sons of in­cred­ible brain power ( for ex­am­ple), who are bet­ter able to ma­nipu­late, with dex­ter­ity and ease, an enor­mous mass of data, ideas and spe­cial­ized skills. Such gifts are un­de­ni­ably valuable, and cer­tainly wor­thy of envy from those who ( like my­self) were not so en­dowed at birth,” far be­yond the or­di­nary”.

Yet it is not these gifts, nor the most de­ter­mined am­bi­tion com­bined with ir­re­sistible will-power, that en­ables one to sur­mount the “in­visi­ble yet formidable bound­aries ” that en­cir­cle our uni­verse. Only in­no­cence can sur­mount them, which mere knowl­edge doesn’t even take into ac­count, in those mo­ments when we find our­selves able to listen to things, to­tally and in­tensely ab­sorbed in child play.

Read­ers are wel­come to spec­u­late on what Grothendieck had in mind in writ­ing this.

Cross-posted from my web­site.