Poll results: LW probably doesn’t cause akrasia

Test of: De­ci­sion Fa­tigue, Ra­tion­al­ity, and Akra­sia.

Shortly be­fore the Sum­mit, Alexan­dros posted a short dis­cus­sion post won­der­ing whether ra­tio­nal­ity train­ing might cause akra­sia by prompt­ing folks to make more de­ci­sions us­ing de­liber­ate, con­scious, “sys­tem II” rea­son­ing (in­stead of rapid, au­to­matic, “sys­tem I” heuris­tics) and, thereby, caus­ing de­ci­sion fa­tigue.

This con­jec­ture sounded in­ter­est­ing to me, and I’d won­dered similar things my­self, so I put up a poll to gather data.

Procedure

I put this poll on up LW, ask­ing a num­ber of ques­tions that I hoped bore on: (1) akra­sia lev­els; (2) how the per­son’s akra­sia lev­els had shifted since they came to LW; and (3) how many de­ci­sions they made via de­liber­ate “sys­tem II” pro­cess­ing. 70 LW-ers com­pleted the sur­vey in time to get in­cluded in my data anal­y­sis; per­haps be­cause the sur­vey was on Dis­cus­sion, these were mostly folks who’d been on LW for a while; me­dian re­sponse to “months since you started read­ing LW/​OB” was 19.

I also wanted a con­trol group so as to dis­t­in­guish real LW anoma­lies from ran­dom bugs in most hu­mans’ self-re­port­ing ar­chi­tec­ture. I tried to ask Red­dit, but got only 7 re­sponses; then I tried Me­chan­i­cal Turk and re­ceived my full 100 de­sired re­sponses… but it is hard to be sure that Me­chan­i­cal Turk data is from real hu­mans.

Val­idity of “akra­sia/​pro­cras­ti­na­tion” self-reports

It would be nice if self-re­ported akra­sia lev­els cor­re­lated with (lack of) suc­cess with com­mon goals, such as in­come, ex­er­cise, and liv­ing in a non-filthy house. To as­sess par­ti­ci­pants’ akra­sia, I asked the fol­low­ing ques­tions:

  • 2. As a kid, how much trou­ble did you have with pro­cras­ti­na­tion?[1]

  • 3. How much trou­ble do you have now with pro­cras­ti­na­tion?

  • 4. Have you had any bills go to a col­lec­tion agency in the last six months?

  • 5. When was your kitchen floor most re­cently cleaned?

  • 6. How many times have you ex­er­cised in the last 7 days?

  • 11. What was your col­lege GPA?

  • 12. What was your high school GPA?

  • 18. What’s your pre­sent in­come, in dol­lars per year?

I re­placed all re­sponses with z-score es­ti­mates, re­placed in­come with “in­come con­trol­led for age”, and started look­ing at cor­re­la­tions. Of these items, #3 and #4 failed to cor­re­late with the other “akra­sia” ques­tions[2], so I dis­carded them and noted that “akra­sia” might be less of a di­men­sion than I was hop­ing. The oth­ers all cor­re­lated in the ex­pected di­rec­tions, al­though weakly, as shown:

(This table con­tains all cor­re­la­tions be­tween the listed vari­ables that oc­curred with p-value < .25, to­gether with the as­so­ci­ated p-value; note that the dataset was fairly small, so the ab­sence of a statis­ti­cally sig­nifi­cant cor­re­la­tion does not nec­es­sar­ily im­ply the ab­sence of a cor­re­la­tion.)

LW (thinks it is?) more akratic than average

On av­er­age, LW-er sur­vey par­ti­ci­pants re­garded them­selves as hav­ing more trou­ble than usual with pro­cras­ti­na­tion:

Me­chan­i­cal Turk-ers re­garded them­selves as more av­er­age (and, es­pe­cially, re­garded their child­hoods as more av­er­age), sug­gest­ing that this isn’t just a “ev­ery­one thinks they have the most trou­ble” effect (though such effects do ex­ist)[3]:

My guess is that LWers’ per­cep­tions of hav­ing more trou­ble than av­er­age with pro­cras­ti­na­tion rep­re­sents a real differ­ence in folks’ get­ting-things-done pow­ers, and not just a differ­ence in self-image. One piece of data sup­port­ing this is that LW-ers re­port higher than baseline rates of autism/​as­perger’s: 5% of sur­vey par­ti­ci­pants, com­pared about one per­cent of the gen­eral pop­u­la­tion. 20% of LW-ers re­port hav­ing ever had a de­pres­sion di­ag­no­sis, which also seems to be some­what above baseline rates. (De­pres­sion and autism both cor­re­late with difficulty get­ting things done.)

Im­prove­ment over time?

LW-ers re­port im­prov­ing over time. More­over, they re­port im­prov­ing more *since find­ing LW* than in an equal-sized chunk of time be­fore find­ing LW; the differ­ence in re­ported im­prove­ment over the two in­ter­vals is fairly small, but is statis­ti­cally sig­nifi­cant at the p=.001 sig­nifi­cance level.

Me­chan­i­cal Turk-ers do not re­port im­prov­ing, and are not rosier about their last two years than about the two years be­fore that -- sug­gest­ing that the above isn’t just due to a bug in the hu­man self-as­sess­ment sys­tem:

How­ever, re­ported LW-er akra­sia lev­els do not de­crease with re­spon­dent age, which pulls against the the­sis that LWers start out akratic but gener­i­cally im­prove as we get older. They also do not de­crease with “months since dis­cov­er­ing LW/​OB”, which pulls against the the­sis that LW helps.

De­liber­ate de­ci­sion mak­ing not harmful

To test the con­jec­ture that ex­ces­sive con­scious de­ci­sion-mak­ing (over-re­li­ance on de­liber­ate, con­scious, “Sys­tem II” rea­son­ing in­stead of on au­to­matic heuris­tics) causes akra­sia, I asked:

  • 7. When you go gro­cery shop­ping, how of­ten do you think care­fully about which product to buy, vs. just grab­bing some­thing and putting it in your cart?

  • 8. When you sit down to do work, how of­ten do you think care­fully about what sub­tasks to do and/​or how to do them, vs. just do­ing things?

  • 19. Anne is look­ing at Bob, and Bob is look­ing at Carol. Anne is mar­ried; Carol is un­mar­ried. Is a mar­ried per­son look­ing at an un­mar­ried per­son? (Yes /​ No /​ Can’t be de­ter­mined)

Ques­tion 19 is a ques­tion from the re­search liter­a­ture that is de­signed to test in­di­vi­d­u­als’ ten­dency to en­gage in “fully dis­junc­tive rea­son­ing”, and, thus, to as­sess at least one as­pect of folks’ ten­dency to use sys­tem II rea­son­ing in prefer­ence to au­to­matic sys­tem I heuris­tics. The CRT is more stan­dardly used for such mea­sure­ment, but pre­vi­ous test­ing had in­di­cated that LW-ers mostly hit the ceiling on the CRT, so I used the more difficult Anne ques­tion in­stead.

None of these ques­tions cor­re­lated pos­i­tively (to any dis­cern­ably above-chance ex­tent) with the in­di­ca­tors of akra­sia. In fact, ques­tion 8 had a sig­nifi­cant nega­tive cor­re­la­tion with cur­rent self-re­ported pro­cras­ti­na­tion lev­els (cor­re­la­tion -.26, p-value .03), sug­gest­ing that a ten­dency to de­liber­ately choose one’s work tasks may help pro­cras­ti­na­tion/​akra­sia, and does not harm it.

On the other hand, ques­tions 7, 8, and 19 also did not cor­re­late strongly with one an­other; so it is pos­si­ble that these are just not good in­di­ca­tors of folks’ de­gree of re­li­ance on de­liber­ate, con­scious, “Sys­tem II” de­ci­sion-mak­ing. Nev­er­the­less, when you com­bine these fact that none 7, 8, or 19 in­di­cated pro­cras­ti­na­tion with the re­ported im­prove­ment af­ter LWers find LW, it seems to to­gether con­sti­tute rea­son­able ev­i­dence against LW and ra­tio­nal­ity train­ing be­ing harm­ful, or at min­i­mum against them be­ing suffi­ciently harm­ful to show up with such datasets.

Other correlations

Most of the re­main­ing vari­ables cor­re­lated with one an­other in the man­ner that com­mon sense would sug­gest (e.g., be­ing in school cor­re­lated with be­ing young). Still, for com­plete­ness, here are all cor­re­la­tions among the ques­tions that ap­peared cor­re­lated with a p-value <0.03; note that since I com­pared 25 vari­ables with one an­other, we should ex­pect about (25*24/​2)*.03 = 9 cor­re­la­tions at this sig­nifi­cance level, and 0 to 1 cor­re­la­tions at the p=.001 sig­nifi­cance level just by chance. [4]

For ease of scan­ning, cor­re­la­tions that I per­son­ally found in­ter­est­ing are in bold. “In­come” is in­stead “in­come ad­justed for age and stu­dent sta­tus”; I ad­justed kitchen clean­li­ness for stu­dent sta­tus as well.

The cor­re­la­tions:

  • High re­ported lev­els of pro­cras­ti­na­tion as a kid cor­re­lated with:

    • Re­ported cur­rent pro­cras­ti­na­tion lev­els (corr. coef. = .3, p-value = .02);

    • Low high school GPA (c = .3, p=.01);

    • ADD di­ag­no­sis (c=.3, p=.02);

    • Con­sum­ing coffee/​tea/​caf­feine most days (now, not as a kid) (c=.3, p=.02)

    • Not be­ing in school right now (c=.3, p=.005).

  • High re­ported lev­els of cur­rent pro­cras­ti­na­tion cor­re­lated with:

    • Pro­cras­ti­na­tion as a kid (see above);

    • Lack of pre­sent ex­er­cise (c=.3, p=.02);

    • Work­ing on some­thing at ran­dom, in­stead of mak­ing de­liber­ate choices as to what to work on (Q8 above) (c=.26; p=.03);

    • Re­port­ing a lack of im­prove­ment, or a wors­en­ing, of pro­cras­ti­na­tion since find­ing LW (c=.4, p<.001);

    • Di­ag­noses of de­pres­sion and of “other” (c = .3, p= .01);

    • Un­hap­piness (c = .4, p=.0003);

    • Anx­iety (c =.3 , p=.01).

  • Hav­ing bills go to a col­lec­tions agency cor­re­lated with:

    • Di­ag­noses of ADD, of autism/​as­perger’s, and of de­pres­sion (c=.5, .4, and .3 re­spec­tively; as­so­ci­ated p-val­ues were p=.0001, p=.0007, and p=.02 re­spec­tively).

  • Hav­ing a kitchen floor that had/​hadn’t been cleaned in the last month cor­re­lated with: noth­ing.

  • Reg­u­lar re­cent ex­er­cise cor­re­lated with:

    • Lack of re­ported cur­rent pro­cras­ti­na­tion (see above);

    • Choos­ing items de­liber­ately in the gro­cery store (c = .3, p=.01);

    • Con­sum­ing coffee/​tea/​caf­feine most days (c = .3, p=.008).

  • Choos­ing items de­liber­ately in the gro­cery store cor­re­lated with: reg­u­lar re­cent ex­er­cise (see above).

  • Choos­ing work tasks and sub­tasks de­liber­ately (vs. just do­ing things) cor­re­lated with:

    • Re­ported lack of cur­rent pro­cras­ti­na­tion (see above);

    • Hav­ing never had a de­pres­sion di­ag­no­sis (c =.3, p=.02).

  • Re­port­ing im­proved pro­cras­ti­na­tion since find­ing LW cor­re­lated with:

    • Re­ported lack of cur­rent pro­cras­ti­na­tion (see above);

    • Lack of de­pres­sion di­ag­noses (c = .36, p=.003).

  • Re­port­ing im­proved pro­cras­ti­na­tion in an equal-sized time pe­riod be­fore find­ing LW cor­re­lated with: noth­ing.

  • High col­lege GPA cor­re­lated with:

    • High high school GPA (c = .4, p=.0006);

    • Be­ing in school (c = .3, p=.01).

  • High school GPA cor­re­lated with:

    • Lack of pro­cras­ti­na­tion as a kid; high col­lege GPA (see above)

    • Lack of “other” di­ag­noses (c = .3, p=.01);

    • Get­ting a lot of sleep (now, not as a kid) (c = .36, p=.002 ).

  • IQ cor­re­lated with: noth­ing.

  • Hav­ing “other” di­ag­noses cor­re­lated with:

    • Re­ported cur­rent pro­cras­ti­na­tion; low high school GPA (see above);

    • Di­ag­noses of de­pres­sion (c = .5, p<.0001);

    • Lack of sleep (c = .4, p=.0004);

    • Re­ported anx­iety(c = .4, p=.0002).

  • ADD di­ag­noses cor­re­lated with:

    • Re­ported child­hood pro­cras­ti­na­tion; bills sent to col­lec­tion agen­cies (see above);

    • Di­ag­noses of autism/​as­perger’s and de­pres­sion (c = .55 and .44 re­spec­tively, p=.0001 for both).

  • Autism/​Asperger’s di­ag­noses cor­re­lated with:

    • Bills sent to col­lec­tion agen­cies; di­ag­noses of ADD (see above);

    • Di­ag­noses of de­pres­sion (c = .36, p=.003).

  • De­pres­sion di­ag­noses cor­re­lated with:

    • Re­ported cur­rent pro­cras­ti­na­tion; bills sent to col­lec­tion agen­cies; ten­dency to just do work with­out think­ing about which tasks or sub­tasks to do; lack of re­ported pro­cras­ti­na­tion im­prove­ment since find­ing LW; and di­ag­noses of ADD, autism/​as­perger’s, and “other” (see above);

    • Re­ported un­hap­piness (c = .4, p=.0002);

    • Re­ported anx­iety (c = .3, p=.01).

  • In­come cor­re­lated with: noth­ing. [After con­trol­ling for age, stu­dent sta­tus].

  • An­swer­ing the Anne ques­tion cor­rectly (Q19 above) cor­re­lated with:

    • Con­sum­ing coffee/​tea/​caf­feine most days (c=.34, p=.004).

  • Time since the re­spon­dent started read­ing LW cor­re­lated with: noth­ing.

  • Hours of sleep per night cor­re­lated with:

    • High col­lege GPA; and not hav­ing “other” di­ag­noses (see above).

  • Age cor­re­lated with: not be­ing a stu­dent.

  • Con­sum­ing coffee/​tea/​caf­feine most days cor­re­lated with:

    • Pro­cras­ti­na­tion in child­hood; suc­cess­fully get­ting ex­er­cise; an­swer­ing the Anne ques­tion cor­rectly (see above).

  • Re­ported hap­piness cor­re­lated with:

    • Re­ported lack of cur­rent pro­cras­ti­na­tion; and lack of de­pres­sion di­ag­noses (see above);

    • Be­ing a stu­dent (c=.3, p=.02).

  • Re­ported anx­iety cor­re­lated with:

    • Re­ported cur­rent pro­cras­ti­na­tion; di­ag­noses of “de­pres­sion” and of “other” (see above).

  • Be­ing a stu­dent cor­re­lated with:

    • Lack of re­ported pro­cras­ti­na­tion as a child; high col­lege GPA; youth; hap­piness (see above).

Raw data

In case you want to play with the raw data your­self, here it is:

Given the im­por­tance of tak­ing ac­tions that ac­tu­ally re­late to one’s goals (for hap­piness, in­come, world-sav­ing, you name it—and, hence, for real ra­tio­nal­ity), fur­ther in­ves­ti­ga­tion here would be wel­come, ei­ther via fur­ther polls on LW-ers or oth­ers, or, per­haps even more eas­ily and use­fully, via Google Scholar.


[1] Origi­nally, I asked about “trou­ble with akra­sia” (now and as a kid) rather than about “trou­ble with pro­cras­ti­na­tion”. I ed­ited the ques­tion af­ter re­al­iz­ing I’d want a con­trol group with non-LWers, and that that group would not re­li­ably know the term “akra­sia”. So, the LW-er re­sponses are partly to one word­ing and partly to the other.

[2] Both items cor­re­lated strongly with stu­dent sta­tus; when I con­trol­led for stu­dent sta­tus (sub­tracted out the con­stant nec­es­sary to re­move the cor­re­la­tion), “have had late bills re­ported to credit agen­cies” cor­re­lated strongly with di­ag­noses of ADD, autism/​Asperger’s, and de­pres­sion, but with noth­ing else; dirty kitchen floors cor­re­lated with noth­ing.

[3] This is fur­ther sug­gested by the fact that Me­chan­i­cal Turk-ers may well *have* more akra­sia than av­er­age at pre­sent; they are work­ing on Me­chan­i­cal Turk, and have fairly high num­bers of de­pres­sion di­ag­noses.

[4] There are 34 cor­re­la­tions at the p<.03 sig­nifi­cance level, and 12 at the p<.001 sig­nifi­cance level, which is more than we should ex­pect by chance; this is not sur­pris­ing, since of course e.g. be­ing in school is cor­re­lated with be­ing young, and so of course we see some non-chance cor­re­la­tions; the ques­tion is how many of the non-ob­vi­ous cor­re­la­tions are just chance.