Poll results: LW probably doesn’t cause akrasia
Test of: Decision Fatigue, Rationality, and Akrasia.
Shortly before the Summit, Alexandros posted a short discussion post wondering whether rationality training might cause akrasia by prompting folks to make more decisions using deliberate, conscious, “system II” reasoning (instead of rapid, automatic, “system I” heuristics) and, thereby, causing decision fatigue.
This conjecture sounded interesting to me, and I’d wondered similar things myself, so I put up a poll to gather data.
Procedure
I put this poll on up LW, asking a number of questions that I hoped bore on: (1) akrasia levels; (2) how the person’s akrasia levels had shifted since they came to LW; and (3) how many decisions they made via deliberate “system II” processing. 70 LW-ers completed the survey in time to get included in my data analysis; perhaps because the survey was on Discussion, these were mostly folks who’d been on LW for a while; median response to “months since you started reading LW/OB” was 19.
I also wanted a control group so as to distinguish real LW anomalies from random bugs in most humans’ self-reporting architecture. I tried to ask Reddit, but got only 7 responses; then I tried Mechanical Turk and received my full 100 desired responses… but it is hard to be sure that Mechanical Turk data is from real humans.
Validity of “akrasia/procrastination” self-reports
It would be nice if self-reported akrasia levels correlated with (lack of) success with common goals, such as income, exercise, and living in a non-filthy house. To assess participants’ akrasia, I asked the following questions:
2. As a kid, how much trouble did you have with procrastination?[1]
3. How much trouble do you have now with procrastination?
4. Have you had any bills go to a collection agency in the last six months?
5. When was your kitchen floor most recently cleaned?
6. How many times have you exercised in the last 7 days?
11. What was your college GPA?
12. What was your high school GPA?
18. What’s your present income, in dollars per year?
I replaced all responses with z-score estimates, replaced income with “income controlled for age”, and started looking at correlations. Of these items, #3 and #4 failed to correlate with the other “akrasia” questions[2], so I discarded them and noted that “akrasia” might be less of a dimension than I was hoping. The others all correlated in the expected directions, although weakly, as shown:
(This table contains all correlations between the listed variables that occurred with p-value < .25, together with the associated p-value; note that the dataset was fairly small, so the absence of a statistically significant correlation does not necessarily imply the absence of a correlation.)
LW (thinks it is?) more akratic than average
On average, LW-er survey participants regarded themselves as having more trouble than usual with procrastination:
Mechanical Turk-ers regarded themselves as more average (and, especially, regarded their childhoods as more average), suggesting that this isn’t just a “everyone thinks they have the most trouble” effect (though such effects do exist)[3]:
My guess is that LWers’ perceptions of having more trouble than average with procrastination represents a real difference in folks’ getting-things-done powers, and not just a difference in self-image. One piece of data supporting this is that LW-ers report higher than baseline rates of autism/asperger’s: 5% of survey participants, compared about one percent of the general population. 20% of LW-ers report having ever had a depression diagnosis, which also seems to be somewhat above baseline rates. (Depression and autism both correlate with difficulty getting things done.)
Improvement over time?
LW-ers report improving over time. Moreover, they report improving more *since finding LW* than in an equal-sized chunk of time before finding LW; the difference in reported improvement over the two intervals is fairly small, but is statistically significant at the p=.001 significance level.
Mechanical Turk-ers do not report improving, and are not rosier about their last two years than about the two years before that -- suggesting that the above isn’t just due to a bug in the human self-assessment system:
However, reported LW-er akrasia levels do not decrease with respondent age, which pulls against the thesis that LWers start out akratic but generically improve as we get older. They also do not decrease with “months since discovering LW/OB”, which pulls against the thesis that LW helps.
Deliberate decision making not harmful
To test the conjecture that excessive conscious decision-making (over-reliance on deliberate, conscious, “System II” reasoning instead of on automatic heuristics) causes akrasia, I asked:
7. When you go grocery shopping, how often do you think carefully about which product to buy, vs. just grabbing something and putting it in your cart?
8. When you sit down to do work, how often do you think carefully about what subtasks to do and/or how to do them, vs. just doing things?
19. Anne is looking at Bob, and Bob is looking at Carol. Anne is married; Carol is unmarried. Is a married person looking at an unmarried person? (Yes / No / Can’t be determined)
Question 19 is a question from the research literature that is designed to test individuals’ tendency to engage in “fully disjunctive reasoning”, and, thus, to assess at least one aspect of folks’ tendency to use system II reasoning in preference to automatic system I heuristics. The CRT is more standardly used for such measurement, but previous testing had indicated that LW-ers mostly hit the ceiling on the CRT, so I used the more difficult Anne question instead.
None of these questions correlated positively (to any discernably above-chance extent) with the indicators of akrasia. In fact, question 8 had a significant negative correlation with current self-reported procrastination levels (correlation -.26, p-value .03), suggesting that a tendency to deliberately choose one’s work tasks may help procrastination/akrasia, and does not harm it.
On the other hand, questions 7, 8, and 19 also did not correlate strongly with one another; so it is possible that these are just not good indicators of folks’ degree of reliance on deliberate, conscious, “System II” decision-making. Nevertheless, when you combine these fact that none 7, 8, or 19 indicated procrastination with the reported improvement after LWers find LW, it seems to together constitute reasonable evidence against LW and rationality training being harmful, or at minimum against them being sufficiently harmful to show up with such datasets.
Other correlations
Most of the remaining variables correlated with one another in the manner that common sense would suggest (e.g., being in school correlated with being young). Still, for completeness, here are all correlations among the questions that appeared correlated with a p-value <0.03; note that since I compared 25 variables with one another, we should expect about (25*24/2)*.03 = 9 correlations at this significance level, and 0 to 1 correlations at the p=.001 significance level just by chance. [4]
For ease of scanning, correlations that I personally found interesting are in bold. “Income” is instead “income adjusted for age and student status”; I adjusted kitchen cleanliness for student status as well.
The correlations:
High reported levels of procrastination as a kid correlated with:
Reported current procrastination levels (corr. coef. = .3, p-value = .02);
Low high school GPA (c = .3, p=.01);
ADD diagnosis (c=.3, p=.02);
Consuming coffee/tea/caffeine most days (now, not as a kid) (c=.3, p=.02)
Not being in school right now (c=.3, p=.005).
High reported levels of current procrastination correlated with:
Procrastination as a kid (see above);
Lack of present exercise (c=.3, p=.02);
Working on something at random, instead of making deliberate choices as to what to work on (Q8 above) (c=.26; p=.03);
Reporting a lack of improvement, or a worsening, of procrastination since finding LW (c=.4, p<.001);
Diagnoses of depression and of “other” (c = .3, p= .01);
Unhappiness (c = .4, p=.0003);
Anxiety (c =.3 , p=.01).
Having bills go to a collections agency correlated with:
Diagnoses of ADD, of autism/asperger’s, and of depression (c=.5, .4, and .3 respectively; associated p-values were p=.0001, p=.0007, and p=.02 respectively).
Having a kitchen floor that had/hadn’t been cleaned in the last month correlated with: nothing.
Regular recent exercise correlated with:
Lack of reported current procrastination (see above);
Choosing items deliberately in the grocery store (c = .3, p=.01);
Consuming coffee/tea/caffeine most days (c = .3, p=.008).
Choosing items deliberately in the grocery store correlated with: regular recent exercise (see above).
Choosing work tasks and subtasks deliberately (vs. just doing things) correlated with:
Reported lack of current procrastination (see above);
Having never had a depression diagnosis (c =.3, p=.02).
Reporting improved procrastination since finding LW correlated with:
Reported lack of current procrastination (see above);
Lack of depression diagnoses (c = .36, p=.003).
Reporting improved procrastination in an equal-sized time period before finding LW correlated with: nothing.
High college GPA correlated with:
High high school GPA (c = .4, p=.0006);
Being in school (c = .3, p=.01).
High school GPA correlated with:
Lack of procrastination as a kid; high college GPA (see above)
Lack of “other” diagnoses (c = .3, p=.01);
Getting a lot of sleep (now, not as a kid) (c = .36, p=.002 ).
IQ correlated with: nothing.
Having “other” diagnoses correlated with:
Reported current procrastination; low high school GPA (see above);
Diagnoses of depression (c = .5, p<.0001);
Lack of sleep (c = .4, p=.0004);
Reported anxiety(c = .4, p=.0002).
ADD diagnoses correlated with:
Reported childhood procrastination; bills sent to collection agencies (see above);
Diagnoses of autism/asperger’s and depression (c = .55 and .44 respectively, p=.0001 for both).
Autism/Asperger’s diagnoses correlated with:
Bills sent to collection agencies; diagnoses of ADD (see above);
Diagnoses of depression (c = .36, p=.003).
Depression diagnoses correlated with:
Reported current procrastination; bills sent to collection agencies; tendency to just do work without thinking about which tasks or subtasks to do; lack of reported procrastination improvement since finding LW; and diagnoses of ADD, autism/asperger’s, and “other” (see above);
Reported unhappiness (c = .4, p=.0002);
Reported anxiety (c = .3, p=.01).
Income correlated with: nothing. [After controlling for age, student status].
Answering the Anne question correctly (Q19 above) correlated with:
Consuming coffee/tea/caffeine most days (c=.34, p=.004).
Time since the respondent started reading LW correlated with: nothing.
Hours of sleep per night correlated with:
High college GPA; and not having “other” diagnoses (see above).
Age correlated with: not being a student.
Consuming coffee/tea/caffeine most days correlated with:
Procrastination in childhood; successfully getting exercise; answering the Anne question correctly (see above).
Reported happiness correlated with:
Reported lack of current procrastination; and lack of depression diagnoses (see above);
Being a student (c=.3, p=.02).
Reported anxiety correlated with:
Reported current procrastination; diagnoses of “depression” and of “other” (see above).
Being a student correlated with:
Lack of reported procrastination as a child; high college GPA; youth; happiness (see above).
Raw data
In case you want to play with the raw data yourself, here it is:
The poll questions: for the LW poll; for the mechanical Turk poll.
The raw responses: for the LW poll; for the mechanical Turk poll.
The raw responses converted into numbers, in case you want to run correlations but don’t want to mess with text: for the LW poll (note that this includes only the first 68 responses, which are all I analyzed above); for the mechanical Turk poll.
Given the importance of taking actions that actually relate to one’s goals (for happiness, income, world-saving, you name it—and, hence, for real rationality), further investigation here would be welcome, either via further polls on LW-ers or others, or, perhaps even more easily and usefully, via Google Scholar.
[1] Originally, I asked about “trouble with akrasia” (now and as a kid) rather than about “trouble with procrastination”. I edited the question after realizing I’d want a control group with non-LWers, and that that group would not reliably know the term “akrasia”. So, the LW-er responses are partly to one wording and partly to the other.
[2] Both items correlated strongly with student status; when I controlled for student status (subtracted out the constant necessary to remove the correlation), “have had late bills reported to credit agencies” correlated strongly with diagnoses of ADD, autism/Asperger’s, and depression, but with nothing else; dirty kitchen floors correlated with nothing.
[3] This is further suggested by the fact that Mechanical Turk-ers may well *have* more akrasia than average at present; they are working on Mechanical Turk, and have fairly high numbers of depression diagnoses.
[4] There are 34 correlations at the p<.03 significance level, and 12 at the p<.001 significance level, which is more than we should expect by chance; this is not surprising, since of course e.g. being in school is correlated with being young, and so of course we see some non-chance correlations; the question is how many of the non-obvious correlations are just chance.
- The curse of identity by (17 Nov 2011 19:28 UTC; 203 points)
- Some potential dangers of rationality training by (21 Jan 2012 4:50 UTC; 28 points)
- 's comment on LessWrong opinion of Nietzsche? by (26 Nov 2011 6:11 UTC; 10 points)
- 's comment on Rationality Verification Opportunity? by (16 Dec 2011 0:39 UTC; 9 points)
- 's comment on Decision Fatigue, Rationality, and Akrasia. by (19 Sep 2011 22:13 UTC; 7 points)
- 's comment on Interesting critique of British education by outgoing advisor (warning: some politics) by (14 Oct 2013 7:12 UTC; 5 points)
- 's comment on Decision Fatigue, Rationality, and Akrasia. by (16 Nov 2011 5:48 UTC; 5 points)
- 's comment on 2012 Less Wrong Census/Survey by (4 Nov 2012 21:50 UTC; 1 point)
- 's comment on Irrationality Game II by (6 Jul 2012 5:11 UTC; 0 points)
- 's comment on Handling Emotional Appeals by (11 Dec 2011 0:44 UTC; -1 points)
Tribal signalling? Akrasia is something we care about, and rationalists are supposed to win.
Could also be that reading articles about akrasia makes you feel like you’re more productive even if you’re really not.
I would have suggested, and still do suggest if another poll will happen, the question,
to see if that correlates with e.g. procrastination. Large any-given-moment tab count, I hypothesize, correlates negatively with focus, and maybe slightly negatively with conscientiousness.
I open lots of tabs, then close them. I am pretty sure I have an internet addiction.
it’s not quite trivial to actually measure, but total tabs opened in the last, say, hour is probably a better measurement than how many you have open right now.
After writing that I started thinking “maybe a large number of tabs open with a slow turnover/new tabs opening rate doesn’t even correlate at all with procrastination”, but I suspect that’s just me coming up with excuses for things and isn’t actually true. Could try measuring both if the survey actually works, shrug.
I generally have lots of tabs open (to the point of being made fun of) and my tendency is to open and close them swiftly in the course of multi-pronged subject exploration, with a small handful of “best of exploration” that I retain so that they prime me in subsequent hours or days or weeks with reminders, re-reading opportunities, and the possibility of being folded into longer term projects. Every so often I clean them up by transcribing URLs and notes into text files that accumulate in an idea-archive. I endorse some of this behavior, but suspect that it could become problematic in the long term… not because of “akrasia”, but as part of a more specific problem called “hoarding”.
Hoarding appears to be a mental disorder that can start in one’s teens, but really starts to become visible in one’s 30′s or 40′s, growing with time until you’re an 80-year-old living in a pile of useless trash. My current working model for it is that retention behavior is the default behavior, mostly driven by positive emotions triggered by objects. To throw something away, a hoarder needs to consciously override this default using fluid intelligence (calculating that expected use-value in realistic plans are less than inventory costs?). As aging progresses, fluid mental abilities decline, and you’re less able to decide that something isn’t worth keeping, until there are tiny trails between the bed, the toilette, and the microwave and the rest of the house is full of piles of boxes full of sorted boxes of crap.
Amusingly, I found out about hoarding via my tab-heavy information searches and left the tabs open for days, and cleaned them up into a TODO file to have a conversation with family about hoarding, which is part of why I’m aware of this. Within five years I plan to do some debugging of space management habits and policies to make sure they’re solid and clean, but I expect it to take 30 minutes per day of thinking, acting, measuring, and updating for around six months and it isn’t that high a priority relative to other things currently on my plate.
I find that regular house moves drastically improves hoarding behaviour. Especially international moves :) It doesn’t stop you hoarding—but gives you good opportunity to prune/discard.
A survey could also ask how many hours you spend online each day.
I never know how to respond to this question. Do I count device-hours? Only time when information is transferring via the Internet to some device I own? Only when I ask it to? Do I need to be looking at a screen? If so, then printouts of ffdn don’t count? What about programs I’ve downloaded and then are running offline? (&c.)
Except you’d have to exclude IT workers (especially web developers). :)
That’s probably not the best question to ask people.
One of the effects of my recent switch from Firefox to Google Chrome is that I came to keep much fewer (less than one third as many) tabs open. When I used the back button or the history menu in Firefox to revisit a page I had recently visited, sometimes (not always, but sometimes) the operation took a long time. I had subconsciously changed my behavior to avoid this delay by opening more new tabs. (Or maybe the change in behavior was conscious, but I forgot about it. Point is that I did not realize why I was opening a lot of new tabs until someone brought it up in conversation.) The reason opening a page in a new tab helps is that switching from the new tab back to an old tab does not incur the delay.
Although someone claims Firefox has a solution for the problem most readers are probably using a version of Firefox that still has the problem. (Versions 3 through 7 have it in my experience.) Moreover, I do not particularly believe that the problem was really fixed and that it will stay fixed.
So what should we ask instead of how many tabs the person has open? Firefox and Google Chrome both keep a record or list, which can be accessed through the History menu, of every web page visited, and this list is divided into 24-hour periods. So we might ask the person to go through the last complete 24-hour period on the list (i.e., yesterday) and count how many procrastination web pages the person visited.
People weight their identities more than they epistemically should.
You can come up with all kinds of excuses for how and why your current tab count is X, but at the end of the day, what really matters is how well the number predicts focus, conscientiousness, akrasia, etc.
Your reply to my attempt to inform makes me sad.
I have ten tabs open right now.
This is probably because of my habit to opening almost all links in a new tab (because it’s easier to get back to where I came from) and can’t be bothered to close a tab unless I really have too many tabs open.
I have ten tabs open right now… in my second browser window. And another 42 in the first one…
112 in 5 windows.
I regularly let tabs proliferate until I have a dozen windows open and hundreds of tabs between them and the browser gets so slow that I have to restart it or it just crashes. When this happens, I usually don’t feel like waiting for hundreds of tabs to reload, so I move the saved browser session aside, telling myself I’m just making a “temporary” new session, and then the same thing happens and I never revisit any of my past sessions. I have browser session files dating back to… 2007? That can’t be right, this has been going on for way longer than that… maybe I have the older ones on a backup somewhere.
Anyway, I think I have a problem.
o_O
16 in one window.
Jawdrop. 42? Wow.
As for myself, I have 8.
14 in two different windows.
26 tabs in one window.
73 tabs, 4 windows.
15 tabs
5 tabs. Three of them are related to the Akrasia poll and I will be closing them within 5 minutes. I do not like large numbers of open tabs.
I like this idea better than asking people how many hours they spend on the web because people are terrible at estimating their time usage, but you probably want to also ask people which browser they use: when I switched from Firefox to Chrome, the average number of tabs I have open dropped manyfold because Firefox has a bug that sometimes causes annoying delays when the back button or the history menu is used. The bug conditions Firefox users to open a new tab whenever he or she suspects he or she might need to refer again to the current page. In my case the conditioning was unconscious: I became aware of the bug only after years of using Firefox and years of being in the habit of opening a lot of new tabs.
Procrastination is worst for me on tasks where effort is only ambiguously related to goals. For example I know I need to study to do well on a test, but am poorly calibrated on how much I need to study.
If effort and goal completion become strongly linked I seem to have no trouble.
Upvoted. This seems like an incredibly insightful observation, which has not been addressed much in the procrastination or self-discipline literature much, if at all.
This observation certainly would explain why “writer’s block” is a thing. ;-)
More formally, such a correlation predicts that writer’s block would 1) exist, and 2) be worse for some sorts of writing than others. It would also predict difficulties achieving such outcomes such as losing weight or developing some degree of fitness, where the causal links between inputs and outcomes are tenuous or unclear.
ISTM that one way to harness this observation would be to shift one’s goals to refer to things that are fully under one’s control. This is already part of standard self-help advice, but it’s usually justified on the grounds that it’s discouraging to work as hard as you can but still not get your desired outcome.
“shift one’s goals to refer to things that are fully under one’s control.”
Almost impossible, we’re too causally entangled with uncertainties for most of our goals. The obvious method is to hold oneself blameless for the parts not under control but the brain seems to have a hard time with this, stressing yourself out wondering if there was anything else you could have done. This combined with hindsight bias makes people feel awful because they could have had their goal “if only they had done X” ignoring that there was no plausible way for them to have known that beforehand. I think this behavior is adaptive as it seems like part of our learning algorithm. In particular the feeling awful is a motivator to think about what information you should seek out in future similar situations that you didn’t this time. That this adaptation continues to execute even when there actually is no way to get the information is another cognitive cost shortcut bug.
Thanks. This little observation makes a lot of sense, and explains aspects of my behaviour that I really didn’t have a good explanation for. In particular, I’ve been trying to grow a small business for a couple of years now, and akrasia has been very difficult to fight at times. The correlation between effort and reward is virtually non-existent except over a multi-month period, and the constant second guessing can easily lead to paralysis.
Did I take the most optimal approach? Should I have done something different? I feel bad about it, but there just isn’t any way to obtain the necessary data in a timely fashion. This shouldn’t cause akrasia, but it does.
I don’t follow you. I may have little control over what I weigh or what I look like, but I certainly have more control over how many repetitions of an exercise I perform or how many calories I eat.
If you’re doing that, then you haven’t actually set a goal of the type I’m describing. The goal, “eat no more than X calories today” is not the same as “lose X tenths of a pound today”. The former goal is unlikely to create any wondering about whether you could have done something else, unless you actually fail to achieve it. ;-)
But those are instrumental sub-goals. The worrying about whether you should have done something else is worrying about whether you should have had different sub-goals in pursuit of your terminal goal.
Oh. Then don’t do that. ;-)
More usefully (I hope): what I’m saying is that you determine that you are better off having acheived that subgoal, even if it doesn’t appear to produce any progress towards the supergoal. At the very least, you have gathered some information that you didn’t previously have; at most, you will have moved some of the way towards the supergoal.
right, that’s specifically what I’m saying the human brain refuses to do. We seem incapable of not thinking about the utility we would have gained from alternate sub-goal selection.
Let me illustrate. I believe playing the lottery is of negative expected utility. But let’s just say that if I were to play the lottery that I would play the numbers 4-16-72-65-43-97
Now say I check the next lottery and those numbers happen to win a jackpot. I will be UNABLE to avoid feeling angry I did not choose to buy a lottery ticket even though I know my past self had no logical reason to buy one. To clarify I think I am much better at not feeling angry in this type of situation than most, but still predict that it would induce an emotional reaction.
Who’s “we”? This sounds like an instance of “typical mind fallacy” here.
myself and everyone I’ve discussed it with so far. I believe you if you claim to be a counter example.
I have set goals to follow a particular diet without having any particular goal for how much weight to lose, and not experienced the regret you describe. I would certainly prefer that any given day’s weight be less than the day before, and I occasionally wish I’d done better at following my plan(s), but have not experienced any sort of desire that I’d done better so that I would have lost more weight.
It’s important to clarify that I am emphatically not saying to set a fake process-goal as a means to achieving your end-product goal. Rather, I’m saying that you must actually abandon your product goal and replace it with a pure process goal, having no attachment to achieving a particular result.
So, the lottery example is really quite unrelated to this sort of thing, unless your goal were to play the lottery every day, not to win the lottery. ;-) In that sense, I could say that by making a process goal (e.g. certain amount of salad eaten each day), I am “buying tickets” in the hopes of getting a win on average, without paying much attention to individual wins and losses.
In the last year or so, I’ve lost 40+ pounds by my rather erratic method, despite having had relatively few days where I’ve done all that well at achieving my process goals. It’s pretty clear to me that I could (potentially) be going a LOT faster if I were better at following my process goals, but at the same time, it’s rather nice that I’m making progress at all. If I’d been following product goals instead (how much weight to lose), I undoubtedly would’ve spent most of my time frustrated, and likely given up on some approaches way too prematurely.
That seems like a recipe for lost purposes to me...
Attachment and desire are two separate things. I can desire to lose weight, without being attached to losing weight on any given day. As mentioned in the grandparent comment, this has actually netted me a good 40 pounds of weight loss. How are your non-”lost purposes” doing, instrumentally?
Okay, I understand what you mean now.
I’ve actually read about this connection in several self help blogs and posts. I also casually mention it in my comment earlier today.
I kind of remember getting the same idea from reading your book, maybe memory is foggy or you only hint at it.
This was perhaps the result I was most excited for. Finally, scientific validation.
Now whenever someone complains about your floor, you can point here! Who says LW is never instrumentally useful?
For someone to complain about my floor, someone besides me would have to come over.
Forever Alone.
Forever alone
It probably correlates with having a clean kitchen floor, which can be good for reasons that have nothing to do with akrasia
The most important thing i learned from less wrong is “Cause and Effect”.
Since i filled out the poll i would like to say something.
What i think the poll did not take into account is that procrastination seems to be caused by conflicting beliefs. I can completely cure myself from procrastination on one specific subject if i manage to find and correct all the conflicting beliefs. It also helps to allign them with a goal you have, this turns a once procastination inducing task to something you get OCD about doing whenever it pops up.
But thats also a downside, you have to learn the habits of tracking down the conflicting beliefs, and you are likely to procrastinate at learning these new habits (as do I). It’s also a bit depressing to realise you have so many conflicting beliefs,but that effect is outweighed by knowing you CAN fix them some day.
In my experience it can take anywhere from 1 to dozens of belief changes to fix chronic procrastination on a single task. I have noticed some bleeding out effect though, some of the other tasks i used to procrastinate on have become easier to do, but they clearly still have conflicting beliefs of their own.
My point is: Generally you need to attack each individual thing you procrastinate on one by one, so progress is VERY slow. But i would never have gotten to step one without lesswrong and particularly PJ Eby’s http://thinkingthingsdone.com/
Just a guess, but if you’re gettting “dozens”, I would ask: were you actually successful on the ones with “dozens” or are you still working on them?
If the latter, I’d guess that you’re working on the problem at too low a level.
I have a bunch that i cannot solve, maybe i have gone to low on them? in fact i’ve only fixed 3 or 4 so far, and they where all done in in a couple of beliefs.
I guess i never learned how to spot “going too low”
My biggest problem is doing school. I have been procrastinating on it for years and i can only bring my self to do brief bursts of work for a few days followed by weeks of doing nothing. I have tackled several conflicting beliefs but this one won’t budge.
P.S. I would like to know about the progress on your new books. P.P.S It would be great if you could help/coach me somehow.
This may sound a bit flippant, but most likely the problem here is that you don’t really want to do it. You want some other outcome that you think the school will get you—not a real-world outcome but a relational outcome like your parents’ approval. When the work doesn’t result in you feeling like it’s getting you any closer to your (covert subconscious) goal, it becomes discouraging and you give up.
Does that sound about right?
If so, it’s not going to go anywhere as long as you’re futzing around with beliefs that relate directly to your overt, school-related goals.
I am doing school to improve my chances on the market. I have been working since i quit school at 18(31 now). The reason i need to finish school is so i can apply for better jobs or get a raise in my current one. There is also other financial benefits involved with finishing school (i’ll get a 10.000 euro bonus when i do). But all that still does not motivate me to do it. There IS a lot of fear involved, i was a straight A student before i met the teacher from hell and soon after quit school.
EDIT: forgot to mention that the contents of the classes are for 80% or more relevant to my dayjob.
There are your two conflicting beliefs, right there. ;-)
I don’t see it?
“I am doing X because Y.” “Y does not motivate me to do X.”
That’s a flat-out contradiction. It means you’ve either mis-stated something or one of the statements is wrong.
In this case, it’s the first statement that’s inaccurate. First, you’re not “doing school to improve your chances on the market”… because, as stated, you’re not actually doing it, except for a few days out of most weeks.
So what you probably mean is, “I intend to do school to improve my chances on the market”. But this statement is still false, unless it is also true that “I intend to improve my chances on the market”. Do you, in actual fact, intend to improve your chances on the market?
I expect not. Rather, I expect that your motivation is to appear to be the sort of person who you think you would be if you were ambitiously attempting to improve your chances on the market… which is not really motivating enough to actually DO the work. However, by persistently trying to do so, and presenting yourself with enough suffering at your failure to do it, you get to feel as if you are that sort of person without having to actually do the work. This is actually a pretty optimal solution to the problem, if you think about it. (Or rather, if you DON’T think about it!)
In other words, your real conflicts here quite likely have almost nothing to do with your overtly stated goals and problems.
[Edit to add: properly expanded, your real intention statement is probably something more like, “I am having (an intention to do school to improve on the market) in order to be consistent with characteristic X of my desired ideal self-image”]
I cannot find any flaw in your statement. If it where just up to me I would just forget about school and give up. But it’s not up to me.
I need to find a way to find the motivation to actually do the school work, but nothing i can come up with seems to work..
Blockers that come to mind are: “I’ll fail the exams and have to pay extra money to do them again” “The information is so densly packed i can’t follow half of it” “I say that i can accept a 6⁄10 but i don’t really believe that(i want at least 8⁄10)” The list goes on, i think they are just excuses that hide the real feeling.
EDIT: i figured something out, I need to find a pet project or work project that the class is relevant for and then follow the class as a way to get that project done/improved. That way i have a directly related goal and no reason to procrastinate.
My guess is that will only help to the extent that it doesn’t get you any closer to actually succeeding in school (in the big picture), but admittedly that guess is based on very little information. However, my general observation that such tricks work on the small scale but not the larger scale is a very well-established pattern, so I will be quite surprised and curious should your trick actually work.
Do you know of a better strategy?
Find out what is objectionable to you about being the kind of person who would enjoyably and successfully achieve the outcome you’re intending to intend to have, and stop objecting to it.
For example, if your overt intention were to make a lot of money, but you noticed that your actual actions mysteriously kept interfering with that goal, then perhaps you apply the label “greedy” to people who actually make money, and would thus object to actually succeeding at that goal, because you yourself would become “greedy” by your own rules. The solution would be to stop disapproving of “greedy” people or change your rules for what constitutes being “greedy”.
Of course, it’s rarely that simple: you might actually avoid making money because it involves being “a suit” or “a sellout” or a “drudge” or any number of a bazillion labels you could attach to people on the basis of actions like yours.
The trick is to pay attention not only to what labels you’d apply to yourself if you achieved your ultimate goal (which itself is above the level of your overt goal), but also to what labels you’d apply if you were staying on task for more than a few days every few weeks. If you persisted, for example, would that make you a nerd or a bookworm, or be perceived as such by people in your life?
On the ultimate goal level, if you actually succeeded in improving your job marketability, would that make you be seen as an overambitious wannabe or “thinks he’s better than us” by significant people in your life (e.g. family or co-workers)?
I don’t claim that finding and fixing this is easy or even a “better” strategy, since (at least from your brain’s point of view), your continual trying and failing may actually be an optimal compromise. ;-)
(Consider that, if you succeed, you very well may lose some of your existing friends and allies.)
Ok so i thought about it hard.
Here’s what i think the problem is: I think people who finished school and have diploma’s are chumps who fell for paper-to-prove skills game in a system that is easily gamable and as a result doesn’t actually show anything about your skill.
I guess i’ve made it a point to prove that you don’t need school to get shit done (i have the biggest house, earn the most, best car, etc.. of all my friends and family, it seems i DO care about showing off?)
What i’ve been stumped on for years though, is how to respect being a graduate.
Test: imagine finishing school, and see if your spontaneous reaction is thinking you’re a chump. If not, you’re just speculating rather than actually observing your beliefs.
Stop disapproving of graduates. I.e., refrain from withholding your approval of them. Imagine a person who’s a graduate, notice your disapproval (really, the muscle tensions that go with it), and then physically begin releasing them. You may also wish to ask yourself if there is any benefit to you from continuing to disapprove, or whether anything bad will happen if you begin approving of them.
If you have difficulty just letting go of it, I suggest this book, even though it is annoyingly repetitive and simple-minded. The repetition and simple-mindedness are actually a feature, not a bug, though it may not seem that way at first. If you aren’t willing to endure a little boring repetition and simple-mindedness, though, you probably don’t want your goal that much. ;-)
Still another method: if you can state your disapproval in the form of a “should” or “shouldn’t” statement, you can rephrase to a statement of preference instead of one of judgment. e.g., “I would prefer not having to graduate” instead of “I shouldn’t have to graduate.”
Test result: I feel really happy and relieved, i finally belong in the group and people will stop nagging me about it, i can put it all behind me. I’m not a chump at all, in fact, i’m (finally) normal.
I do feel forced to finish school, just because everyone else has done so and I have to live up to the expectations of society. (I also live in a country where without a diploma they don’t even invite you for an interview).
I have trouble letting go of stuff in general (i have OCD) so i might just read that book anyway.
I shouldn’t have to finish school because others want me to, but because i want to of my own free will.
(You know i love playing RPG’s and maxing out all the skill trees and side quest badges, these are really no different from school (in my school i’ll actually get 32 different badges in addition to the diploma)
Upvoted for apparently taking more than 5 minutes.
For this type of “thinking”, it’s not really duration that matters. 5 seconds of noticing your spontaneous emotional reaction to an imagined stimulus is much more likely to be useful than 5 minutes of “thinking” that consists of making up stories that seem to explain things.
I’ve had luck with “Once I get this done, it’ll be done, and that’ll feel awesome.”
This is a little surprising to me, since Conscientiousness is supposed to improve somewhat with age. Obvious explanation: a selection effect inasmuch as older people with greater skills and Conscientiousness will be busy with non-LW things.
I have no idea whether this applies to the poll respondents, but adult lives tend to be a lot more exhausting than adolescent lives, especially if you have children. And being tired definitely correlates with akrasia.
Data point: I believe the amount of time I spend on LW/Internet conversations has gone down after I’ve started beating some akrasia issues. Also, the causality seems to go in both ways. (Less time online leads to getting more done, while getting more done means less time-wasting online.)
Amusingly, Google Plus has helped eliminate the time lost to Facebook usage, though in an odd way. After I started using G+, I would cross-post all of my links and status updates to both services. Now, to limit my Internet usage, I have intentionally avoided getting a broadband connection to my home: my primary Internet access is the GPRS in my phone. GPRS is too slow to use G+ with: the page simply won’t load well enough to allow for the posting of updates. I can still update Facebook with it, but since it would feel wrong towards my G+ followers to make an FB post without cross-posting it to G+, I don’t do that either. As a result, I don’t post anything on FB/G+, and don’t get stuck in checking my FB/G+ updates for likes and comments every couple of hours. This helps considerably in getting things done. I now update them only once a week or so, when I’m at the university.
Also, that Ann/Bob/Carol question snagged a bunch of people. We have a lot of work to do.
Agreed. I wonder how many people got it wrong because they thought “married” and “unmarried” were not jointly exhaustive.
The use of phrases like “help procrastination/akrasia, and not harm it”, “improved procrastination”, etc. is a bit confusing. It’s not immediately clear whether you mean “more procrastination” or “better, that is less procrastination” in each particular case.
These results were in bold and were personally striking to me. Also, I’m not sure what to make of it, but it looks like the Anne question is not significantly correlated with the exercise question.
If I remember correctly, I contributed to these correlations with positive answers on all three, but I would not have expected those things in my life to be related to each other. It suggests that some aspect of my life is systematically related to these features, which seems mysterious and pragmatically important. I would love to hear guesses about what’s going on from anyone with insights or links :-)
Well, the dataset was small, and the correlations were all fairly weak (which is normal in social science; high-validity tests are generally made from multiple questions, and the correlation between individual questions on e.g. an IQ test is I think similarly weak, though I don’t remember). The Anne and exercise questions kind of correlated in the data set, but not at the significance level I was using as cutoff: c=.2, p=.12 (i.e., there is a 12% chance of seeing a correlation that strong by chance).
It’s possible that both correlates these of caffeine were due to chance (p=.004 is significant in most contexts, but less so when one is comparing 25 questions against one another); but also possible that they are real, and I find I’m tempted to start using caffeine again; anyone want to do some literature searches and find corroborating or anti-corroborating evidence for us?
No, a 12% chance of seeing a correlation at least as strong. Confusion about p-values is endemic! Please be super-careful explaining what they mean! (Specifically, in this case, you don’t want people thinking something like P(result | effect is real) = 1 and P(result | effect is false) = .12; I think that would be overstating the evidence.)
The issue isn’t how many questions one is comparing, but rather what is the prior for this specific correlation. I don’t think the prior for caffeine correlating with productivity is that low, and the .004 probably translates to pretty strong evidence. Of course, separately from that, you have to worry whether the correlation represents coffee → productivity causation.
Good clarifications; thanks. Also, I hadn’t noticed that the p-value for the caffeine/exercise comparison was also small (p=.008); on reflection, I agree with you that one or both correlations is likely to be real. Maybe I really will start drinking coffee again. (Though note that caffeine did not correlate with self-reported procrastination levels, nor with income, nor happiness (which is otherwise a strong anti-procrastination predictor).
Huh; I think I was in fact making thinking errors here, even though I understood your points enough to have explained them many times to others. My thought had been that, while it would be better to directly estimate a prior if I could do so accurately, doing so would be hard for two reasons:
Hindsight bias (plus the fact that I wrote no such priors down ahead of time);
Lack of practice with questions generated in this manner. In daily life (or while playing calibration games with trivia cards), questions are selected so that quite a high frequency have both “yes” and “no” as answers. Given the prior one should have over a randomly selected such question, I am therefore usually hesitant to assign < a 5% probability to anything that doesn’t make me think “no way could that possibly happen”, because, when I’ve done calibration practice, I’ve found that that’s what “only a 5% chance” feels like from the inside. But when one’s questions are generated from a process of automatic comparisons rather than a process of deliberate conjecture, the priors can be lower.
I was hoping, by attention to the number of questions involved, to get a feel for the latter effect. But on reflection it seems you’re right and I would have done better, in practice, to have thought about the odds of the kinds of comparisons I was actually running being true; hindsight bias or not, caffeine helping with System II overrides is clearly in a different hypothesis category than e.g. the sorts of compound / health value correlations that are sometimes mass-evaluated in medical research.
Steven, or others: if someone were to do a survey of e.g. Berkeley math majors, or others who seem similar to the LW population, what odds would you give on the caffeine/Anne correlation holding up? This might be a good subject for bets and rationality practice.
It sounds like a lot of effort for just one question. It would be easier to have someone transparently pick some science papers that people hadn’t heard of, and then have people guess at what their conclusions were.
This is a good idea; however, it is possible to update fairly strongly on the fact that a paper on the subject in question was published.
Oh, that was sloppy of me, I read your “should I use caffeine again” comment and jumped to the conclusion that the correlation being discussed was between caffeine and self-reported akrasia. The correlation still doesn’t seem out of the question.
My cop-out answer: it depends on sample size, other aspects of the experiment design, and how you construe “holding up”, but probably not much less than 50%.
Suppose, for simplicity, that there is either a real correlation of about this size, or no correlation at all. (This is of course a simplification, since there could also be correlations of smaller or greater size.)
So, if we assign a 5% prior to caffeine in fact helping with the Anne question, and if we assume that the chance of seeing a chance at least this large if there is a real correlation is 50% (I’m not sure whether this is reasonable? but I’m hoping the observed correlation would be approximately centered around the actual correlation) our posterior should be:
.05.5 / (.05 .5 + .95 *.004) = 87%.
If we assign a 1% prior to caffeine in fact helping with the Anne question, the posterior (under the same simplified assumptions) should be: 56%.
If we assign a 10% prior (under the same simplified assumptions), the posterior should be: 93%.
I haven’t thought much about whether this is the correct way to formalize/simplify the question, so these numbers may be misleading.
You’re updating on the fact that we observed at least some value, when what we know is we observed exactly that value. I think this overstates the evidence against the null, because all those higher results would have been stronger evidence against the null than the actual result is, and you haven’t told the formalism that those higher results in fact didn’t happen. (ETA: I recommend the Goodman paper linked toward the end of the great-great-grandparent comment, if you haven’t seen it already. It sets upper bounds on the amount of Bayesian evidence you can get from any given p-value. The image with the table has some concrete numbers.)
If the results were by chance, that’s one reason why they probably wouldn’t replicate in Berkeley students, but there are other ways. For example, there could be some common factor like age in the LW population that caused both coffee use and correct answers to the Anne question, but that wasn’t important in the Berkeley population. Or they could just fail to replicate by coincidence, with a probability depending on what counts as “fail to replicate”. So even if I were, say, 95% convinced that this result wasn’t a coincidence, I think I still wouldn’t be more than say 70% sure that it would replicate.
Nice job, thanks.
Couple of nitpicks:
I find that numbers in a table are easier to read when aligned around a decimal point. The same effect makes it harder to interpret .26 right below .3 when skimming. I suggest using the same number of digits and writing .30 there.
Thanks; typo fixed.
Great results, beautifully reported. Thanks, Anna.
I have been having some success getting more work done using the trivial website mytomatoes.com. This is a 25 minute timer, you essentially precommit to working for 25 minutes. I find it relatively easy to nip in the bud my urges to diversion when I am still in the 25 minute precommitment.
In hindsight I’m not sure I answered the Anne question correctly. Is everyone either married or unmarried or do divorced and widowed people count as neither? This is not obvious to me (I am not a native speaker of English).
In a certain sense divorced could even count as both.
As can certain marriages.
E.g., my husband and I are married as far as the state of Massachusetts is concerned, but unmarried as far as the federal government is concerned. We’re married as far as we and our friends and families are concerned, and unmarried as far as various people we don’t really know are concerned.
I don’t worry about it much, except for practical purposes. Marriage is a property of maps, not territories.
I believe I agree with your general point here, but I am pretty sure I disagree with this precise formulation of it. I would consider “X is a property of maps” to mean “X makes claims about maps”—a key example would be “uncertainty”. Marriage, for any given use of the term, is a claim about interpersonal relationships; this may be between two people, between two people of differing genders, or either of the aforementioned plus relationships (of, typically, a different form) with a broader community. These are reducible in principle to mental states, which are properties of the territory.
The fact that there are multiple competing sets of properties that lay claim to the label in common use in different portions of our society is relevant to the broader discussion, but should not be taken to indicate that “marriage” itself applies to anything but the territory. Nor should the fact that there is nothing but social usage and practical considerations to map the phonemes to the concepts—this is true of any word.
Doesn’t “uncertainty” also reduce to a mental state?
Yes, in humans. And granted, in any other reasoning engine it will reduce to a physical state. This is mostly irrelevant, however. It reduces to a physical state because your map reduces to a physical state. This makes it still a claim about the territory when you’re a part of the territory (as is true for us, but wouldn’t necessarily be true for an entity watching our simulation or whatever), but it doesn’t make it not a claim about the map. Marriage, I assert, is not a claim about maps at all.
The most practical way to consider it for this puzzle, is to figure out the puzzle. As far as I can tell, the “right” answer is Yes, because if Bob is married, he is looking at an explicitly stated unmarried person (Carol). If Bob is unmarried, an explicitly stated married person (Ann) is looking at Bob (unmarried)--both of which make the answer “Yes.” Any other possibilities such as handfasting, estranged, or the possibility that Bob is not even a person, would fall into the “Quit being a wiseass and and answer the question already.” category.
Everyone is either married or unmarried. (At least for the purposes of a logic puzzle like this.)
Not so fast. If your definitions of “married” v. “unmarried” are non-binary—that is, if you consider “widowed” or “divorced” or “handfasted” to be categories of “other”, then you’re going to “get the puzzle wrong”.
If, on the other hand, it were a more definitionally binary question—“wearing a wedding ring” for example—that would be another story. Though I suspect it would be ‘gotten wrong’ roughly as frequently, since such views of “married”/”unmarried” are likely unconventional.
Is it worth putting a “tl;dr” summary at the top of this post? Something like “System II reasoning apparently not correlated with akrasia. Caffeine may be good. Exercise may be good. Deliberately choosing which tasks to work on may be good. Akrasia correlated with unhappiness and anxiety”.
Many thanks for the interesting data & write-up.
No. Put a summary summary!
The exercise/akrasia correlation found here was pretty much nonexistent, and you’d expect lack of akrasia to cause exercise, so if anything this seems to me to be evidence for exercise being harmful.
ETA: On second thought, I’m assuming (p=.3, c=.02) is a typo for (c=.3, p=.02). Then it’s not clear to me, after accounting for the above effect, which way the evidence goes.
Sorry, yes, that was a typo. Fixed now.
I’m wondering a little about what that means. I tend to choose carefully for quality or at least how much I like it, give some attention to price, and don’t give nearly enough attention to knowing what I’ve already got.
I think your table has two missing zeros- you report two p values of .8, which is much higher than your threshold. (The 3,18 cells)
Fixed; thanks.
I’m astonished by how many people attach such a great significance to that test. Three questions all of which are relatively common trick questions. For each question, I would expect about 60% of people to have already encountered it (or a very similar one) before somewhere else, about 20% of people who encounter it for the first time to give the correct answer, and about 90% of people who have encountered it before to remember (or remember how to derive) the correct answer.
In fact, I had thought that those three questions were only a sample, and when I realised they were the whole test, I was like ‘WTF?’
I think the researchers who use it were probably equally astonished at how much weight they end up putting on the results of the CRT, but it turns out to have massive predictive power—way more than you could possibly expect from a 3 question test. This is surprising, but that is not enough to reject it out of hand. Remember, if you think that a test which has been used repeatedly in a large body of scientific literature over many years has some obvious flaw or seems counter-intuitive for some reason, you should bear in mind that the scientists using it were probably at least as surprised as you are by the result, and will have made every effort to nail it down
As for the numbers you give, they cannot possibly be true. Even as late as 2011 (eg, in this paper) the mean number of correct answers given by participants was 0.7, and fewer than 5% of students got all three answers correct (your numbers would suggest a mean number of correct answers more than 1.5, and that more than 12% should get all three questions right, even if prior exposure to the three questions was independent).
Yes, it would be nice to have alternative tests to the CRT (I’d love to see a copy of the 8 question CRT Fredericks mentions briefly in one of his early papers on the topic), but the CRT happens to do its job pretty well. Intuitively, it seems as though it shouldn’t, but that’s why we don’t trust our intuition when we have data which contradict it.
Well, I must have overestimated those numbers (or maybe those kind of riddles are less popular where the study was performed than where I come from). Still, it might be interesting to also ask participants “Have you encountered this question (or a very similar question) before?” with possible answers “Yes, and I remember the answer (or how to derive it”, “Yes, but I have forgotten the answer”, “Yes, but I was never told the answer” and “No”.
I concur that the Ann/Bob/Carol question is more taxing than the Cognitive Reflection Test.
In fact I can prove for my own case sample size N = 1. I scored 3⁄3 on the CRT and I missed Ann/Bob/Carol as I did not look at Bob as being unambiguously either married or unmarried and shot myself in my own damn foot on the sucker.
I think I recall reading that when they actually give someone the CRT, they often mix those three questions in with a whole bunch of others… and just ignore all the other answers.
When I was a kid, I didn’t have a problem with procrastination. I procrastinated on purpose. Now, I have a problem with procrastination. I did it so much as a kid that I ingrained it into the depths of my mind.
Fortunately, I just now realized this. And now that I know it exists, it’s really easy to fix.
Oh, it definitely causes akrasia, for me. The sequences are a seemingly endless supply of novelty!
Edit: Never mind...it doesn’t cause my akrasia; it is merely how the tendency that is already there now expresses itself.
Edit 2: From now on, I commit to reading any post I desire to comment on in its entirety, before doing so. The original comment was pathetically irrelevant, sorry.