Musings on the LSAT: “Reasoning Training” and Neuroplasticity
The purpose of this post is to provide basic information about the LSAT including the format of the test and a few sample questions. I also wanted to bring light to some research that has found LSAT preparation to alter brain structure in ways that strengthen hypothesized “reasoning pathways”. These studies have not been discussed here before; I thought they were interesting and really just wanted to call your collective attention to them.
I really like taking tests; I get energized by intense race-against-the-clock problem solving and, for better or worse, I relish getting to see my standing relative to others when the dust settles. I like the the purity of the testing situation—how conditions are standardized in theory and more or less the same for all comers. This guilty pleasure has played no small part in the course my life has taken: I worked as a test prep tutor for 3 years and loved every minute of it, I met my wife through academic competitions in high school, and I am a currently a graduate student doing lots of coursework in psychometrics.
Well, my brother-in-law is a lawyer, and when we chat the topic of the LSAT has served as some conversational common ground. Since I like taking tests for fun, he suggested I give it a whirl because he thought it was interesting and felt like it was a fair assessment of one’s logical reasoning ability. So I did, I took a practice test cold a couple Saturdays ago and I was very impressed. Here the one I took. (This is a full practice exam provided by the test-makers; it’s also like the top google result for “LSAT practice test”.) I wanted to post here about it because the LSAT hasn’t been discussed very much on this site and I thought that some of you might find it useful to know about.
A brief run-down of the LSAT:
The test has four parts: two Logical Reasoning sections, a Critical Reading section (akin to SAT et al.), and an Analytical Reasoning, or “logic games”, section. Usually when people talk about the LSAT, the logic games get emphasized because they are unusual and can be pretty challenging (the only questions I missed were of this type; I missed a few and I ran out of time). Essentially, you get a premise and a bunch of conditions from which you are required to draw conclusions. Here’s an example:
A cruise line is scheduling seven week-long voyages for the ship Freedom.
Each voyage will occur in exactly one of the first seven weeks of the season: weeks 1 through 7.
Each voyage will be to exactly one of four destinations:Guadeloupe, Jamaica, Martinique, or Trinidad.
Each destination will be scheduled for at least one of the weeks.
The following conditions apply: Jamaica will not be its destination in week 4.
Trinidad will be its destination in week 7. Freedom will make exactly two voyages to Martinique,
and at least one voyage to Guadeloupe will occur in some week between those two voyages.
Guadeloupe will be its destination in the week preceding any voyage it makes to Jamaica.
No destination will be scheduled for consecutive weeks.
11. Which of the following is an acceptable schedule of destinations in order from week 1 through week 7?
(A) Guadeloupe, Jamaica, Martinique, Trinidad,Guadeloupe, Martinique, Trinidad
(B) Guadeloupe, Martinique, Trinidad, Martinique, Guadeloupe, Jamaica, Trinidad
(C) Jamaica, Martinique, Guadeloupe, Martinique, Guadeloupe, Jamaica, Trinidad
(D) Martinique, Trinidad, Guadeloupe, Jamaica, Martinique, Guadeloupe, Trinidad
(E) Martinique, Trinidad, Guadeloupe, Trinidad, Guadeloupe, Jamaica, Martinique
Clearly, this section places a huge burden on working memory and is probably the most g-loaded of the four. I’d guess that most LSAT test prep is about strategies for dumping this burden into some kind of written scheme that makes it all more manageable. But I just wanted to show you the logic games for completeness; what I was really excited by were the Logical Reasoning questions (sections II and III). You are presented with some scenario containing a claim, an argument, or a set of facts, and then asked to analyze, critique, or to draw correct conclusions. Here are most of the question stems used in these sections:
Which one of the following most accurately expresses the main conclusion of the economist’s argument?
Which one of the following uses flawed reasoning that most closely resembles the flawed reasoning in the argument?
Which one of the following most logically completes the argument?
The reasoning in the consumer’s argument is most vulnerable to criticism on the grounds that the argument...
The argument’s conclusion follows logically if which one of the following is assumed?
Which one of the following is an assumption required by the argument?
Heyo! This is exactly the kind of stuff I would like to become better at! Most of the questions were pretty straightforward, but the LSAT is known to be a tough test (score range: 120-180, 95th %ile: ~167, 99th %ile: ~172) and these practice questions probably aren’t representative. What a cool test though! Here’s a whole question from this section, superficially about utilitarianism:
3. Philosopher: An action is morally right if it would be reasonably expected
to increase the aggregate well-being of the people affected by it. An action
is morally wrong if and only if it would be reasonably expected to reduce the
aggregate well-being of the people affected by it. Thus, actions that would
be reasonably expected to leave unchanged the aggregate well-being of the
people affected by them are also right.
The philosopher’s conclusion follows logically if which one of the following is assumed?(A) Only wrong actions would be reasonably expected to reduce the aggregate
well-being of the people affected by them.
(B) No action is both right and wrong.
(C) Any action that is not morally wrong is morally right.
(D) There are actions that would be reasonably expected to leave unchanged the
aggregate well-being of the people affected by them.
(E) Only right actions have good consequences.
Also, the LSAT is a good test, in that it measures well one’s ability to succeed in law school. Validity studies boast that “LSAT score alone continues to be a better predictor of law school performance than UGPA [undergraduate GPA] alone.” Of course, the outcome variable can be regressed on both predictors and account for more of the variance than either one taken singly, but it is uncommon for a standardized test to beat prior GPA in predicting a students future GPA.
Intensive LSAT preparation and neuroplasticity:
In two recent studies (same research team), learning to reason in the logically formal way required by the LSAT was found to alter brain structure in ways consistent with literature reviews of the neural correlates of logical reasoning. Note: my reading of these articles was pretty surface-level; I do not intend to provide a thorough review, only to bring them to your attention.
These researchers recruited pre-law students enrolling in an LSAT course and imaged their brains at rest using fMRI both before and after 3 months of this “reasoning training”. As controls, they included age- and IQ-matched pre-law students intending to take LSAT in the future but not actively preparing for it.
The LSAT-prep group was found to have significantly increased connectivity between parietal and prefrontal cortices and the striatum, both within the left hemisphere and across hemispheres. In the first study, the authors note that
These experience-dependent changes fall into tracts that would be predicted by prior work showing that reasoning relies on an interhemispheric frontoparietal network (for review, see Prado et al., 2011). Our findings are also consistent with the view that reasoning is largely left-hemisphere dominent (e.g., Krawczyk, 2012), but that homologous cortex in the right hemisphere can be recruited as needed to support complex reasoning. Perhaps learning to reason more efficiently involves recruiting compensatory neural circuitry more consistently.
And in the second study, they conclude
An analysis of pairwise correlations between brain regions implicated in reasoning showed that fronto-parietal connections were strengthened, along with parietal-striatal connections. These findings provide strong evidence for neural plasticity at the level of large-scale networks supporting high-level cognition.
I think this hypothesized fronto-parietal reasoning network is supposed to go something like this:
The LSAT requires a lot of relational reasoning, the ability to compare and combine mental representations. The parietal cortex holds individual relationships between these mental representations (A->B, B->C), and the prefrontal cortex integrates this information to draw conclusions (A->B->C, therefore A->C). The striatum’s role in this network would be to monitor the success/failure of reward predictions and encourage flexible problem solving. Unfortunately, my understanding here is very limited. Here are several reviews of this reasoning network stuff (I have not read any; just wanted to share them): Hampshire et al. (2011), Prado et al. (2011), Krawczyk (2012).
I hope this was useful information! According to the 2013 survey, only 2.2% of you are in law-related professions, but I was wondering (1) if anyone has personal experience studying for this exam, (2) if they felt like it improved their logical reasoning skills, and (3) if they felt that these effects were long-lasting. Studying for this test seems to have the potential to inculcate rationalist habits-of-mind; I know it’s just self-report, but for those who went on to law school, did you feel like you benefited from the experience studying for the LSAT? I only ask because the Law School Admission Council, a non-profit organization made up of 200+ law schools, seems to actively encourage preparation for the exam, member schools say it is a major factor in admissions, preparation tends to increase performance, and LSAT performance is correlated moderately-to-strongly with first year law school GPA (r= ~0.4).
I can share my personal experience, but I’m an outlier on many axes.
I identify strongly with a lot of the things you wrote about liking tests.
I studied for and took the LSAT in the mid-2000s. I scored either a 166 or a 168 (can’t remember) on my first practice exam, which I took cold. Nearly all of the points I lost were on the games section.
Incidentally, this was a pretty emotionally difficult experience for me, because I had never done this poorly on a 200-800 (120-180) scored standardized test before. This one felt especially like an IQ test, so not being able to crack the 700 (170) barrier was very threatening to my self-esteem.
Your guess that most LSAT prep focuses on developing written systems for the games section is correct. Powerscore was the most highly regarded producer of LSAT prep materials for high scorers when I took the LSAT, and their reputation was larlgey due to their succeess in teaching people how to create accurate and efficient diagrams for the games. I think they also produced fairly well-regarded materials for the other sections too, but I didn’t investigate those because my deficiency was the games section.
My preparation consisted of working through the Powerscore games materials and taking lots of practice tests. LSAC published old LSATs in books of ten tests each, and three such books had been published at the time I was studying, so I bought them all and just took those 30 LSATs with a stopwatch. I ran out of old LSATs prior to the test, so I went online and found a torrent of dubiously legal low-quality scans of more recent LSATs, though I only had time to take one of those before the exam because the school year started.
On the logical reasoning and reading comprehension sections, I would guess that my mean missed questions per section prior to studying was about 3, with a standard deviation of about 1. After studying, it was probably about a mean of 2 with a standard deviation of maybe 0.7. I typically missed questions because I would read too quickly and miss something subtle and important. I did not improve significantly at this, even though I did a lot of practice. I did not use symbolic logic or any other systematized written system to answer any logical reasoning or reading comprehension questions; it’s possible that this would have helped.
On the games sections, my pre-studying mean missed questions per section was probably about 10, with a standard deviation of 3. After studying, the mean was probably 1 with a standard deviation of 1. Games went from being my weakest to my strongest section once I learned how to make and use good diagrams, and it became a fairly regular occurrence for me to finish the entire section with time to spare and complete certainty in the correctness of all of my answers.
Doing some of the older tests before I mastered the technique for the games, I scored as low as, IIRC, 158. This resulted in much hyperventilating and cruel self-talk. As I improved at the games and worked my way up through more recent tests, my scores improved to the point where I was consistently in the 170s, and achieved a few 180s. That one torrented practice test that I took the week before the actual test was the last practice test I took, and I got a 169 on it somehow, which was, of course, terrible for my confidence going into the exam.
I scored a 175 on the actual test.
The entire reason the LSAT is hard is the time pressure. I think that, given enough time, any 150s-scorer could arrive at the right answer to any LSAT question. I agree with ThisSpaceAvailable that the games section is very much a “can-you-follow-instructions-under-time-pressure” test, and my experience was like ioshva’s in that my missed logical reasoning questions were due to lapses in focus while reading under time pressure rather than inability to grasp the logic.
I do not believe the process of studying for the LSAT inculcated any rationalist habits of mind in me. For me, because of my strengths and weaknesses, the process of studying for the LSAT was largely the process of learning how to complete contrived little puzzles that are isomorphic to sudoku. I suppose there might be some Lumosity-type minor positive effect on working memory, but I doubt the LSAT is much better for that than any other mentally engaging task. The research you cited in the OP just makes me want a control group of people playing Lumosity games. I can imagine that, for someone whose weakness was the logical reasoning section and who was missing lots of logical reasoning questions because of a deficit in understanding of the basic mechanics of logic, studying for the LSAT would be one way to correct that deficiency. But I suspect that ioshva is correct in that just learning logic directly would be a more effective way of achieving the same result.
I do not believe the process of studying for the LSAT helped me in law school. I think the correlation between LSAT score and first-year law school GPA exists largely because high-IQ and high-conscientiousness people tend to do well on both. I performed abysmally in law school despite my relatively high LSAT score because of pathologically low conscientiousness. I was only able to overcome low conscientiousness during the LSAT preparation process because I was living with my parents at the time and they put considerable pressure on me to study hard.
LSAT score is certainly a huge factor in law school admissions, and I was admitted to a moderately prestigious law school solely on the strength of my LSAT score and in spite of a truly awful undergraduate GPA. In general, LSAT score, undergraduate GPA, and underrepresented racial minority status are really the only meaningful factors in the law school admission process. This holds except in weird edge cases where people have distinguished themselves in exceptional ways, like former pro athletes or people who have founded successful businesses or nonprofits. Things like recommendation letters and essays can potentially keep you out if they’re bad, but will not get you in except, again, in weird edge cases like having a Supreme Court justice write you a recommendation insisting that you’re the brightest mind of your generation.
Incidentally, on the “think like a lawyer” point raised by MaximumLiberty—in my experience, this usually refers to a couple of specific analytic skills that lawyers are trained to develop: (1) reasoning by analogy and distinction, as with case law analysis, and (2) thinking strategically about the way systems of rules constrain or fail to constrain the behavior of agents within them, as with advising a client on how to avoid punishment. I’m not sure that this sort of software specialization would be reflected in the hardware imaged by an fMRI.
I wanted to thank you for your response and to apologize for not getting to read it sooner (I am in the throes of final exams, project due dates, et c.). The Lumosity control group or some similar intervention is a great idea and probably the only way to know for sure if LSAT prep had any unique effects.
I’m a lawyer, over 20 years out from law school. I took the LSAT cold, so I’m not a good candidate for your questions. I’ve always liked taking tests and always did well on standardized ones. I did well on the LSAT.
The reason I am responding is to add a bit of information. Lawyers talk, among ourselves and to law students, about what it means to “think like a lawyer.” It is a topic of fairly serious debate in jurisprudence for a number of reasons. One is that lawyers have a lot of power in American society. There are issues of justification and effects there. Another is the underlying sense that we really do think differently from most people. We see it in our everyday lives and it sparks our curiosity. There are many other reasons.
So, it makes me wonder what the MRI images would show when comparing lawyers’ brains to comparable non-lawyers brains.
It seems to me that if you remove the time constraint, the g-level is really quite low. It says “Jamaica will not be its destination in week 4.”, so just go through the options and cross out all the options where Jamaica is its destination in week 4. That gets rid of D. “Trinidad will be its destination in week 7.” gets rid of E. “Guadeloupe will be its destination in the week preceding any voyage it makes to Jamaica.” gets rid of C. “at least one voyage to Guadeloupe will occur in some week between those two voyages.” gets rid of B. That leaves A. I don’t see how intelligence comes in, except insofar as it allows one to perform the task more quickly. It’s not a “Can you do logical reasoning”? test, it’s a “Can you follow simple instructions under time pressure?” test.
(1) Yes, but I’m an outlier. I started in the 99th percentile and “improved” 6 points through self-study.
(2) Honestly? Not really. For me, most of the difference in performance from test to test was not due to logical misapprehension, but because I skimmed a question or misjudged the time limit. If you have taken an undergrad logic course and have a grasp on conjunction/disjunction, sufficient/necessary, etc., then your experience will likely be similar. For instance, you said this about the games:
This is true, but it’s also true of the args, too. I’ve taught for two of the major test prep companies and the courses are mostly just an undergrad logic course bolted onto their own proprietary shorthand and systematic categorization so students can recognize types of questions and diagram them accordingly. When I tutored independently, I just used regular old symbolic logic.
The only tricky part about teaching that question (which I can’t recall teaching specifically) would be that most novices will diagram AW– or something similarly distinct from the premise elements for neutral welfare. So you have to teach them to diagram conclusions in the form of premises whenever possible.
(3) Well, I already had most of these skills, but I would say I definitely got a lot out of teaching. It’s a fun test compared to, say, the MCAT. I don’t think it would be all that great as a self-improvement tool, though. Without a tutor you won’t always understand where you’re screwing up, and some of the questions are sometimes tricky for the sake of being tricky. For instance, there may be a version of that question where the answer turns not on logic, but on one of the premises subtly leaving out the word “reasonably.” Additionally, there really aren’t that many different kinds of questions. Once you start looking at more than a few tests you will start to recognize lots of questions that are logical repeats with different subject matter, or maybe a reversed answer condition (which of the following is NOT implied versus which of the following IS). If all you want is to improve your logical reasoning personally, I’d just take the undergraduate logic courses.
Thank you for this awesome, informative comment. I’m glad to get some perspective on this; at the end of the day I guess it is just a test of basic logic concepts… I guess I shouldn’t expect that to carry over to other areas of one’s daily life.
1) I took it, but I didn’t do much studying for it. (Basically, I signed up for it at nearly the very last moment after I saw someone mention that all it took to get into law school was a good LSAT—I had been pursuing a different career and had not previously thought of going to law school, but I had started doing legal-related work in a volunteer gig.) Maybe a week before the exam I went to the library and checked out a prep book. And the logic games section was already something I basically knew, so what I did spend time on was careful reading of the critical reading sections; I tend not to read carefully and miss instructions, and I wanted to learn the kinds of tricks they were likely to use to get me to do just that.
2 and 3) No; I used the logical reasoning skills I had already from studying math. (Also, from having taken every vaguely logic-related course at my undergrad.) Those were long-lasting. But I enjoyed math because many of those skills were already natural to me. I learned refinements and additional techniques and became better at it, but I was already inclined to thinking that way and enjoyed it.
As a lawyer now, one of my major strengths lies in analytical reasoning—I like to consider situations and take apart the possible situations that may arise, what happens if they’re taken to their logical conclusions, where contradictions might arise from sets of terms, what logical inconsistencies exist in a proposal. (The biggest and most enjoyable project I’ve worked on has been license drafting.)
Also, logical reasoning of the type on the test hardly showed up at all in law school—most of the reasoning required was not very complicated, so most reasonably intelligent college graduates would already be able to do it.. (Some more complicated logic showed up in Conflicts of Laws, also.)
Would you consider schedule C,
to fail due to the following requirement?
The language seems slightly ambiguous in the case where the trip to Jamaica is in the first week, but my guess is that they intended to rule that out.
Yeah, I think that one is ruled out because they are scheduling only 7 voyages (with no memory of prior voyages). I see what you mean though; it doesn’t say anything about prior voyages, but I think of it as 7 slots to fill, and since there is no slot before 1, it can’t be Jamaica. The answers are at the end of the test (pg 38).
I don’t understand what the neuro stuff in the second half of the post is meant to contribute here.
If the LSAT (or studying for it) has measurable cognitive benefits, or is predictive of future success, awesome! So who cares whether you can detect some cerebral blood flow difference with fMRI?
If the LSAT has no predictive (or other) value, sad! So who cares whether you can detect some cerebral blood flow difference with fMRI?
Obviously this is in no way specific to your post, don’t mean to single you out. But the “it happens IN THE BRAIN so it’s super important” way of thinking is to my mind a corrosive trope in contemporary discourse which I like to call attention to whenever I see it. In my opinion these kind of New Yorker-friendly brain imaging studies tend to obscure more than they illuminate for most issues.
But to answer your questions in the final paragraph 1) Yes I studied for it, quite a bit actually (was studying for the GRE in parallel, which was actually more important, but the LSAT studying was more fun). 2) I’d say yes it definitely improved my logical reasoning skills but no, (3) there was likely little lasting benefit.
Two reasons: (a) because as a general rule interventions that boost cognitive performance in the short term tend to wash out over longer timescales, so my default position would be that the same is true for the LSAT, and (b) LSAT-type reasoning wasn’t terribly far removed from the kind of thinking I was used to, so the marginal benefit was probably small.