Reverse-engineering what really counts as “smart”
I: The game
In business undergrad, a third of your grades was based on in-class participation, which was scored across five tiers:
-1: You literally said something sexist.
0: No contribution.
1: A “case fact;” surfacing a relevant detail from the reading.
2: An analysis; some work required to get to a fact.
3: An insight; something really smart.
By the final semester, it was more interesting to try maximizing contribution grades while minimizing case reading. And because the fuzzy boundary between 2-to-3 felt more like a “you know it when you see it,” we spent that semester reverse-engineering past top contributions we’d seen.
That semester, in a grading system curved tightly around an 80-82, our average contribution grades across 10 courses combined was above 92 (roughly +2 standard deviations above average).
Importantly, the effort we input did not change, only the strategy. Instead of preparing for class, we tracked along with class discussion and made points opportunistically. Our contributions improved while somehow doing less work than before.
We first identified at least three common types of Level 3 contributions:
Expertise: Somebody volunteers ‘proprietary’ knowledge from their work or life to explain how something works, or they surface considerations not mentioned in the reading.
“At my internship, 80% of my learning was through osmosis, and I would have been less likely to return if it was all-virtual, which this case hasn’t considered.”
Deductions: Revealing that if an idea is implemented, it may have indirect consequences, so we should either choose another option, or generate some mitigations.
“Switching to remote reduces office overhead, but it’ll weaken onboarding, which increases ramp time, which slows growth; which was the firm’s top priority.”
Implication: Showing that an idea should actionably lead the case decisionmaker to one of the potential options.
“We can still go remote to save money; we’d just have to implement any low-cost mitigations within our control, for example, structured mentorship layers.”
The common thread: each required doing a piece of invisible work that no one else had done yet.
II: Unit of Work
Effective contributions require some work to achieve, but not because effort is intrinsically useful. Calculating an irrelevant financial ratio can get scored as a 1, or even a 0.
Instead, work is incidentally useful because of a selection effect: If the contribution was valuable and the answer was obvious, everyone would have known it already—being valuable and new means an obstacle had blocked people’s knowledge of it.
A unit of work is defined as the abstract quantity equal to ‘whatever it takes’ to overcome that obstacle, and bring new knowledge to the audience.
Expertise requires work: not just in the moment, but earlier; and then in identifying if your fact is calibrated to aid a classmate’s understanding.
Deductions are a unit of work: modeling Nth-order-effects that others didn’t have the intuition to recognize.
Implications are a unit of work: converting facts into prescriptions by matching decisionmaking levers to goals.
After recognizing this pattern, you can shed the particular “contribution types,” and focus on the unit of work generally.
Any analysis can be insightful if it unblocks a conceptual phase shift from Understanding A → Understanding B.
Noticing a connection between disparate concepts; identifying two different observations as instances of the same thing; realizing you can question a taken-for-granted assumption; can all be units of work.
And the most thoroughly ‘blocked’ understandings aren’t the facts you had no clue about, but the facts you thought you already knew. The most Herculean units of work don’t guide you from A → B but swing you from A → Opposite-of-A.
Critically, class contributions were not graded proportional to their labor, but to the magnitude of the conceptual phase shift they induced.
For this reason, you could often tell a comment was a “Level 3” the moment it landed: it produced an epiphany.
That’s why calculating Operations or Marketing math to arrive at a number, while still ‘work,’ wasn’t an ‘insight,’ because the outcome isn’t of a fundamentally different sort than what you expect.
Centering ‘insights’ or ‘conceptual phase shifts’ in your contributions can cloak you in the guise of expertise, even when you have little.
III: Modeling Expertise
The most common Level 3 contribution was ‘expertise.’ But because students tend to have so little of their own, it usually took the form of “proprietary” knowledge they inherited from elsewhere in their life, like processes codified at their internships. This worked for at least two reasons:
First, the “contact with reality” from real-world experience selects for empirically tested explanations, and pragmatically feasible solutions.
Since most students lacked the experience to have generated their own knowledge, their insightful contributions were trojan horses: smuggling in ideas borrowed from others’ hard-earned experience, disguised as personal insight.
Second, experts can better model the ‘ladder of knowledge’ in their field. Suppose a field has 5 levels of understanding.
There exist “merely credentialed” experts who traffic only in facts subversive to their peers, whose perspectives are already absurd to the average person.
But there also exist “good experts” who can first identify that you’re at Level 2, then know which facts & framing can flip your understanding into Level 3. The best experts do this incisively at every stage, so everyone they meet leaves impressed that they only ever reveal ‘Herculean’ insights.
True expertise of that sort is unsustainable for class contributions, because you can’t be an expert on everything. Which is why “units of work” and “conceptual phase shifts” help.
The average layperson doesn’t just lack the sense of what knowledge is at each rung—they’re oblivious to being on any ladder at all. Discovering insights is accidental, “lightbulb moments” out of nowhere.
But by knowing a ladder exists, and that climbing it requires subverting your misconceptions, you can sometimes guess the expert “A → Opposite-of-A” conclusions without being an expert yourself.
And this became our strategy in class.
Consider a simple question from an accounting class: Which airline routes depreciate planes more: long or short flights? A student intuitively answered “Long routes—more activity means more depreciation.”
But with a sense for the structure of knowledge you try asking, if the kneejerk counterintuitive position were true, what would the likeliest explanation be? If nothing sparks, no harm, because the real, intuitive explanation would not have been insightful anyway (it would have been obvious). But if something does, you get to bring a conceptual phase shift to the class.
“Short routes—more frequent trips mean more takeoffs and landings, which is when most of the depreciation happens.”
The professor (actually!) replied, “Are you an engineer?”
Though this wasn’t an exceptionally impressive answer, she still in other words asked, are you an expert? Simulating expertise, by spotting and articulating the right phase shifts, can lead to the same insights a true expert might offer.
IV: The Ultimate Failure
Over time, many students started exhibiting the verbal “artifacts” of good contributions, without understanding the mechanics of why they worked. They had induced what a good contribution sounded like.
This is how some classmates can misunderstand “bring outside experience,” to mean “connect the case to the Bolshevik revolution.”
Another common problem was contrarianism for its own sake. Anything can be negated (any “A” can be rephrased into “Opposite-of-A”), but most intuitions exist because they’re largely correct.
Even when you do have a “true insight”, the failure can be the mismatch between it and the audience.
If you imply that “most people believe A, when actually, opposite-of-A”, but everyone already believed opposite-of-A, you’re the fool. However, there truly are people who believe A, and if you told them, your reception would be far better.
“Knowing your audience” is often taken for granted; because when you assess an insight objectively, there is no audience in the frame. At least, it appears that way. But an insight only matters if it lands. And the only place it can land is in someone else’s mind.
In terms of class participation, the objective components of insights don’t tell the whole story. There’s a unique constraint to being graded at school: Pedagogy. As a professor, you must cover a case’s main takeaways in an hour of class, otherwise your class won’t learn the discrete pieces of knowledge they’ll need for the final exam.
So as a student, if you anticipate what the case is trying to teach (which set of critical insights the professor is hoping to surface), you can delight the professor with natural segues to help them keep their lectures at pace. You become a reliable tool for advancing the professor’s goals.
Assisting their pedagogy makes you likeable and memorable. And as the prof sits eyes-closed in their office trying to pin a contribution grade for 80 students per class, you’ve gifted them a no-brainer there, too.
Consider the types of contributions again, not their objective properties this time, but their subjective properties:
Deductions are insightful for the person to whom the knock-on effects didn’t occur; and in the case, it is insightful because those effects impact the decisionmaker’s goals.
Implications are insightful for the person who didn’t yet connect facts to the case goal, and for the decisionmaker for showing how they have control over solving the problem relevant to them.
Expertise feels insightful if it plugs a gap in your existing knowledge that was otherwise outside of your access.
An evergreen problem people have is coming up with a “so what”—in essays, in work projects—when the solution is to instead ask the nearly identical question, “who cares?” Colloquially, it’s the same question; but it shifts the focus from the objective properties of your work to the invisible, subjective properties of the audience.
Work can only be valuable in the mind of an evaluator. There is no other place for value to live. And once you realize that, insight becomes less mystery, more skill.
You no longer have to flail; throwing out units of work at random, hoping to stumble into insight.
Epiphany becomes engineered.
Ironically, trying to “hack” school taught us even more.
It sounds like you successfully mastered a number of important business skills. Like understanding the voice of the customer, market trend analysis, designing a product the customer wants to buy and pay more for without raising costs, business model innovation, and effective advertising.
I don’t think your educators are self-aware enough to have intentionally Miyagi’d you, but I’d say the system worked anyway. It speaks well of you that you’d rather give away this advantage than exploit it.
This was painful to read. I feel like three—maybe five—paragraphs would be sufficient to tell your story. As it is, I kept wondering why I’m learning jargon like “level 3 insights” or what bits of information an entire section added. My version:
Hacking Business School
Goodharting “insights”.
In business school, in-class conversations constitute a significant portion of your grade. These are scored roughly by:
-1: You said something that actively hurt the learning process (e.g. blatant falsehoolds or -isms).
0: You said nothing, or nothing useful.
1: You brought up points from the readings.
2: You did some analysis on the readings (e.g. expected cost of policy A vs. policy B).
3: An insight; something nonobvious, and nonobvious how to arrive there (this is what makes it different from a mere analysis).
Obviously we wanted the highest grades, but we also didn’t want to (1) spend many more hours searching for insights, or (2) rely on luck to generate them for us. Just because it is not obvious how to arrive at insights to the rest of the class—or even the professor—does not mean there is not a formula to find them. We started by noting down which comments received ’3’s, then Goodharted on this objective.
We found three kinds of insights: those based on prior knowledge, inductive reasoning, and deep deductive reasoning, where deep means going farther than most people would assume necessary. Unfortunately, as students, we didn’t have a wealth of prior experience to draw upon, and inductive reasoning is hard, like NP-hard, so only the third kind of insight seemed like a plausible path. However, searching deeper than everyone else for insights would obviously require more work, which we’re quite opposed to.
But then… we stumbled upon a solution. Any time someone stops a search is an opportunity to go a little deeper. If someone says, “let’s assume for now...” and the rest of the class is nodding along, well let’s check that assumption! Most of the time, our assumptions will be right, but even if they’re wrong 10% of the time, we can expect to find a “nonobvious insight”.
For example, one class our professor began an analysis, and in a leading-the-witness style asked us, “Which airline routes depreciate planes more: long or short flights?” The knee-jerk response from most of the class was longer routes—because the plane is being used more (per flight). No one’s brain registered the “per flight”, until I questioned if being “used more” really means it depreciates faster. Suddenly, an insight was generated: maybe it depreciates more in the air on longer routes, but shorter routes have more frequent takeoff and landing—which might actually make them worse for wear and tear!
This simple trick, to question everything, ended up being the best insight I made in that class.