“How could I have thought that faster?”

[mesaoptimizer]

Link post

I stumbled upon a Twitter thread where Eliezer describes what seems to be his cognitive algorithm that is equivalent to Tune Your Cognitive Strategies, and have decided to archive /​ repost it here.

Sarah Constantin: I really liked this example of an introspective process, in this case about the “life problem” of scheduling dates and later canceling them: malcolmocean.com/​2021/​08/​int…

Eliezer Yudkowsky: See, if I’d noticed myself doing anything remotely like that, I’d go back, figure out which steps of thought were actually performing intrinsically necessary cognitive work, and then retrain myself to perform only those steps over the course of 30 seconds.

SC: if you have done anything REMOTELY like training yourself to do it in 30 seconds, then you are radically smarter/​more able/​etc than me and all the other people who do slower introspective practices.

SC: I don’t know whether to be impressed or to roll to disbelieve.

EY: I mean I suspect that this actually requires something like a fast perceptual view of minds as engines and thoughts as doing work and like actually draws on my mind design knowledge, but, even so, I ask: Do you constantly look back and ask “How could I have thought that faster?”

SC: No, I’ve never asked that.

EY: Okay, well, every time I’m surprised by reality I look back and think “What about my model and my way of thinking could I change that would have predicted that better, without predicting a bunch of other things worse?”

EY: When somebody at a MIRI workshop comes up with a math proof, I look over it and ask if there’s a way to simplify it. Usually, somebody else does beat me to inventing a proof first; but if my intuition says it was too complicated, I often am first to successfully simplify it.

EY: And every time I complete a chain of thought that took what my intuition says was a lot of time, I look back and review and ask myself “How could I have arrived at the same destination by a shorter route?”

EY: It’s not impossible that you have to be Eliezer Yudkowsky for this to actually work—I am never sure about that sort of thing, and have become even less so as time goes on—but if AI timelines were longer I’d tell somebody, like, try that for 30 years and see what happens.

EY: Man, now I’m remembering when I first started doing this consciously as a kid. I called it Shortening the Way, because a rogue rabbi had recently told me that “Kwisatz Haderach” was actually a reference to a Kabbalistic concept about teleportation, so that term was on my mind.

[Ruby]

I read this and think “ah, yes, this is valuable and important and I should be trying to do that more”. And thought as much when I first read it. I don’t think it stayed on my mind. It’s too compressed and not a ready a cognitive strategy.

But taking a few moments to extrapolate it into something better, starting with why I’m not doing it to begin with:

• A reason I don’t do more of this is I can’t do this on the order of 30 seconds. My guess is constructing a picture of what mental operations I did and what I could have done alone is the work of many minutes.

• For the kinds of reasoning I really wish I’d done faster, they happened over time and it really would take a bunch of mental excavation to reconstruct my reasoning.

• I don’t have a well-specified ontology for mental operations such that it’s easy to specify changes. (In contrast, I have a very clear ontology for driving a car and realize error + rehearse doing it differently in 30 seconds feels doable there). This means the work of figuring out how to do better is trying to carve out descriptions of what went wrong.

• The things that went wrong run deep, or something, into weird emotional territory that are hard to analyze.

• Solving problems and reaching true conclusions is hard enough that I’m caught up on that level, from one problem to the next, such that I feel too busy for reflection.

Yet I don’t fully buy all the above.

I do think that to do more of this, to make it a habit, it’ll need intentional practice. Scheduled blocks of 30-min on the schedule. Seems worth it, I should add it to ye old exobrain to remind me. I’m forming an intention to try it.

The other piece is the noticing. I don’t think I have a part of my brain that registers a “reached some milestone” event such that other actions could be triggered by it. Something, something Logan’s Noticing sequence. I’ll try that.

Ok, so where does that leave me regarding this crosspost?

I want to give this a 4 because it’s Rationality stuff from Eliezer. I don’t think I can because that great seeming, I don’t see that people will have a lot to do with it, without a bunch of unpacking (as I’m attempting). Then again, if I do the post-inspired work for a while and get great gains, I might want to say “it was short, but it had such a large effect on me it, it was def worth a 4 or even 9!”

[Raemon]

I’ve spent a lot of time figuring out how to implement this exercise, which I wrote up in The “Think It Faster” Exercise (and slightly more streamlined “Think it Faster” worksheet).

I’ve reviewed it more thoroughly over there.

[XelaP]

I would not normally vote on this post, as the technique of “How could I have thought that faster?” seems extremely obvious to me but also very important if you are not in fact trying to improve your thinking after being surprised (or any other shortcoming). Since this post has 241 upvotes and multiple comments from people (example: Said Achmiz, who is not an idiot!) and others disagreeing with the framing, I have review-upvoted this post.

I think the framing of “think it faster” is specifically something you should track, beyond just “What did I learn here really?” (which I see as important subskills that help you figure out how to think it faster) or “How could I have thought that with less information?” (which I see as fully subordinate to thinking it faster, because you get later info later). By focusing on thinking it faster, you focus on cognitive strategies—on how you could’ve approached the issue differently with what you knew at the time, or maybe you should’ve put more/​less stock in a certain kind of evidence.

The main problem with this post is that it gives no guide for how to go about learning how to think faster. Maybe you can’t come up with a good guide, but for this sort of thing a list of examples is itself useful.

Here’s a list of examples (that are too abstracted—next time I encounter something that I see how I could’ve thought it faster, I’ll write it down, and when I’ve gotten a bunch I’ll either post about it or add to this comment):

Say I am trying to prove a theorem. I will pursue a couple approaches, and then finally get something that works. When I look back on what I did, I will often find that I should’ve known better. Common problems:

1. Spending too much time up front trying a direct approach instead of just looking at small examples.

2. Not previously picking up on a general strategy for problems similar to the one at issue.

3. Spending a bunch of time trying to prove the theorem true/​false and could’ve quickly figured it out had I switched to the other one earlier. Performing the “error analysis” feels like it’s a significant contributor to my mathematical (or physics) progress.

Likewise, for more mundane life stuff:

1. Caching a thought on poor justification back when I was younger (and dumber). Especially if I was a kid when I cached it! Examples: Using shaving cream (it in fact actually works to make shaving not hurt), becoming a vegetarian (younger me had some shaky justification for not doing so). The implication is that I should more readily question what I take for granted, and mentally “decay” the trust I put in them as time elapses from when I last considered it.

2. Rationalizing to myself at multiple earlier points, or that I had ignored intuitive feelings of wrongness or confusion.

3. Ignoring plenty of early warning signals that I should’ve acted on.

4. Consistently making an error in the same direction instead veering hard the other way (assuming that it’s not risky to do so). For example, when learning to park a car I noticed that I was constantly turning in too early, and so I then decided to turn way later (against my intuitions). This instantly improved my parking a lot. It also suggests that I should apply this strategy everywhere. For tasks where failing in one direction isn’t much worse than failing in the other, you should shoot for being just as likely to miss in every direction—so if you expect yourself to turn too early, you should turn later until you are just as worried about turning too late (assuming you’re in an empty parking lot where you don’t have to worry about overshoots hitting someone).

Or when I’m surprised by e.g. the news or a factoid, I might’ve:

1. Under (or over) estimated how common a certain phenomena is (“Weird outcome X actually occurs in 70% of cases”)

2. Underestimated the state of the art in some field (like gwern’s list of side channel attacks like determining what someone was saying by watching a bag of potato chips or identifying you by your heartbeat as detected by an invisible laser from 200 m away

3. That there a bunch of pieces of evidence that I failed to notice or consider that could’ve predicted it (usually I discover these by making surprised sounds at my friends. Your mileage might vary if your friends aren’t as good at examining their reasons for not being surprised or if they wouldn’t feel surprised by it either way)

The best updates are more general, but unfortunately those are harder to discover.

[Eli Tyre]

So I think this post is pointing at something very important for my personal rationality practice, but it gives me almost none of what I need to actually do it successfully.

[mesaoptimizer]

Bonus conversation from the root of the tree that is this Twitter thread:

Eliezer Yudkowsky: Your annual reminder that you don’t need to resolve your issues, you don’t need to deal with your emotional baggage, you don’t need to process your trauma, you don’t need to confront your past, you don’t need to figure yourself out, you can just go ahead and do the thing.

Benquo: By revealed preferences almost no one wants to just go ahead and do the thing, even if they expect that things would go better for them if they did. Seems reasonable to try to figure out why that’s the case and how to change it, starting with oneself.

Benquo: Most of this trying will be fake or counterproductive, for the same reasons people aren’t doing the sensible object-level thing, but we don’t get to assume or pretend our way out of a problem, we just get to investigate and think about it and try out various promising solutions.

Given my experiences with both TYCS-like methods and parts-work methods (which is what Benquo is likely proposing one invest in, instead), I’d recommend people invest more in learning and using parts-work techniques first, before they learn and try to use TYCS-like techniques.

[Trinley Goldenberg]

The way I do this with my clients is that we train cognitive tools first, then find the resistance to those habits and work on it using parts work

[Raemon]

Say more?

[Trinley Goldenberg]

I usually explain my process these days to clients with the acronym LIFE

Learn New Tools Integrate Resistance Forge an Identity Express Yourself

Learn New Tools is cognitive-emotional strategies, of which TYCS is an example. Fwiw a some of TYCS is actually deliberate practice to discover cognitive strategies ( as compared to something like CFAR which extracts and teaches them directly), but the result is the same.

The important thing is to just have a clear tool, give people something they know they can use in certain situations, that works immediately to solve their problems.

But the thing is, people don’t use them, because they have resistance. That’s where parts work and other resistance integration tools come into play.

Even when thata done, there’s still the issue that you don’t automatically use the techniques. This is where forge an Identity comes in, where you use identity change techniques to make the way you see yourself be in alignment with a way of being that the technique brings out. (This is one thing TYCS gets wrong in my opinion, trying to directly reinforce the cognitive strategies instead of creating an identity and reinforcing the strategies as affirming that identity.)

Finally that identity needs to propogate to every area of your life, so there’s not situations where you fail to use the technique and way of being. This is just a process of looking at each area, seeing where it’s not in alignment with the identity, then deliberately taking an action to bring it to that area.

IME all of these pieces are needed to make a life change from a technique, although it’s rarely as linear as I describe it.

[Olli Järviniemi]

In addition to “How could I have thought that faster?”, there’s also the closely related “How could I have thought that with less information?”

It is possible to unknowingly make a mistake and later acquire new information to realize it, only to make the further meta mistake of going “well I couldn’t have known that!”

Of which it is said, “what is true was already so”. There’s a timeless perspective from which the action just is poor, in an intemporal sense, even if subjectively it was determined to be a mistake only at a specific point in time. And from this perspective one may ask: “Why now and not earlier? How could I have noticed this with less information?”

One can further dig oneself to a hole by citing outcome or hindsight bias, denying that there is a generalizable lesson to be made. But given the fact that humans are not remotely efficient in aggregating and wielding the information they possess, or that humans are computationally limited and can come to new conclusions given more time to think, I’m suspicious of such lack of updating disguised as humility.

All that said it is true that one may overfit to a particular example and indeed succumb to hindsight bias. What I claim is that “there is not much I could have done better” is a conclusion that one may arrive at after deliberate thought, not a premise one uses to reject any changes to one’s behavior.

[CstineSublime]

Eliezer Yudkowsky: See, if I’d noticed myself doing anything remotely like that, I’d go back, figure out which steps of thought were actually performing intrinsically necessary cognitive work, and then retrain myself to perform only those steps over the course of 30 seconds.

I wouldn’t mind seeing an annotated narrative or description of what that process of distilling a habit down into the parts which do the cognitive heavy lifting looks like

[Seth Herd]

This seems critical. The description given is very vague relative to actual cognitive steps that could happen for specific conclusions. How anyone could “retrain” themselves in 30 seconds is something different than what we usually mean by training.

[ChristianKl]

Eliezer wrote The 5-Second Level.

If you identify a bad 5-Second step, 30 seconds give you six training runs where you can go through the step to train it.

[HoVY]

I interpreted “retrain myself to perform only those steps over the course of 30 seconds” to mean that after training for n seconds/​minutes/​hours, he could solve an equivalent problem in 30 seconds (via the distilled steps). You seem to interpret it to mean that the training takes 30 seconds, and the length of time to solve the problem after training is unspecified.

I don’t know which it is, the wording seems ambiguous.

[Said Achmiz]

I would also like to see this. As it is, I’m not sure what the OP is even describing. (As noted in a sibling comment, description is very vague.)

[Mo Putera]

If I take this claimed strategy as a hypothesis (that radical introspective speedup is possible and trainable), how might I falsify it? I ask because I can already feel myself wanting to believe it’s true and personally useful, which is an epistemic red flag. Bonus points if the falsification test isn’t high cost (e.g. I don’t have to try it for years).

[DPiepgrass]

I guess you could try it and see if you reach wrong conclusions, but that only works isn’t so wired up with shortcuts that you cannot (or are much less likely to) discover your mistakes.

I’ve been puzzling over why EY’s efforts to show the dangers of AGI (most notably this) have been unconvincing enough so that other experts (e.g. Paul Christiano) and, in my experience, typical rationalists have not adopted p(doom) > 90% like EY, or even > 50%. I was unconvinced because he simply didn’t present a chain of reasoning that shows what he’s trying to show. Rational thinking is a lot like math: a single mistake in a chain of reasoning can invalidate the whole conclusion. Failure to generate a complete chain of reasoning is a sign that the thinking isn’t rational. And failure to communicate a complete chain of reasoning, as in this case, should fail to convince people (except if the audience can mentally reconstruct the missing information).

I read all six “tomes” of Rationality: A-Z and I don’t recall EY ever writing about the importance of having a solid and complete chain (or graph) of reasoning―but here is a post about the value of shortcuts (if you can pardon the strawman; I’m using the word “shortcut” as a shortcut). There’s no denying that shortcuts can have value, but only if it leads to winning, which for most of us including EY includes having true beliefs, which in turn requires an ability to generate solid and complete chains of reasoning. If you used shortcuts to generate it, that’s great insofar as it generates correct results, but mightn’t shortcuts make your reasoning less reliable than it first appears? When it comes to AI safety, EY’s most important cause, I’ve seen a shortcut-laden approach (in his communication, if not his reasoning) and wasn’t convinced, so I’d like to see him take it slower and give us a more rigorous and clear case for AI doom ― one that either clearly justifies a very high near-term catastrophic risk assessment, or admits that it doesn’t.

I think EY must have a mental system that is far above average, but from afar it seems not good enough.

On the other hand, I’ve learned a lot about rationality from EY that I didn’t already know, and perhaps many of the ideas he came up with are a product of this exact process of identifying necessary cognitive work and casting off the rest. Notable if true! But in my field I, too, have had various unique ideas that no one else ever presented, and I came about it from a different angle: I’m always looking for the (subjectively) “best” solutions to problems. Early in my career, getting the work done was never enough, I wanted my code to be elegant and beautiful and fast and generalized too. Seems like I’d never accept the first version, I’d always find flaws and change it immediately after, maybe more than once. My approach (which I guess earns the boring label ‘perfectionism’) wasn’t fast, but I think it built up a lot of good intuitions that many other developers just don’t have. Likewise in life in general, I developed nuanced thinking and rationalist-like intuitions without ever hearing about rationalism. So I am fairly satisfied with plain-old perfectionism―reaching conclusions faster would’ve been great, but I’m uncertain whether I could’ve or would’ve found a process of doing that such that my conclusions would’ve been as correct. (I also recommend always thinking a lot, but maybe that goes without saying around here)

I’m reminded of a great video about two ways of thinking about math problems: a slick way that finds a generalized solution, and a more meandering, exploratory way way that looks at many specific cases and examples. The slick solutions tend to get way more attention, but slower processes are way more common when no one is looking, and famous early mathematicians haven’t shied away from long and even tedious work. I feel like EY’s saying “make it slick and fast!” and to be fair, I probably should’ve worked harder at developing Slick Thinking, but my slow non-slick methods also worked pretty well.

[Raemon]

Curated.

I’ve spent the past few weeks independently interested in this concept (before mesaoptimizer posted it, actually). I reread the Eliezer tweet while investigating “deliberate practice for solving Confusing Problems™”.

I still have a lot of open questions on “how do you actually do this effectively?” and “how long does it take to pay off in ‘you actually think faster?‘”. But I’ve at least transitioned from “I feel like there’s no way I could have ‘thought it faster’” to “I observe specific earlier moments where I failed to notice clues that could have pointed me at the right solution” and “I’ve identified skills I could have had that would have made it possible to identify and act on those clues.”

I’ve personally gotten mileage from writing out in detail what my thought process was, and then writing out in detail “what’s the shortest way I could imagine a superintelligence or someone 40 IQ points higher than me would have reliably done it?”. The process currently takes me ~30 minutes.

A thing I haven’t attempted yet is:

Eliezer Yudkowsky: See, if I’d noticed myself doing anything remotely like that, I’d go back, figure out which steps of thought were actually performing intrinsically necessary cognitive work, and then retrain myself to perform only those steps over the course of 30 seconds.

I’m interested in other people trying this and seeing if useful stuff falls out.

[LGS]

Eh, I feel like this is a weird way of talking about the issue.

If I didn’t understand something and, after a bunch of effort, I managed to finally get it, I will definitely try to summarize the key lesson to myself. If I prove a theorem or solve a contest math problem, I will definitely pause to think “OK, what was the key trick here, what’s the essence of this, how can I simplify the proof”.

Having said that, I would NOT describe this as asking “how could I have arrived at the same destination by a shorter route”. I would just describe it as asking “what did I learn here, really”. Counterfactually, if I had to solve the math problem again without knowing the solution, I’d still have to try a bunch of different things! I don’t have any improvement on this process, not even in hindsight; what I have is a lesson learned, but it doesn’t feel like a shortened path.

Anyway, for the dates thing, what is going on is not that EY is super good at introspecting (lol), but rather that he is bad at empathizing with the situation. Like, go ask EY if he never slacks on a project; he has in the past said he is often incapable of getting himself to work even when he believes the work is urgently necessary to save the world. He is not a person with a 100% solved, harmonic internal thought process; far from it. He just doesn’t get the dates thing, so assumes it is trivial.

[RobertM]

Having said that, I would NOT describe this as asking “how could I have arrived at the same destination by a shorter route”. I would just describe it as asking “what did I learn here, really”.

I mean, yeah, they’re different things. If you can figure out how to get to the correct destination faster next time you’re trying to figure something out, that seems obviously useful.

[quetzal_rainbow]

“Lesson overall” can contain idiosyncratic facts that you can learn iff you run into problem and try to solve it, you can’t know them (assuming you are human and not AIXI) in advance. But you can ask yourself “how would someone with better decision-making algorithm solve this problem having the same information as me before I tried to solve this problem” and update your decision-making algorithm accordingly.

[lc]

I thought everybody did this. It seems like the only way to get better at certain things like computer programming. Every time you do something and it takes a while (including realize something), you try and figure out how you could’ve done the cognitive labor a little quicker.

[Said Achmiz]

I’ve gotten better at computer programming (as demonstrated by the fact that I used to not know how to code and now I can code pretty well), and not only have I never done anything that sounds like this, I am not sure I even understand what it would mean to do this. (Is it just “optimize your workflow on a task”? If so, then it seems very mis-decribed. Or is it something else?)

[lc]

1. Do a task that feels like it should have taken 3 hours in 6 hours

2. Think about what mistakes you made (maybe I should have tested this functionality, before attempting to build that entire system out)

3. Turn it into a larger lesson (if cheap, create small test programs instead of writing 2500 new lines and debugging all of them in one pass)

4. Apply the larger lesson going forward

[Said Achmiz]

I am not sure what you mean by step #1 (when something “feels like it should” take some amount of time, but ends up taking more time, it’s generally not because I made some mistake, but rather because my initial “feeling” turned out to be mistaken about how much time the task “should” take—which is not shocking, as such “feelings” are necessarily probabilistic).

The rest of it seems like… learning from mistakes and optimizing your practices/​workflows/​etc. based on experience. Is that what you’re talking about?

I confess that I’m still confused about how any of this could be described as “how could I have thought that faster”. Eliezer writes about “retrain[ing] [himself] to perform only those steps over the course of 30 seconds”, and… that just does not seem like it has anything to do with what you’re describing? Am I missing some analogy here, or what?

[ChristianKl]

How often do you do this per week?

[Malentropic Gizmo]

I also thought that it was very common. I would say it’s necessary for competition math.

[Jacob G-W]

Not everybody does this. Another way to get better is just to do it a lot. It might not be as efficient, but it does work.

[jacquesthibs]

Self-plug, but I think this is similar to the kind of reflection process I tried to describe in “Kolb’s: an approach to consciously get better at anything”.

[Nicholas Kross]

This sure does update me towards “Yudkowsky still wasn’t good enough at pedagogy to have made ‘teach people rationality techniques’ an ‘adequately-covered thing by the community’”.

[quetzal_rainbow]

Do you mean “If EY was good enough we would knew this trick many years ago”?

[Nicholas Kross]

That’s technically a different update from the one I’m making. However, I also update in favor of that, as a propagation of the initial update. (Assuming you mean “good enough” as “good enough at pedagogy”.)

[Blacknsilver]

Isn’t this a normal thing all humans do? “What did I intend, what actually happened, where can I Improve?” along with a quick cost-benefit analysis.

[quetzal_rainbow]

I think difference between what you are describing and what is meant here is captured in this comment:

There’s a phenomenon where a gambler places their money on 32, and then the roulette wheel comes up 23, and they say “I’m such a fool; I should have bet 23”.

More useful would be to say “I’m such a fool; I should have noticed that the EV of this gamble is negative.” Now at least you aren’t asking for magic lottery powers.

Even more useful would be to say “I’m such a fool; I had three chances to notice that this bet was bad: when my partner was trying to explain EV to me; when I snuck out of the house and ignored a sense of guilt; and when I suppressed a qualm right before placing the bet. I should have paid attention in at least one of those cases and internalized the arguments about negative EV, before gambling my money.” Now at least you aren’t asking for magic cognitive powers.

[Sherrinford]

Thanks for giving a useful example.

For most people I guess it would be better to delete the phrase “I’m such a fool” from the evaluation, in order to avoid self-blame that becomes a self-image.

[XelaP]

“Do you constantly look back and ask “How could I have thought that faster?”

I frequently ask myself that. The problem for me is: a lot of the time I either don’t know, or are too lazy to think harder about figuring it out. As an example, it’s very useful to do this after figuring out (or looking at the answer after failing to figure out) how to prove a math theorem, both by making you faster next time and by having you understand the proof to a deeper level so that you can compress it.

However, often the feeling of reward of understanding happens at a lower level than what I should be shooting for, and so I’ll fundamentally feel like it’s not worth it to think harder. Compounding on this is the fact that before you come up with an idea, it often feels like there’s no way you’ll be able to and you’re not making any progress (especially since sometimes there may not be some clever way to think about the proof that lets you understand and reproduce the whole thing easily.

I am trying to adjust my expectation of the chance that I’ll come up with something if I try. However, that alone might not be motivating enough—it feels like looking for a jewel that you’re not even sure is in this cave instead of going on to the next new problem.

Even if I fixed my laziness issue there, I’d still have the problem of often not knowing how to change my thinking to be faster next time.

[Emrik]

Made a post with my reply:

While obviously both heuristics are good to use, the reasons I think asking “which chains-of-thought was that faster than?” tends to be more epistemically profitable than “how could I have thought that faster?” include:

• It is easier to find suboptimal thinking-habits to propagate an unusually good idea into, than to find good ideas for improving a particular suboptimal thinking-habit.

  • Notice that in my technique, the good ideais cognitively proximal and the suboptimal thinking-habits are cognitively distal, whereas in Eliezer’s suggestion it’s the other way around.

  • A premise here is that good ideas are unusual (hard-to-find) and suboptimal thinking-habits are common (easy-to-find)—the advice flips in domains where it’s the opposite.

  • It relates to the difference between propagating specific solutions to plausible problem-domains, vs searching for specific solutions to a specific problem.

    • The brain tends to be biased against the former approach because it’s preparatory work with upfront cost (“prophylaxis”), whereas the latter context sort of forces you to search for solutions.

[Review Bot]

The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

[deleted]