I remember this post very fondly. I often thought back to it and it inspired some thoughts of my own about rationality (which I had trouble writing down and are waiting in a draft to be written fully some day). I haven’t used any of the phrases introduced here (Underperformance Swamp, Sinkholes of Sneer, Valley of Disintegration...), and I’m not sure whether it was the intention.
The post starts with the claim that rationalists “basically got everything about COVID-19 right and did so months ahead of the majority of government officials, journalists, and supposed experts”. Since it’s not the point of the post I won’t review this claim in depth, but it seems basically true to me. Elizabeth’s review here gives a few examples.
This post is about the difficulty and even danger in becoming a rationalist, or more generally, in using explicit reasoning (Intuition and Social Cognition being the alternatives).
The first difficulty is that explicit reasoning alone often fails to outperform intuition and social cognition where those perform well. I think this is true, and as the rationality community evolved it came to appreciate intuition and social cognition more, without devaluing explicit reason.
The second is persevering through the sneer and social pressure that comes from trying to use explicit reason to do things, often coming to very different approaches from other people, and often also failing.
The third is navigating the strange status hierarchy in the community, which mostly doesn’t depend on regular things like attractiveness and more often on our ability to apply explicit reason effectively, as well as being scared by strange memes like AI risk and cryonics. I don’t know to what extent the first part is true in the physical communities, but it definitely is in the virtual community.
The fourth is where the danger comes in. When you’re in the Valley of Bad Rationality your life can get worse, and if you don’t get out of it some way it might stay worse. So people either try to go back, which may or may not work, or try go through and become even better at explicit reasoning, which may or may not work (The author says it did for him).
The main error this post points at is the failure mode of treating Explicit Reasoning as the only tool in your belt, instead of one of many tools that is also used to improve those tools. And the main difficulty is one of motivation and perseverance.
I think this post is important reading for a young rationalist, as well as for more mature rationalists, so they know better who would fit to learn rationality and who would have trouble with it.
I think the advice the post gives to people already on the path is good:
try again after the early failures, ignore the sneers, find a community with good norms, and don’t let the memes scare you — it all adds up to normalcy in the end. Let reason be the instrument that sharpens your other instruments, not the only tool in your arsenal.
I remember this post very fondly. I often thought back to it and it inspired some thoughts of my own about rationality (which I had trouble writing down and are waiting in a draft to be written fully some day). I haven’t used any of the phrases introduced here (Underperformance Swamp, Sinkholes of Sneer, Valley of Disintegration...), and I’m not sure whether it was the intention.
The post starts with the claim that rationalists “basically got everything about COVID-19 right and did so months ahead of the majority of government officials, journalists, and supposed experts”. Since it’s not the point of the post I won’t review this claim in depth, but it seems basically true to me. Elizabeth’s review here gives a few examples.
This post is about the difficulty and even danger in becoming a rationalist, or more generally, in using explicit reasoning (Intuition and Social Cognition being the alternatives).
The first difficulty is that explicit reasoning alone often fails to outperform intuition and social cognition where those perform well. I think this is true, and as the rationality community evolved it came to appreciate intuition and social cognition more, without devaluing explicit reason.
The second is persevering through the sneer and social pressure that comes from trying to use explicit reason to do things, often coming to very different approaches from other people, and often also failing.
The third is navigating the strange status hierarchy in the community, which mostly doesn’t depend on regular things like attractiveness and more often on our ability to apply explicit reason effectively, as well as being scared by strange memes like AI risk and cryonics. I don’t know to what extent the first part is true in the physical communities, but it definitely is in the virtual community.
The fourth is where the danger comes in. When you’re in the Valley of Bad Rationality your life can get worse, and if you don’t get out of it some way it might stay worse. So people either try to go back, which may or may not work, or try go through and become even better at explicit reasoning, which may or may not work (The author says it did for him).
The main error this post points at is the failure mode of treating Explicit Reasoning as the only tool in your belt, instead of one of many tools that is also used to improve those tools. And the main difficulty is one of motivation and perseverance.
I think this post is important reading for a young rationalist, as well as for more mature rationalists, so they know better who would fit to learn rationality and who would have trouble with it.
I think the advice the post gives to people already on the path is good: