Twelve Virtues of Rationality
The first virtue is curiosity. A burning itch to know is higher than a solemn vow to pursue truth. To feel the burning itch of curiosity requires both that you be ignorant, and that you desire to relinquish your ignorance. If in your heart you believe you already know, or if in your heart you do not wish to know, then your questioning will be purposeless and your skills without direction. Curiosity seeks to annihilate itself; there is no curiosity that does not want an answer. The glory of glorious mystery is to be solved, after which it ceases to be mystery. Be wary of those who speak of being open-minded and modestly confess their ignorance. There is a time to confess your ignorance and a time to relinquish your ignorance.
The second virtue is relinquishment. P. C. Hodgell said: “That which can be destroyed by the truth should be.”1 Do not flinch from experiences that might destroy your beliefs. The thought you cannot think controls you more than thoughts you speak aloud. Submit yourself to ordeals and test yourself in fire. Relinquish the emotion which rests upon a mistaken belief, and seek to feel fully that emotion which fits the facts. If the iron approaches your face, and you believe it is hot, and it is cool, the Way opposes your fear. If the iron approaches your face, and you believe it is cool, and it is hot, the Way opposes your calm. Evaluate your beliefs first and then arrive at your emotions. Let yourself say: “If the iron is hot, I desire to believe it is hot, and if it is cool, I desire to believe it is cool.” Beware lest you become attached to beliefs you may not want.
The third virtue is lightness. Let the winds of evidence blow you about as though you are a leaf, with no direction of your own. Beware lest you fight a rearguard retreat against the evidence, grudgingly conceding each foot of ground only when forced, feeling cheated. Surrender to the truth as quickly as you can. Do this the instant you realize what you are resisting, the instant you can see from which quarter the winds of evidence are blowing against you. Be faithless to your cause and betray it to a stronger enemy. If you regard evidence as a constraint and seek to free yourself, you sell yourself into the chains of your whims. For you cannot make a true map of a city by sitting in your bedroom with your eyes shut and drawing lines upon paper according to impulse. You must walk through the city and draw lines on paper that correspond to what you see. If, seeing the city unclearly, you think that you can shift a line just a little to the right, just a little to the left, according to your caprice, this is just the same mistake.
The fourth virtue is evenness. One who wishes to believe says, “Does the evidence permit me to believe?” One who wishes to disbelieve asks, “Does the evidence force me to believe?” Beware lest you place huge burdens of proof only on propositions you dislike, and then defend yourself by saying: “But it is good to be skeptical.” If you attend only to favorable evidence, picking and choosing from your gathered data, then the more data you gather, the less you know. If you are selective about which arguments you inspect for flaws, or how hard you inspect for flaws, then every flaw you learn how to detect makes you that much stupider. If you first write at the bottom of a sheet of paper “And therefore, the sky is green!” it does not matter what arguments you write above it afterward; the conclusion is already written, and it is already correct or already wrong. To be clever in argument is not rationality but rationalization. Intelligence, to be useful, must be used for something other than defeating itself. Listen to hypotheses as they plead their cases before you, but remember that you are not a hypothesis; you are the judge. Therefore do not seek to argue for one side or another, for if you knew your destination, you would already be there.
The fifth virtue is argument. Those who wish to fail must first prevent their friends from helping them. Those who smile wisely and say “I will not argue” remove themselves from help and withdraw from the communal effort. In argument strive for exact honesty, for the sake of others and also yourself: the part of yourself that distorts what you say to others also distorts your own thoughts. Do not believe you do others a favor if you accept their arguments; the favor is to you. Do not think that fairness to all sides means balancing yourself evenly between positions; truth is not handed out in equal portions before the start of a debate. You cannot move forward on factual questions by fighting with fists or insults. Seek a test that lets reality judge between you.
The sixth virtue is empiricism. The roots of knowledge are in observation and its fruit is prediction. What tree grows without roots? What tree nourishes us without fruit? If a tree falls in a forest and no one hears it, does it make a sound? One says, “Yes it does, for it makes vibrations in the air.” Another says, “No it does not, for there is no auditory processing in any brain.” Though they argue, one saying “Yes,” and one saying “No,” the two do not anticipate any different experience of the forest. Do not ask which beliefs to profess, but which experiences to anticipate. Always know which difference of experience you argue about. Do not let the argument wander and become about something else, such as someone’s virtue as a rationalist. Jerry Cleaver said: “What does you in is not failure to apply some high-level, intricate, complicated technique. It’s overlooking the basics. Not keeping your eye on the ball.”2 Do not be blinded by words. When words are subtracted, anticipation remains.
The seventh virtue is simplicity. Antoine de Saint-Exupéry said: “Perfection is achieved not when there is nothing left to add, but when there is nothing left to take away.”3 Simplicity is virtuous in belief, design, planning, and justification. When you profess a huge belief with many details, each additional detail is another chance for the belief to be wrong. Each specification adds to your burden; if you can lighten your burden you must do so. There is no straw that lacks the power to break your back. Of artifacts it is said: The most reliable gear is the one that is designed out of the machine. Of plans: A tangled web breaks. A chain of a thousand links will arrive at a correct conclusion if every step is correct, but if one step is wrong it may carry you anywhere. In mathematics a mountain of good deeds cannot atone for a single sin. Therefore, be careful on every step.
The eighth virtue is humility. To be humble is to take specific actions in anticipation of your own errors. To confess your fallibility and then do nothing about it is not humble; it is boasting of your modesty. Who are most humble? Those who most skillfully prepare for the deepest and most catastrophic errors in their own beliefs and plans. Because this world contains many whose grasp of rationality is abysmal, beginning students of rationality win arguments and acquire an exaggerated view of their own abilities. But it is useless to be superior: Life is not graded on a curve. The best physicist in ancient Greece could not calculate the path of a falling apple. There is no guarantee that adequacy is possible given your hardest effort; therefore spare no thought for whether others are doing worse. If you compare yourself to others you will not see the biases that all humans share. To be human is to make ten thousand errors. No one in this world achieves perfection.
The ninth virtue is perfectionism. The more errors you correct in yourself, the more you notice. As your mind becomes more silent, you hear more noise. When you notice an error in yourself, this signals your readiness to seek advancement to the next level. If you tolerate the error rather than correcting it, you will not advance to the next level and you will not gain the skill to notice new errors. In every art, if you do not seek perfection you will halt before taking your first steps. If perfection is impossible that is no excuse for not trying. Hold yourself to the highest standard you can imagine, and look for one still higher. Do not be content with the answer that is almost right; seek one that is exactly right.
The tenth virtue is precision. One comes and says: The quantity is between 1 and 100. Another says: The quantity is between 40 and 50. If the quantity is 42 they are both correct, but the second prediction was more useful and exposed itself to a stricter test. What is true of one apple may not be true of another apple; thus more can be said about a single apple than about all the apples in the world. The narrowest statements slice deepest, the cutting edge of the blade. As with the map, so too with the art of mapmaking: The Way is a precise Art. Do not walk to the truth, but dance. On each and every step of that dance your foot comes down in exactly the right spot. Each piece of evidence shifts your beliefs by exactly the right amount, neither more nor less. What is exactly the right amount? To calculate this you must study probability theory. Even if you cannot do the math, knowing that the math exists tells you that the dance step is precise and has no room in it for your whims.
The eleventh virtue is scholarship. Study many sciences and absorb their power as your own. Each field that you consume makes you larger. If you swallow enough sciences the gaps between them will diminish and your knowledge will become a unified whole. If you are gluttonous you will become vaster than mountains. It is especially important to eat math and science which impinge upon rationality: evolutionary psychology, heuristics and biases, social psychology, probability theory, decision theory. But these cannot be the only fields you study. The Art must have a purpose other than itself, or it collapses into infinite recursion.
Before these eleven virtues is a virtue which is nameless. Miyamoto Musashi wrote, in The Book of Five Rings:4
The primary thing when you take a sword in your hands is your intention to cut the enemy, whatever the means. Whenever you parry, hit, spring, strike or touch the enemy’s cutting sword, you must cut the enemy in the same movement. It is essential to attain this. If you think only of hitting, springing, striking or touching the enemy, you will not be able actually to cut him. More than anything, you must be thinking of carrying your movement through to cutting him.
Every step of your reasoning must cut through to the correct answer in the same movement. More than anything, you must think of carrying your map through to reflecting the territory.
If you fail to achieve a correct answer, it is futile to protest that you acted with propriety.
How can you improve your conception of rationality? Not by saying to yourself, “It is my duty to be rational.” By this you only enshrine your mistaken conception. Perhaps your conception of rationality is that it is rational to believe the words of the Great Teacher, and the Great Teacher says, “The sky is green,” and you look up at the sky and see blue. If you think, “It may look like the sky is blue, but rationality is to believe the words of the Great Teacher,” you lose a chance to discover your mistake.
Do not ask whether it is “the Way” to do this or that. Ask whether the sky is blue or green. If you speak overmuch of the Way you will not attain it.
You may try to name the highest principle with names such as “the map that reflects the territory” or “experience of success and failure” or “Bayesian decision theory.” But perhaps you describe incorrectly the nameless virtue. How will you discover your mistake? Not by comparing your description to itself, but by comparing it to that which you did not name.
If for many years you practice the techniques and submit yourself to strict constraints, it may be that you will glimpse the center. Then you will see how all techniques are one technique, and you will move correctly without feeling constrained. Musashi wrote: “When you appreciate the power of nature, knowing the rhythm of any situation, you will be able to hit the enemy naturally and strike naturally. All this is the Way of the Void.”
These then are twelve virtues of rationality:
Curiosity, relinquishment, lightness, evenness, argument, empiricism, simplicity, humility, perfectionism, precision, scholarship, and the void.
- Elements of Rationalist Discourse by 12 Feb 2023 7:58 UTC; 179 points) (
- Creating a truly formidable Art by 14 Oct 2021 4:39 UTC; 117 points) (
- A summary of every “Highlights from the Sequences” post by 15 Jul 2022 23:01 UTC; 93 points) (
- Bayesian Mindset by 21 Dec 2021 19:54 UTC; 73 points) (EA Forum;
- On sincerity by 23 Dec 2022 17:13 UTC; 69 points) (
- Curating “The Epistemic Sequences” (list v.0.1) by 23 Jul 2022 22:17 UTC; 65 points) (
- The Berkeley Community & The Rest Of Us: A Response to Zvi & Benquo by 20 May 2018 7:19 UTC; 56 points) (
- Introducing School of Thinking by 17 Nov 2022 13:03 UTC; 55 points) (EA Forum;
- Quantum Non-Realism by 8 May 2008 5:27 UTC; 49 points) (
- A summary of every “Highlights from the Sequences” post by 15 Jul 2022 23:05 UTC; 47 points) (EA Forum;
- The Practice & Virtue of Discernment by 26 May 2021 0:34 UTC; 41 points) (
- “Infohazard” is a predominantly conflict-theoretic concept by 2 Dec 2021 17:54 UTC; 39 points) (
- On sincerity by 23 Dec 2022 17:14 UTC; 38 points) (EA Forum;
- (Summary) Sequence Highlights—Thinking Better on Purpose by 2 Aug 2022 17:45 UTC; 33 points) (
- EA Forum Prize: Winners for February 2021 by 27 Apr 2021 9:32 UTC; 32 points) (EA Forum;
- What are the counterarguments to a Faustian Vaccine Hypothesis? ($2k charity bounty) by 25 Jan 2022 10:33 UTC; 31 points) (
- 12 Apr 2021 2:44 UTC; 30 points)'s comment on People Will Listen by (
- An unofficial “Highlights from the Sequences” tier list by 5 Sep 2022 14:07 UTC; 29 points) (
- Reflections on Zen and the Art of Motorcycle Maintenance by 26 Feb 2023 20:46 UTC; 27 points) (
- Trust and The Small World Fallacy by 4 Oct 2021 0:38 UTC; 26 points) (
- Some of the best rationality essays by 19 Oct 2021 22:57 UTC; 25 points) (
- Zen and Rationality: Trust in Mind by 11 Aug 2020 20:23 UTC; 25 points) (
- Zen and Rationality: Don’t Know Mind by 6 Aug 2020 4:33 UTC; 25 points) (
- 27 Jan 2023 12:06 UTC; 22 points)'s comment on Basics of Rationalist Discourse by (
- The Archetypal Rational and Post-Rational by 15 Apr 2020 9:40 UTC; 21 points) (
- 13 Mar 2021 0:40 UTC; 18 points)'s comment on How can we stop talking past each other when it comes to postrationality? by (
- Notes on Humility by 29 Nov 2020 19:50 UTC; 18 points) (
- Good and bad ways to think about downside risks by 11 Jun 2020 1:38 UTC; 17 points) (
- Notes on Rationality by 16 Jan 2022 19:05 UTC; 16 points) (
- Training Regime Day 21: Executing Intentions by 25 Apr 2020 22:16 UTC; 13 points) (
- 1 Jan 2020 17:27 UTC; 13 points)'s comment on Circling as Cousin to Rationality by (
- 31 Mar 2021 18:16 UTC; 13 points)'s comment on Rationalism before the Sequences by (
- Godshatter Versus Legibility: A Fundamentally Different Approach To AI Alignment by 9 Apr 2022 21:43 UTC; 12 points) (
- 11 Jun 2019 20:02 UTC; 12 points)'s comment on Naked mole-rats: A case study in biological weirdness by (
- VA Rationalists—Norfolk Meetup by 14 Sep 2022 0:22 UTC; 12 points) (
- Notes on Simplicity by 2 Dec 2020 23:14 UTC; 10 points) (
- 30 Apr 2021 8:09 UTC; 9 points)'s comment on AMA: Paul Christiano, alignment researcher by (
- Sentience, Sapience, Consciousness & Self-Awareness: Defining Complex Terms by 20 Oct 2021 13:48 UTC; 9 points) (
- 1 Mar 2021 7:56 UTC; 6 points)'s comment on RadVac Commercial Antibody Test Results by (
- 20 Oct 2022 3:14 UTC; 6 points)'s comment on Shortform by (
- 16 Jan 2021 0:25 UTC; 5 points)'s comment on Meditations on faith by (
- Implementing the Second Virtue: Feel the Emotions Which Fit the Facts by 9 May 2021 5:47 UTC; 5 points) (
- 20 Mar 2023 18:52 UTC; 5 points)'s comment on Shutting Down the Lightcone Offices by (
- 21 Mar 2021 10:57 UTC; 4 points)'s comment on On LessWrong/Rationality and Political Debates by (
- 24 Sep 2021 22:30 UTC; 4 points)'s comment on Explanations as Hard to Vary Assertions by (
- Norfolk Social—VA Rationalists by 10 Oct 2022 0:09 UTC; 4 points) (
- 26 Jan 2022 11:34 UTC; 3 points)'s comment on Arkhejinn’s Shortform by (
- A Fight is a Faster Tax on Bullshit by 17 Sep 2021 16:26 UTC; 2 points) (
- 10 Mar 2021 19:41 UTC; 2 points)'s comment on Deflationism isn’t the solution to philosophy’s woes by (
- The Twelve Virtues of Rationality by 11 Aug 2022 3:27 UTC; 2 points) (
- 14 Jan 2021 2:58 UTC; 2 points)'s comment on Dishonest Update Reporting by (
- 6 Aug 2022 23:30 UTC; 1 point)'s comment on Qazzquimby Shortform by (
- 14 Mar 2023 18:54 UTC; 1 point)'s comment on LVSN’s Shortform by (
- 21 Jul 2021 9:29 UTC; 1 point)'s comment on One Study, Many Results (Matt Clancy) by (
- 13 Sep 2022 1:35 UTC; 1 point)'s comment on Willa’s Shortform by (
- 2 Feb 2023 23:08 UTC; 0 points)'s comment on [META] ‘Rational’ vs ‘Optimized’ by (
I translated this essay to Hebrew, here it is, enjoy :)
This asks a rhetorical question in passing which has a rather delightful answer:
Everything which photosynthesizes nourishes us with oxygen, whether or not it fruits.
Or, if “nourish” means “provide food or medicine”: Acer saccharum nourishes us with its sap; Cinnamomum verum and Cinnamomum cassia flavor our food with their bark; the roots of Sassafras albidum flavor drinks and its leaves thicken sauces; Cinchona officinalis and Taxus baccata yield essential medicinal compounds from their bark; the flowers of Hibiscus rosa-sinensis make a delicious drink; the leaves of Moringa oleifera are edible and packed with nutrients. Those are only the trees which nourish us directly; anything which flowers gives us honey through bees; many species of tree return atmospheric nitrogen to the soil to enhance the growth of nearby crops.
Formatting error: The end of the first paragraph is at the start of the second paragraph.
A trick to remember: the first letter of each virtue gives (in blocks): CRL EAES HP PSV, which can easily be remembered as “cooperative reinforcement learning, EAs, Harry Potter, PS: The last virtue is the void.”
(Obviously remembering these is pointless, but memorizing lists is a nice way to practice mnemonic technique.)