What simple rationality techniques give the most bang for the buck? I’m talking about techniques you might be able to explain to a reasonably smart person in five minutes or less: really the basics. If part of the goal here is to raise the sanity waterline in the general populace, not just among scientists, then it would be nice to have some rationality techniques that someone can use without much study.
Carl Sagan had a slogan: “Extraordinary claims require extraordinary evidence.” He would say this phrase and then explain how, when someone claims something extraordinary (i.e. something for which we have a very low probability estimate), they need correspondingly stronger evidence than if they’d made a higher-likelihood claim, like “I had a sandwich for lunch.” Now, I’m sure everybody here can talk about this very precisely, in terms of Bayesian updating and odds ratios, but Sagan was able to get a lot of this across to random laypeople in about a minute. Maybe two minutes.
What techniques for rationality can be explained to a normal person in under five minutes? I’m looking for small and simple memes that will make people more rational, on average. I’ll try a few candidates, to get the discussion started.
Candidate 1: Carl Sagan’s concise explanation of how evidence works, as mentioned above.
Candidate 2: Everything that has an effect in the real world is part of the domain of science (and, more broadly, rationality). A lot of people have the truly bizarre idea that some theories are special, immune to whatever standards of evidence they may apply to any other theory. My favorite example is people who believe that prayers for healing actually make people who are prayed for more likely to recover, but that this cannot be scientifically tested. This is an obvious contradiction: they’re claiming a measurable effect on the world and then pretending that it can’t possibly be measured. I think that if you pointed out a few examples of this kind of special pleading to people, they might start to realize when they’re doing it.
Candidate 3: Admitting that you were wrong is a way of winning an argument. There’s a saying that “It takes a big man to admit he’s wrong,” and when people say this, they don’t seem to realize that it’s a huge problem! It shouldn’t be hard to admit that you were wrong about something! It shouldn’t feel like defeat; it should feel like victory. When you lose an argument with someone, it should be time for high fives and mutual jubilation, not shame and anger. I know that it’s possible to retrain yourself to feel this way, because I’ve done it. This wasn’t even too difficult; it was more a matter of just realizing that feeling good about conceding an argument was even an option.
Anti-candidate: “Just because something feels good doesn’t make it true.” I call this an anti-candidate because, while it’s true, it’s seldom helpful. People trot out this line as an argument against other people’s ideas, but rarely apply it to their own. I want memes that will make people actually be more rational, instead of just feeling that way.
Any ideas? I know that the main goal of this community is to strive for rationality far beyond such low-hanging fruit, but if we can come up with simple and easy techniques that actually help people be more rational, there’s a lot of value in that. You could use it as rationalist propaganda, or something.
I think some of the statistical fallacies that most people fall for are quite high up the list.
One such is the “What a coincidence!” fallacy. People notice that some unlikely event has occurred, and wonder how many millions to one against this event must have been—and yet it actually happenned ! Surely this means that my life is influenced by some supernatural influence!
The typical mistake is to simply calculate the likelihood of the occurrence of the particular event that occurred. Nothing wrong with that, but one should also compare that number against the whole basket of other possible unlikely events that you would have noticed if they’d happenned (of which there are surely millions), and all the possible occasions where all these unlikely events could have also occurred. When you do that, you discover that the likelihood of some unlikely thing happenning is quite high—which is in accordance with our experience that unlikely events do actually happen.
Another way of looking at it is that non-notable unlikely events happen all the time. Look, that particular car just passed me at exactly 2pm ! Most are not noticable. But sometimes we notice that a particular unlikely event just occurred, and of course it causes us to sit up and take notice. The question is how many other unlikely events you would also have noticed.
The key rational skill here is noticing the actual size of the set of unlikely things that might have happenned, and would have caught our attention if they had.
I’m going to be running a series of Rationality & AI seminars with Alex Flint in the Autumn, where we’ll introduce aspiring rationalists to new concepts in both fields; standard cognitive biases, a bit of Bayesianism, some of the basic problems with both AI and Friendliness. As such, this could be a very helpful thread.
We were thinking of introducing Overconfidence Bias; ask people to give 90% confidence intervals, and then reveal (surprise surprise!) that they’re wrong half the time.
The concept of inferential distance is good. You wouldn’t want to introduce it in the context of explaining something complicated—you’d just sound self-serving—but it’d be a good thing to crack out when people complain about how they just can’t understand how anyone could believe $CLAIM.
Edit: It’s also a useful concept when you are thinking about teaching.
That’s a good lesson to internalize, but how do you get someone to internalize it? How do you explain it (in five minutes or less) in such a way that someone can actually use it?
I’m not saying that there’s no easy way to explain it; I just don’t know what that way would be. When I argue with someone who acts like their intuitions are magic, I usually go back to basic epistemology: define concisely what it means to be right about whatever we’re discussing, and show that their intuitions here aren’t magic. If there’s a simple way to explain in general that intuition isn’t magic, I’d really love to hear it. Any ideas?
Given that we haven’t constructed a decent AI, and don’t know how those intuitions actually work, we only really believe they’re not magic on the grounds that we don’t believe in magic generally, and don’t see any reason why intuitions should be an exception to the rule that all things can be explained.
Perhaps an easier lesson is that intuitions can sometimes be wrong, and it’s useful to know when that happens so we can correct for it. For example, most people are intuitively much more afraid of dying in dramatic and unusual ways (like air crashes or psychotic killers) than in more mundane ways like driving the car or eating unhealthy foods, Once it’s established that intuitions are sometimes wrong, the fact that we don’t exactly know how they work isn’t so dangerous to one’s thinking.
Well, I thought Kaj_Sotana’s explanation was good, but the five-minute constraint makes things very difficult. I tend to be so long-winded that I’m not sure I could get across any insight in five minutes, honestly, but you’re right that “Your intuitions are not magic” is likely to be harder than many.
What simple rationality techniques give the most bang for the buck? I’m talking about techniques you might be able to explain to a reasonably smart person in five minutes or less: really the basics. If part of the goal here is to raise the sanity waterline in the general populace, not just among scientists, then it would be nice to have some rationality techniques that someone can use without much study.
Carl Sagan had a slogan: “Extraordinary claims require extraordinary evidence.” He would say this phrase and then explain how, when someone claims something extraordinary (i.e. something for which we have a very low probability estimate), they need correspondingly stronger evidence than if they’d made a higher-likelihood claim, like “I had a sandwich for lunch.” Now, I’m sure everybody here can talk about this very precisely, in terms of Bayesian updating and odds ratios, but Sagan was able to get a lot of this across to random laypeople in about a minute. Maybe two minutes.
What techniques for rationality can be explained to a normal person in under five minutes? I’m looking for small and simple memes that will make people more rational, on average. I’ll try a few candidates, to get the discussion started.
Candidate 1: Carl Sagan’s concise explanation of how evidence works, as mentioned above.
Candidate 2: Everything that has an effect in the real world is part of the domain of science (and, more broadly, rationality). A lot of people have the truly bizarre idea that some theories are special, immune to whatever standards of evidence they may apply to any other theory. My favorite example is people who believe that prayers for healing actually make people who are prayed for more likely to recover, but that this cannot be scientifically tested. This is an obvious contradiction: they’re claiming a measurable effect on the world and then pretending that it can’t possibly be measured. I think that if you pointed out a few examples of this kind of special pleading to people, they might start to realize when they’re doing it.
Candidate 3: Admitting that you were wrong is a way of winning an argument. There’s a saying that “It takes a big man to admit he’s wrong,” and when people say this, they don’t seem to realize that it’s a huge problem! It shouldn’t be hard to admit that you were wrong about something! It shouldn’t feel like defeat; it should feel like victory. When you lose an argument with someone, it should be time for high fives and mutual jubilation, not shame and anger. I know that it’s possible to retrain yourself to feel this way, because I’ve done it. This wasn’t even too difficult; it was more a matter of just realizing that feeling good about conceding an argument was even an option.
Anti-candidate: “Just because something feels good doesn’t make it true.” I call this an anti-candidate because, while it’s true, it’s seldom helpful. People trot out this line as an argument against other people’s ideas, but rarely apply it to their own. I want memes that will make people actually be more rational, instead of just feeling that way.
Any ideas? I know that the main goal of this community is to strive for rationality far beyond such low-hanging fruit, but if we can come up with simple and easy techniques that actually help people be more rational, there’s a lot of value in that. You could use it as rationalist propaganda, or something.
EDIT: I’ve expanded this into a top-level post.
I think some of the statistical fallacies that most people fall for are quite high up the list.
One such is the “What a coincidence!” fallacy. People notice that some unlikely event has occurred, and wonder how many millions to one against this event must have been—and yet it actually happenned ! Surely this means that my life is influenced by some supernatural influence!
The typical mistake is to simply calculate the likelihood of the occurrence of the particular event that occurred. Nothing wrong with that, but one should also compare that number against the whole basket of other possible unlikely events that you would have noticed if they’d happenned (of which there are surely millions), and all the possible occasions where all these unlikely events could have also occurred. When you do that, you discover that the likelihood of some unlikely thing happenning is quite high—which is in accordance with our experience that unlikely events do actually happen.
Another way of looking at it is that non-notable unlikely events happen all the time. Look, that particular car just passed me at exactly 2pm ! Most are not noticable. But sometimes we notice that a particular unlikely event just occurred, and of course it causes us to sit up and take notice. The question is how many other unlikely events you would also have noticed.
The key rational skill here is noticing the actual size of the set of unlikely things that might have happenned, and would have caught our attention if they had.
I’m going to be running a series of Rationality & AI seminars with Alex Flint in the Autumn, where we’ll introduce aspiring rationalists to new concepts in both fields; standard cognitive biases, a bit of Bayesianism, some of the basic problems with both AI and Friendliness. As such, this could be a very helpful thread.
We were thinking of introducing Overconfidence Bias; ask people to give 90% confidence intervals, and then reveal (surprise surprise!) that they’re wrong half the time.
Since it seemed like this could be helpful, I expanded this into a top-level post.
That 90% confidence interval thing sounds like one hell of a dirty trick. A good one, though.
The concept of inferential distance is good. You wouldn’t want to introduce it in the context of explaining something complicated—you’d just sound self-serving—but it’d be a good thing to crack out when people complain about how they just can’t understand how anyone could believe $CLAIM.
Edit: It’s also a useful concept when you are thinking about teaching.
#3 is a favorite of mine, but I like #1 too.
How about “Your intuitions are not magic”? Granting intuitions the force of authority seems to be a common failure mode of philosophy.
That’s a good lesson to internalize, but how do you get someone to internalize it? How do you explain it (in five minutes or less) in such a way that someone can actually use it?
I’m not saying that there’s no easy way to explain it; I just don’t know what that way would be. When I argue with someone who acts like their intuitions are magic, I usually go back to basic epistemology: define concisely what it means to be right about whatever we’re discussing, and show that their intuitions here aren’t magic. If there’s a simple way to explain in general that intuition isn’t magic, I’d really love to hear it. Any ideas?
Given that we haven’t constructed a decent AI, and don’t know how those intuitions actually work, we only really believe they’re not magic on the grounds that we don’t believe in magic generally, and don’t see any reason why intuitions should be an exception to the rule that all things can be explained.
Perhaps an easier lesson is that intuitions can sometimes be wrong, and it’s useful to know when that happens so we can correct for it. For example, most people are intuitively much more afraid of dying in dramatic and unusual ways (like air crashes or psychotic killers) than in more mundane ways like driving the car or eating unhealthy foods, Once it’s established that intuitions are sometimes wrong, the fact that we don’t exactly know how they work isn’t so dangerous to one’s thinking.
Well, I thought Kaj_Sotana’s explanation was good, but the five-minute constraint makes things very difficult. I tend to be so long-winded that I’m not sure I could get across any insight in five minutes, honestly, but you’re right that “Your intuitions are not magic” is likely to be harder than many.