Here is how I think we should approach the topic of meditation/Buddhism in the rationalist community. The short version is that a meaningful “yes” requires a credible possibility of “no”, and the long version is that:
If we post scientific studies showing that “meditation works”, then we should either also post scientific studies showing that “meditation doesn’t work” or explicitly mention their absence. Otherwise there is a possibility that simply by doing a lot of studies about any topic, 5% of them will confirm the hypothesis at “p<0.05”. In other words, is there a meta-review on meditation research? (Then we should ask Scott Alexander to review it.)
There are many different claims made about the effects of meditation. I find it quite plausible that some of them may be true (e.g. “meditation helps you relax”) and some others may be false (e.g. “meditation helps you remember your previous reincarnations”). So instead of talking about proving “meditation” we should talk about proving specific claims about meditation.
Actually, we should first make the list of “claims usually made about meditation” and then evaluate each of them individually. Otherwise, if we mention the claims that are supported by evidence, but keep silent about those that are not, it creates a biased overall picture, and contributes to a halo effect. (It is easier to assume that X is supported by evidence if all you know is that A, B, C are supported; compared to a situation where you know that A, B, C are supported, but D, E, F are not.)
The problem with anecdotal evidence about meditation is that we would get it even in a universe where meditation helps 1⁄3 of the population, does nothing for another 1⁄3, and actively harms the remaining 1⁄3. The people who get no or harmful results would simply stop doing it, the people who get useful results would continue… and one or two of them would happen to be high-status in the rationalist community.
Generally, how do you distinguish between “meditation only works for some people, or only in some situations” and “you are doing meditation wrong / not enough”?
What about the anecdotal evidence in the opposite direction, such as sex scandals of famous experts on meditation? (In context of Buddhism, “sex scandals” is not just a bad behavior, but specifically the kind of behavior that meditation is supposed to prevent. So I am not mentioning it here as a moral judgment, but as an evidence that the claims of effects of meditation are falsified by the very people who spent huge amounts of time meditating presumably the right way.)
If you find scientific support for some Buddhist dogma, consider the possibility that you could also find scientific support for its opposite, if you approached it with the same degree of charity. For example, if the teaching of “no self” makes you say “yes, mind is composed of agents which are not themselves minds”, maybe a teaching of “all self” would make you say “yes, neurons are all over the human body, not just in brain; also our mood is influenced by gut bacteria and sunshine and talking to other humans”. Similarly, if the teaching of “impermanence” reminds you of changing moods, growing up, effects of sickness, etc., maybe a teaching of “permanence” would remind you of the stability and heredity of the OCEAN traits. So maybe the actual lesson is not “Buddhism is correct about so many things” but “for a sufficiently general statement one can always find a charitable interpretation”.
Especially if you keep silent about those Buddhist teachings where there is no charitable interpretation that would appeal to the rationalist community. (Such as Buddha doing miracles, using the superpowers he got as a result of meditation.)
How to be skeptical about meditation/Buddhism
Here is how I think we should approach the topic of meditation/Buddhism in the rationalist community. The short version is that a meaningful “yes” requires a credible possibility of “no”, and the long version is that:
If we post scientific studies showing that “meditation works”, then we should either also post scientific studies showing that “meditation doesn’t work” or explicitly mention their absence. Otherwise there is a possibility that simply by doing a lot of studies about any topic, 5% of them will confirm the hypothesis at “p<0.05”. In other words, is there a meta-review on meditation research? (Then we should ask Scott Alexander to review it.)
There are many different claims made about the effects of meditation. I find it quite plausible that some of them may be true (e.g. “meditation helps you relax”) and some others may be false (e.g. “meditation helps you remember your previous reincarnations”). So instead of talking about proving “meditation” we should talk about proving specific claims about meditation.
Actually, we should first make the list of “claims usually made about meditation” and then evaluate each of them individually. Otherwise, if we mention the claims that are supported by evidence, but keep silent about those that are not, it creates a biased overall picture, and contributes to a halo effect. (It is easier to assume that X is supported by evidence if all you know is that A, B, C are supported; compared to a situation where you know that A, B, C are supported, but D, E, F are not.)
The problem with anecdotal evidence about meditation is that we would get it even in a universe where meditation helps 1⁄3 of the population, does nothing for another 1⁄3, and actively harms the remaining 1⁄3. The people who get no or harmful results would simply stop doing it, the people who get useful results would continue… and one or two of them would happen to be high-status in the rationalist community.
Generally, how do you distinguish between “meditation only works for some people, or only in some situations” and “you are doing meditation wrong / not enough”?
What about the anecdotal evidence in the opposite direction, such as sex scandals of famous experts on meditation? (In context of Buddhism, “sex scandals” is not just a bad behavior, but specifically the kind of behavior that meditation is supposed to prevent. So I am not mentioning it here as a moral judgment, but as an evidence that the claims of effects of meditation are falsified by the very people who spent huge amounts of time meditating presumably the right way.)
If you find scientific support for some Buddhist dogma, consider the possibility that you could also find scientific support for its opposite, if you approached it with the same degree of charity. For example, if the teaching of “no self” makes you say “yes, mind is composed of agents which are not themselves minds”, maybe a teaching of “all self” would make you say “yes, neurons are all over the human body, not just in brain; also our mood is influenced by gut bacteria and sunshine and talking to other humans”. Similarly, if the teaching of “impermanence” reminds you of changing moods, growing up, effects of sickness, etc., maybe a teaching of “permanence” would remind you of the stability and heredity of the OCEAN traits. So maybe the actual lesson is not “Buddhism is correct about so many things” but “for a sufficiently general statement one can always find a charitable interpretation”.
Especially if you keep silent about those Buddhist teachings where there is no charitable interpretation that would appeal to the rationalist community. (Such as Buddha doing miracles, using the superpowers he got as a result of meditation.)