The whole idea of effective altruism is in getting the biggest bang for your charitable buck. If the evidence about how to do this was simple and incontrovertible, we wouldn’t need advanced rationality skills to do so. In the real world, choosing the best cause requires weighing up subtle balances of evidence on everything from if animals are suffering in ways we would care about, to how likely a super intelligent AI is.
On the other side, effective altruism is only persuasive if you have various skills and patterns of thought. These include the ability to think quantitatively, avoiding scope insensitivity, the ideas of expected utility maximization and the rejection of the absurdity heuristic. It is conceptually possible for a mind to be a brilliant rationalist with the sole goal of paperclip maximization, however all humans have the same basic emotional architecture, with emotions like empathy and caring. When this is combined with rigorous structured thought, the end result often looks at least somewhat utilitarianish.
Here are the kinds of thought patterns that a stereotypical rationalist, and a stereotypical non rationalist would engage in, when evaluating two charities. One charity is a donkey sanctuary, the other is trying to genetically modify chickens that don’t feel pain.
The leaflet has a beautiful picture of a cute fluffy donkey in a field of sunshine and flowers. Aww Don’t you just want to stroke him. Donkeys in medows seem an unambiguous pure good. Who could argue with donkeys. Thinking about donkeys makes me feel happy. Look, this one with the brown ears is called buttercup. I’ll put this nice poster up and send them some money.
Genetically modifying? Don’t like the sound of that. To not feel pain? Weird? Why would you want to do that? Imagines the chicken crushed into a tiny cage, looking miserable, “its not really suffering” doesn’t cut it. Wouldn’t that encourage people to abuse them? We should be letting them live in the wild as nature intended.
The main component of this decision comes from adding up the little “good” or “bad” labels that they attach to each word. There is also a sense in which a donkey sanctuary is a typical charity (the robin of birds), while GM chickens is an atypical charity (the ostrich).
The rationalist starts off with questions like “How much do I value a year of happy donkey life, vs a year of happy chicken life?”. How much money is needed to modify chickens, and get them used in farms. Whats the relative utility gain from a “non suffering” chicken in a tiny cage, vs a chicken in chicken paradise, relative to a factory farm chicken that is suffering? What is the size of the world chicken industry?
The rationalist ends up finding that the world chicken industry is huge, and so most sensible values for the other parameters lead to the GM chicken charity being better. They trust utilitarian logic more than any intuitions they might have.
Insofar as your answer makes predictions about how actual “rationalists” behave, it would seem to be at least partly falsified: empirically, it turns out that many rationalists do not respond well to that particular suggestion (“modify chickens to not feel pain”).
(The important thing to note, about the above link, isn’t so much that there are disagreements with the proposal, but that the reasons given for those disagreements are fairly terrible—they are mostly non sequiturs, with a dash of bad logic thrown in. This would seem to more closely resemble the way you describe a “stereotypical non-rationalist” behaving than a “stereotypical rationalist”.)
In our argument in the comments to my post on zetetic explanations, I was a bit worried about pushing back too hard socially. I had a vague sense that there was something real and bad going on that your behavior was a legitimate immune response to, and that even though I thought and continue to think that I was a false positive, it seemed pretty bad to contribute to marginalization of one of the only people visibly upset about some sort of hard-to-put-my-finger-on shoddiness going on. It’s very important to the success of an epistemic community to have people sensing things like this, and promote that sort of alarm.
I’ve continued to try to track this, and I can now see somewhat more clearly a really sketchy pattern, which you’re one of the few people to consistently call out when it happens. This comment is a good example. It seems like there’s a tendency to conflate the stated ambitions and actual behavior of ingroups like Rationalists and EAs, when we wouldn’t extend this courtesy to the outgroup, in a way that subtly shades corrective objections as failures to get with the program.
This kind of thing is insidious, and can be done by well-meaning people. While I still think my zetetic explanation post was a different sort of slackness, there was a time when I’d have written posts like Donald Hobson’s, and I wasn’t intentionally trying to fool anyone. I was just … a certain flavor of enthusiastic and hopeful that gets a pass when and only when it flatters the ingroup’s prejudices.
I think it’s helpful and important for you to continue to point out object-level errors like this one, but it’s also important to track which errors seem like part of a pattern of motivated error, and which seem to be mere mistakes. The former class seems much more dangerous to me, since such errors are correlated.
Thank you for the encouragement, and I’m glad you’ve found value in my commentary.
… it’s also important to track which errors seem like part of a pattern of motivated error, and which seem to be mere mistakes. The former class seems much more dangerous to me, since such errors are correlated.
I agree with this as an object-level policy / approach, but I think not quite for the same reason as yours.
It seems to me that the line between “motivated error” and “mere mistake” is thin, and hard to locate, and possibly not actually existent. We humans are very good at self-deception, after all. Operating on the assumption that something can be identified as clearly being a “mere mistake” (or, conversely, as clearly being a “motivated error”) is dangerous.
That said, I think that there is clearly a spectrum, and I do endorse tracking at least roughly in which region of the spectrum any given case lies, because doing so creates some good incentives (i.e., it avoids disincentivizing post-hoc honesty). On the other hand, it also creates some bad incentives, e.g. the incentive for the sort of self-deception described above. Truthfully, I don’t know what the optimal approach is, here. Constant vigilance against any failures in this whole class is, however, warranted in any case.
I agree that not all rationalists would want wireheaded chickens, maybe they don’t care about chicken suffering at all. I also agree that you sometimes see bad logic and non-sequiters in the rationalist community. The non rationalist, motivated, emotion driven thinking, is the way that humans think by default. The rationalist community is trying to think a different way, sometimes successfully. Illustrating a junior rationalist having an off day and doing something stupid doesn’t illuminate the concept of rationality, the way that seeing a beginner juggler drop balls doesn’t show you what juggling is.
The whole idea of effective altruism is in getting the biggest bang for your charitable buck. If the evidence about how to do this was simple and incontrovertible, we wouldn’t need advanced rationality skills to do so. In the real world, choosing the best cause requires weighing up subtle balances of evidence on everything from if animals are suffering in ways we would care about, to how likely a super intelligent AI is.
On the other side, effective altruism is only persuasive if you have various skills and patterns of thought. These include the ability to think quantitatively, avoiding scope insensitivity, the ideas of expected utility maximization and the rejection of the absurdity heuristic. It is conceptually possible for a mind to be a brilliant rationalist with the sole goal of paperclip maximization, however all humans have the same basic emotional architecture, with emotions like empathy and caring. When this is combined with rigorous structured thought, the end result often looks at least somewhat utilitarianish.
Here are the kinds of thought patterns that a stereotypical rationalist, and a stereotypical non rationalist would engage in, when evaluating two charities. One charity is a donkey sanctuary, the other is trying to genetically modify chickens that don’t feel pain.
The leaflet has a beautiful picture of a cute fluffy donkey in a field of sunshine and flowers. Aww Don’t you just want to stroke him. Donkeys in medows seem an unambiguous pure good. Who could argue with donkeys. Thinking about donkeys makes me feel happy. Look, this one with the brown ears is called buttercup. I’ll put this nice poster up and send them some money.
Genetically modifying? Don’t like the sound of that. To not feel pain? Weird? Why would you want to do that? Imagines the chicken crushed into a tiny cage, looking miserable, “its not really suffering” doesn’t cut it. Wouldn’t that encourage people to abuse them? We should be letting them live in the wild as nature intended.
The main component of this decision comes from adding up the little “good” or “bad” labels that they attach to each word. There is also a sense in which a donkey sanctuary is a typical charity (the robin of birds), while GM chickens is an atypical charity (the ostrich).
The rationalist starts off with questions like “How much do I value a year of happy donkey life, vs a year of happy chicken life?”. How much money is needed to modify chickens, and get them used in farms. Whats the relative utility gain from a “non suffering” chicken in a tiny cage, vs a chicken in chicken paradise, relative to a factory farm chicken that is suffering? What is the size of the world chicken industry?
The rationalist ends up finding that the world chicken industry is huge, and so most sensible values for the other parameters lead to the GM chicken charity being better. They trust utilitarian logic more than any intuitions they might have.
Insofar as your answer makes predictions about how actual “rationalists” behave, it would seem to be at least partly falsified: empirically, it turns out that many rationalists do not respond well to that particular suggestion (“modify chickens to not feel pain”).
(The important thing to note, about the above link, isn’t so much that there are disagreements with the proposal, but that the reasons given for those disagreements are fairly terrible—they are mostly non sequiturs, with a dash of bad logic thrown in. This would seem to more closely resemble the way you describe a “stereotypical non-rationalist” behaving than a “stereotypical rationalist”.)
In our argument in the comments to my post on zetetic explanations, I was a bit worried about pushing back too hard socially. I had a vague sense that there was something real and bad going on that your behavior was a legitimate immune response to, and that even though I thought and continue to think that I was a false positive, it seemed pretty bad to contribute to marginalization of one of the only people visibly upset about some sort of hard-to-put-my-finger-on shoddiness going on. It’s very important to the success of an epistemic community to have people sensing things like this, and promote that sort of alarm.
I’ve continued to try to track this, and I can now see somewhat more clearly a really sketchy pattern, which you’re one of the few people to consistently call out when it happens. This comment is a good example. It seems like there’s a tendency to conflate the stated ambitions and actual behavior of ingroups like Rationalists and EAs, when we wouldn’t extend this courtesy to the outgroup, in a way that subtly shades corrective objections as failures to get with the program.
This kind of thing is insidious, and can be done by well-meaning people. While I still think my zetetic explanation post was a different sort of slackness, there was a time when I’d have written posts like Donald Hobson’s, and I wasn’t intentionally trying to fool anyone. I was just … a certain flavor of enthusiastic and hopeful that gets a pass when and only when it flatters the ingroup’s prejudices.
I think it’s helpful and important for you to continue to point out object-level errors like this one, but it’s also important to track which errors seem like part of a pattern of motivated error, and which seem to be mere mistakes. The former class seems much more dangerous to me, since such errors are correlated.
Thank you for the encouragement, and I’m glad you’ve found value in my commentary.
I agree with this as an object-level policy / approach, but I think not quite for the same reason as yours.
It seems to me that the line between “motivated error” and “mere mistake” is thin, and hard to locate, and possibly not actually existent. We humans are very good at self-deception, after all. Operating on the assumption that something can be identified as clearly being a “mere mistake” (or, conversely, as clearly being a “motivated error”) is dangerous.
That said, I think that there is clearly a spectrum, and I do endorse tracking at least roughly in which region of the spectrum any given case lies, because doing so creates some good incentives (i.e., it avoids disincentivizing post-hoc honesty). On the other hand, it also creates some bad incentives, e.g. the incentive for the sort of self-deception described above. Truthfully, I don’t know what the optimal approach is, here. Constant vigilance against any failures in this whole class is, however, warranted in any case.
I agree that not all rationalists would want wireheaded chickens, maybe they don’t care about chicken suffering at all. I also agree that you sometimes see bad logic and non-sequiters in the rationalist community. The non rationalist, motivated, emotion driven thinking, is the way that humans think by default. The rationalist community is trying to think a different way, sometimes successfully. Illustrating a junior rationalist having an off day and doing something stupid doesn’t illuminate the concept of rationality, the way that seeing a beginner juggler drop balls doesn’t show you what juggling is.
Is this GM chicken example just a thought experiment?
I’ve not seen a charity trying to do it, but wouldn’t be surprised if there was one. I’m trying to illustrate the different thought processes.