Why should the smell yield to the argument instead of vice versa?
That’s the $64,000(-a-year) question, and I don’t have an answer I’m happy with for the general case. Here’s roughly what I think for this specific situation.
As you say, my nose can’t describe what it smells. It might be a genuine problem or a false alarm. To find out which, I have to consciously poke around for an overlooked counterargument or a weak spot in the original argument, something to corroborate the bad smell. I did that here and couldn’t find a killer gap in 80,000 Hours’ arguments, nor a strong counterargument for why I should disregard them.
For simplicity, consider a stripped down version of the decision problem where I have only two options: becoming a rich banker vs. getting a normal job paying the median salary.
Suppose I disapprove of the banker option for whatever reason. If I hold my nose and become a banker anyway, it seems very likely to me that (1) I would nonetheless prefer that to having someone with different values in my place instead, and (2) that even if taking a banking job worked against my values or goals, I could compensate for that by hiring other people to further them.
I had thought that reference class forecasting might warn against the get-rich-and-give strategy: people with more income give a smaller percentage of it to charity, so by entering banking one might opt into a less generous reference class. But quick Googling reveals that people with higher incomes give more in absolute terms, at least in the UK, the US, and Canada.
Putting aside the chance of my being wrong, what about the disutility of being wrong? Well, I agree with your final paragraph, so that doesn’t seem to weigh heavily against the 80,000 Hours point of view either.
All in all my nose seems to have overreacted on this one. Maybe it raised the alarm because 80,000 Hours’ conclusion failed a quick universalizability test, namely “would this still be the best choice if everyone else in the same boat made it too?” But that test itself seems to fail here.
I doubt my thoughts on this are bulletproof; there’s a good chance I’m missing part of the puzzle or just plain wrong on some fundamental issue. Maybe I’ve built a convenient, clever meta-argument for arguing myself into something I’d probably want to do anyway! Still, ultimately I can devote only so much thought (and self-distrust) to this. I have to make a judgement call, and this is the best one I can make, whatever the risk of motivated cognition.
That’s the $64,000(-a-year) question, and I don’t have an answer I’m happy with for the general case. Here’s roughly what I think for this specific situation.
As you say, my nose can’t describe what it smells. It might be a genuine problem or a false alarm. To find out which, I have to consciously poke around for an overlooked counterargument or a weak spot in the original argument, something to corroborate the bad smell. I did that here and couldn’t find a killer gap in 80,000 Hours’ arguments, nor a strong counterargument for why I should disregard them.
For simplicity, consider a stripped down version of the decision problem where I have only two options: becoming a rich banker vs. getting a normal job paying the median salary.
Suppose I disapprove of the banker option for whatever reason. If I hold my nose and become a banker anyway, it seems very likely to me that (1) I would nonetheless prefer that to having someone with different values in my place instead, and (2) that even if taking a banking job worked against my values or goals, I could compensate for that by hiring other people to further them.
I had thought that reference class forecasting might warn against the get-rich-and-give strategy: people with more income give a smaller percentage of it to charity, so by entering banking one might opt into a less generous reference class. But quick Googling reveals that people with higher incomes give more in absolute terms, at least in the UK, the US, and Canada.
Putting aside the chance of my being wrong, what about the disutility of being wrong? Well, I agree with your final paragraph, so that doesn’t seem to weigh heavily against the 80,000 Hours point of view either.
All in all my nose seems to have overreacted on this one. Maybe it raised the alarm because 80,000 Hours’ conclusion failed a quick universalizability test, namely “would this still be the best choice if everyone else in the same boat made it too?” But that test itself seems to fail here.
I doubt my thoughts on this are bulletproof; there’s a good chance I’m missing part of the puzzle or just plain wrong on some fundamental issue. Maybe I’ve built a convenient, clever meta-argument for arguing myself into something I’d probably want to do anyway! Still, ultimately I can devote only so much thought (and self-distrust) to this. I have to make a judgement call, and this is the best one I can make, whatever the risk of motivated cognition.