In my model, one should be deeply skeptical whenever the answer to ‘what would do the most good?’ is ‘get people like me more money and/or access to power.’ One should be only somewhat less skeptical when the answer is ‘make there be more people like me’ or ‘build and fund a community of people like me.’ [...] I wish I had a better way to communicate what I find so deeply wrong here
I’d be very curious to hear more fleshed-out arguments here, if you or others think of them. My best guess about what you have in mind is that it’s a combination of the following (lumping all the interventions mentioned in the quoted excerpt into “power-seeking”):
People have personal incentives and tribalistic motivations to pursue power for their in-group, so we’re heavily biased toward overestimating its altruistic value.
Seeking power occupies resources and attention that could be spent figuring out how to solve problems, and figuring out how to solve problems is very valuable.
Figuring out how to solve problems isn’t just very valuable. It’s necessary for things to go well, so mainly doing power-seeking makes it way too easy for us to get the mistaken impression that we’re making progress and things are going well, while a crucial input into things going well (knowing what to do with power) remains absent.
Power-seeking attracts leeches (which wastes resources and dilutes relevant fields).
Power-seeking pushes people’s attention away from object-level discussion and learning. (This is different from (3) in that (3) is about how power-seeking impacts a specific belief, while this point is about attention.)
Power-seeking makes a culture increasingly value power for its own sake, which is bad for the usual reasons that value drift is bad.
If that’s it (is it?), then I’m more sympathetic than I was before writing out the above, but I’m still skeptical:
Re: 1: Speaking of object-level arguments, object-level arguments for the usefulness of power and field growth seem very compelling (and simple enough to significantly reduce room for bias).
4 mainly seems like a problem with poorly executed power-seeking (although maybe that’s hard to avoid?).
2-5 and 6 seem to be horrific problems mostly just if power-seeking is the main activity of a community, rather than one of several activities.
(One view from which power-seeking seems much less valuable is if we assume that, on the margin, this kind of power isn’t all that useful for solving key problems. But if that were the crux, I’d have expected the original criticism to emphasize the (limited) benefits of power-seeking, rather than its costs.)
I agree with and appreciate the broad point. I’ll pick on one detail because I think it matters.
This seems uncharitable? Singer’s thought experiment may have had the above effects, but my impression’s been that it was calculated largely to help people recognize our impartially altruistic parts—parts of us that in practice seem to get hammered down, obliterated, and forgotten far more often than our self-focused parts (consider e.g. how many people do approximately nothing for strangers vs. how many people do approximately nothing for themselves).
So part of me worries that “the drowning child thought experiment is a calculated assault on your personal integrity!” is not just mistaken but yet another hammer by which people will kick down their own altruistic parts—the parts of us that protect those who are small and injured and unable to speak in their own defense.