Under-acknowledged Value Differences
I’ve been reading a lot of the recent LW discussions on politics and gender, and noticed that people rarely bring up or explicitly acknowledge that different people affected by some political or gender issue have different values/preferences, and therefore solving the problem involves a strong element of bargaining and is not just a matter of straightforward optimization. Instead, we tend to talk as if there is some way to solve the problem that’s best for everyone, and that rational discussion will bring us closer to finding that one best solution.
For example, when discussing gender-related problems, one solution may be generally better for men, while another solution may be generally better for women. If people are selfish, then they will each prefer the solution that’s individually best for them, even if they can agree on all of the facts. (It’s unclear whether people should be selfish, but it seems best to assume that most are, for practical purposes.)
Unfortunately, in bargaining situations, epistemic rationality is not necessarily instrumentally rational. In general, convincing others of a falsehood can be useful for moving the negotiated outcome closer to one’s own preferences and away from others’, and this may be done more easily if one honestly believes the falsehood. (One of these falsehoods may be, for example, “My preferred solution is best for everyone.”) Given these (subconsciously or evolutionarily processed) incentives, it seems reasonable to think that the more solving a problem resembles bargaining, the more likely we are to be epistemicaly irrationality when thinking and talking about it.
If we do not acknowledge and keep in mind that we are in a bargaining situation, then we are less likely to detect such failures of epistemic rationality, especially in ourselves. We’re also less likely to see that there’s an element of Prisoner’s Dilemma in participating in such debates: your effort to convince people to adopt your preferred solution is costly (in time and in your and LW’s overall sanity level) but may achieve little because someone else is making an opposite argument. Both of you may be better off if neither engaged in the debate.