One way of looking at biases is that the bias is a heuristic with its own selection criteria. For example, people decide who to trust with authority based on how tall they are. The tall-bias is a heuristic with its own selection criteria (tallness) that doesn’t perfectly match what its’ supposed to be optimizing for (trustworthiness).
You might predict that people would take steps to create the appearance of tallness in order to manipulate this form of selection. Hillary Clinton apparently requested that her podium for a debate with Donald Trump be modified so that both candidates appeared to be the same height relative to the podium when standing in front of them, and for a step stool so that she’d appear to be the same height as DT when they stood behind the podium.
One way of looking at the rationality project is that our social systems have optimized themselves to exploit common biases in the human mind. That intersection will feel “normal.” Pointing out these biases isn’t just about moving from less truth → more truth. It’s also about moving from more commonly exploited heuristics → less commonly exploited heuristics. It may be that the new heuristics also have serious failure modes. But if society isn’t set up to systematically take advantage of them, being divergent might still be beneficial, even if it’s not fundamentally any more secure. It’s sort of like choosing an operating system or program that is obscure, because there’s less attention devoted to hacking it.
If you know how the selection incentive works for a particular situation, you can exploit it to your benefit, or at least prevent yourself from being in an unfavorable position.
One way of looking at biases is that the bias is a heuristic with its own selection criteria. For example, people decide who to trust with authority based on how tall they are. The tall-bias is a heuristic with its own selection criteria (tallness) that doesn’t perfectly match what its’ supposed to be optimizing for (trustworthiness).
You might predict that people would take steps to create the appearance of tallness in order to manipulate this form of selection. Hillary Clinton apparently requested that her podium for a debate with Donald Trump be modified so that both candidates appeared to be the same height relative to the podium when standing in front of them, and for a step stool so that she’d appear to be the same height as DT when they stood behind the podium.
One way of looking at the rationality project is that our social systems have optimized themselves to exploit common biases in the human mind. That intersection will feel “normal.” Pointing out these biases isn’t just about moving from less truth → more truth. It’s also about moving from more commonly exploited heuristics → less commonly exploited heuristics. It may be that the new heuristics also have serious failure modes. But if society isn’t set up to systematically take advantage of them, being divergent might still be beneficial, even if it’s not fundamentally any more secure. It’s sort of like choosing an operating system or program that is obscure, because there’s less attention devoted to hacking it.
If you know how the selection incentive works for a particular situation, you can exploit it to your benefit, or at least prevent yourself from being in an unfavorable position.