Evolution programmed us to avoid doing things that others perceive as unfairly manipulating. -- Because then the other people get angry and punish the unfair manipulator.
Intelligent people are more aware of their own motives. This makes them more likely to label their hypothetical actions as “manipulation”, because they better see how the action contributes to getting what they want. -- A less reflective person would just do the action contributing to getting what they want, on impulse, and then deny any connection.
a) Intelligent people suffer from the illusion of transparency, just like everyone else. They are likely to believe that if they see the “manipulative” aspects of their hypothetical behaviour, other people would see it to. So they abstain from the behavior, to avoid making others angry. -- But in reality they are miscalibrated; most people would not notice anything unusual.
b) People are adaptation executers, not utility maximizers. Hesitating to manipulate other people unfairly is an adaptation we have, and for the average person, the advantage is beneficial; it allows them to keep good social relationship. For an introspective person, this adaptation may become harmful, because it forbids too wide range of human interaction. -- But the intelligent people follow this adaptation even when it decreases their utility, because people are not utility maximizers.
To me, this explanation seems credible.
Then, of course, I tell myself that rationalists should win, so if my sense of detecting manipulation is miscalibrated, I should calibrate it better, and stop using the “manipulation” label with all its bad connotations too widely.
But many people don’t do this. Because people are not automatically strategic.
(Meta: I use a lot of LW keywords in this comment, to make you—the reader of this comment, and a member of LW community—more likely to agree with me. At the same time, I also do sincerely believe everything I wrote here. At this moment, am I manipulating you or not? Should I modify the comment to remove all the applause lights, and make you less likely to agree, probably even less likely to understand precisely what I mean, just to get rid of the feeling of guilt for manipulating you? My decision is no, but there was a time when I would have decided otherwise.)
How about this:
Evolution programmed us to avoid doing things that others perceive as unfairly manipulating. -- Because then the other people get angry and punish the unfair manipulator.
Intelligent people are more aware of their own motives. This makes them more likely to label their hypothetical actions as “manipulation”, because they better see how the action contributes to getting what they want. -- A less reflective person would just do the action contributing to getting what they want, on impulse, and then deny any connection.
a) Intelligent people suffer from the illusion of transparency, just like everyone else. They are likely to believe that if they see the “manipulative” aspects of their hypothetical behaviour, other people would see it to. So they abstain from the behavior, to avoid making others angry. -- But in reality they are miscalibrated; most people would not notice anything unusual.
b) People are adaptation executers, not utility maximizers. Hesitating to manipulate other people unfairly is an adaptation we have, and for the average person, the advantage is beneficial; it allows them to keep good social relationship. For an introspective person, this adaptation may become harmful, because it forbids too wide range of human interaction. -- But the intelligent people follow this adaptation even when it decreases their utility, because people are not utility maximizers.
To me, this explanation seems credible.
Then, of course, I tell myself that rationalists should win, so if my sense of detecting manipulation is miscalibrated, I should calibrate it better, and stop using the “manipulation” label with all its bad connotations too widely.
But many people don’t do this. Because people are not automatically strategic.
(Meta: I use a lot of LW keywords in this comment, to make you—the reader of this comment, and a member of LW community—more likely to agree with me. At the same time, I also do sincerely believe everything I wrote here. At this moment, am I manipulating you or not? Should I modify the comment to remove all the applause lights, and make you less likely to agree, probably even less likely to understand precisely what I mean, just to get rid of the feeling of guilt for manipulating you? My decision is no, but there was a time when I would have decided otherwise.)