The Dark Arts—Preamble
I’d like to tell you all a story.
Once upon a time I was working for a charity – a major charity – going door-to-door to raise money while pretending it wasn’t sales.
This story happened on my last day working there. I didn’t know that at the time; I wouldn’t find out until the following morning when my boss called me up to fire me, but I knew it was coming. For weeks I’d been fed up with the job, milking it for the last few dollars I could pull out, hating every minute of it but needing the money. The Sudden Career Readjustment would come as a relief.
So on that day, my last day, I was moving slowly. I knocked on one particular door and there was no response. I had little desire to walk to the next one, however, and there was an interesting spider who’d built its web below the doorbell. I tapped its belly with the tip of my pen, and it reacted with aggression – trying to envenom and ensnare the tip of my ballpoint. I must have been playing with it for a good minute or so when the door suddenly opened.
A distraught woman stood before me. After a brief period of Relating I launched into my pitch.“So you’re probably wondering why there’s a bald weirdo at your door? Actually I’m just coming around with Major Charity1 on an emergency campaign. You’ve heard of us, right? Brilliant! So obviously you’ve thought of getting involved, right? That’s awesome! You see, the reason I’m coming around is for these guys – some of our emergency cases...”2
I handed her the pictures of the Developing World Children (yeah, it was one of those charities). She took them, a wistful look on her face.
“Oh God, don’t show me these. I’m such a Rescuer.”
“Rescuer? Do you have a Rescue Dog?” [Where I’m from, abused animals brought into a new home are called ‘Rescue Dogs’.]
“No, I...”
“You mean your personality? You care about people, don’t you?”
She nodded slowly. Her face began to crumble.
“I’m sorry – I can’t look at these children,” she handed back the photographs, “Not right now. I’ve been crying all day and I just can’t deal with those emotions...”
I took back the children, a look of honest sympathy on my face. The Demon Wheel began spinning. I could see that she was on the verge of crying again. My gut told me that her father had recently died, but the actual cause didn’t matter. I could discover that information. The upcoming dialogue played itself out in my mind...
“Oh jeez, what happened? Oh my god, seriously..?” Head tilted as an Alpha confidant enough for Beta behaviour, looking down and shaking, “I’m lucky enough to have never been through that. Were the two of you close?” As she talks I nod, prompting her until she breaks out in tears. I put down my binder and step into her house, embracing her as she cries on my shoulder.
She sniffles.
“I’m sorry… sorry to do this to you.”
“No, don’t be. Listen… Mary, is it? What you’re going through is normal. It’s nothing to be ashamed of…” Cue personal anecdote, then pause for a beat. “Listen, about the Major Charity thing; this is something you’ve always wanted to do, isn’t it? Yeah, I can tell. You’re a caring person, after all. I tell you what: we’ll get you set up with this little boy – he’s from Ecuador, and we’re trying to get him eating a healthy diet. We’re going to make you his super hero today. And then you’ll know – Mary, you’ll know that even at your darkest moment, you still have the strength in you to save a life.
“And you know what else?” I reach out to touch her arm, “Tonight you’re going to sleep like a baby knowing that you did this. So you go and get your Credit Card and I’ll start filling out the form.”
* * *
I could have done it. I could have got that child sponsored. I could have kept my job, and Mary could have stopped crying that evening. She’d have thanked me for coming by, and after I left she would have cuddled on the couch with her new Sponsor Child, tears drying as she found hope in the world.
But I didn’t do it. Instead I apologized for interrupting her grief, and left.
Because I am not a Meat Fucker.
* * *
All my life, I’ve had this bad habit. No matter how hard I try and kick it, there it is: Honesty. I can’t tell you how many times it’s dug me into a hole. As far as concepts go, it’s about as foolish and utopian as Truth and Justice, and I know that, but I just can’t seem to let it go. That’s a large part of the reason I left Mary alone to her tears – backed off, rather than digging into her psyche to recalibrate a few clusters of neuron.
The other half is my status as a card-carrying (union-dues-paid-in-full) Anarchist. The way I look at things, the only time you can justify using the Jedi Mind Trick on somebody is when your ethics would stand clean with murdering them as well.
Sending Storm Troopers on a Wild Droid Chase is one thing; scamming Waddo out of a distributor cap for your CGI Space Plane is another.
When you take advantage of the Dark Arts, you’re not simply tricking people into giving you what you want; you’re making them want to give it to you. You’re hacking into their brain and inserting a Murder Pill; afterwards they will literally thank you for doing so (the only sponsor I ever met who wasn’t glad that I’d come by was the lady whose 6 year old daughter I primed into wanting it). In ninety percent of the situations where the Dark Arts are useful or possible, you can’t do it out of spite; when you realign someone’s desires to match your own they want to do what you want them to do.
And yet there’s no clear distinction between using these skills and regular social interaction. Manipulation works best when you’re sincere about it. Ethically speaking it’s a grey, wavy line.
The thing is, we all like to be Sold, Led, Dominated; if I walk into Subway, and I ask the kid at the counter to give me his Best Submarine Sandwich, I want him to tell me what I want, and make me love it after it’s paid for. The last thing he should do is say that “They’re all good!” and make me regret the [(5 breads)x(16 meats)x(212 Toppings)-1] subs that I didn’t get.3 Retail is the Dark Arts Done Right (usually). The Sales Lady figures out what I want, uses her expertise to find the best fit, and then kills the cognitive dissonance that could ruin my enjoyment of the product; “You really pull off that colour. Seriously, that jacket looks great on you – you see how these lines naturally compliment your shoulders? Of course you can!”
Sexual dynamics are similar; if somebody’s drinking in public at 2 in the morning it’s because they’re on the market. Let’s say a ‘faithful wife’ goes to the club one weekend while her husband is out of town, and she has a few drinks with a bunch of college boys she just met. One of them happens to be a PUA. When it comes to things like date rape drugs, or taking advantage of a person who’s sloppy-drunk there is a clear line in the sand. But in this hypothetical the woman’s relatively sober. It’s just that the young rake is so damned charming!
Meanwhile her husband’s having a few pints at the hotel bar with Sheila from accounting, and she just keeps making eyes at him…
Neither Sheila nor the PUA is responsible for the ensuing infidelity. If the husband and wife didn’t want it in the first place, they would have never availed themselves to the temptation. If, on the other hand, you meet somebody at a Neighbourhood Watch meeting, and spend the next three months seducing them… that’s when you’ve got to start questioning your ethics. Anybody is going to be vulnerable at some time or another.
While the Dark Arts are a Power, it’s how you use them that matters, like any other tool. I’m running mind-games on people, but I usually won’t; I’m also good at fighting, but I don’t assault people for no reason. I find both concepts repulsive.
That’s the end of my moralizing on the matter. The upcoming series is going to be purely descriptive in nature, exploring different strategies for manipulating others. I’ll provide tactical examples showing how these strategies can be put into play, but for the most part each battlefield is unique; these are broader methods that apply across the board. What you do with these techniques is up to you.
As for defence… I don’t think I’ll have much to say about that. When done properly, the victim doesn’t realize it until it’s already over, and by then it doesn’t matter. You’re aware that the AI manipulated you into opening the box, but you’re going to open it anyways because that’s your new utility function. It’s like a game of Roshambo, or when you’re thinking about joining Facebook: the only way to win is not to play.
Endnotes
1. Major Charity’s methods of acquiring funding don’t have any bearing on whether or not it’s an effective charity. Whether or not the money going overseas actually makes a difference is a question I cannot answer.
2. The repetition here is intentional. I was trying to prime key concepts.
3. My theory as to what is going on with these sub places and their myriad of options: the target is not a new customers, those people are going to be intimidated by all the choices, and the restaurants know that. Rather, it is to provide ‘fresh’ options so that their current customers don’t get bored and go elsewhere.
- 's comment on The ethics of breaking belief by (9 May 2012 5:44 UTC; 7 points)
- 's comment on post proposal: Attraction and Seduction for Heterosexual Male Rationalists by (6 Feb 2011 18:25 UTC; 5 points)
- 's comment on Rationality Quotes: March 2011 by (3 Mar 2011 4:14 UTC; 5 points)
- 's comment on Open Thread, February 1-14, 2012 by (7 Feb 2012 12:10 UTC; 4 points)
- 's comment on [Question] What’s your Elevator Pitch For Rationality? by (8 Sep 2011 12:56 UTC; 3 points)
- 's comment on So You’ve Changed Your Mind by (30 Apr 2011 17:19 UTC; 3 points)
Translation: you fell prey to a different Dark meme, one that linked your behavior to a perception of status, which you then pursued at the expense of real utility for at least two real people besides yourself.
In other words, your vaunted “honesty” actually equals nothing more than selfish egoism: you prefer to pride yourself on it more than you prefer actually helping people.
That is, the rhetoric around honesty you’re using is nothing more than the creed of the Confessor (“the ultimate sin is the exercise of command”), which is equally countered and mirrored by that of the Kiritsugu (who would see you as flawed for being “one who refuses to help”).
It’s not obvious that either of these is a truly correct stance, vs. just being different from each other. Neither posture absolves you of responsibility for the consequences of your actions or inactions—or even limits the possibility of bad consequences occurring.
(btw, on an entirely unrelated note, it’s Watto, not Waddo.)
I’m not at all sure that the question of “real utility” is so vastly different from his perception of “status”, as you call it.
In the one scenario, he pursues “real utility” for at least two people—but he also helps promote the idea that intruding on someone’s most intense moments of grief in such a way is an acceptable practice. And though he may use such intrusion for the greater good, others will be less benevolent in their intentions.
So how does the world look like if everyone followed the example of the “meat-fucker”, the emotional-manipulator? Every similar cause will have people looking at the newspapers for obituaries, so that they can specifically target people at their most emotionally vulnerable moments. At their most emotionally vulnerable moments, people won’t be thinking clearly. And therefore a majority of less than ideal choices get encouraged. Plus, you know, the extra burden of grief-stricken people having to deal with manipulators.
Does this bigger scenario have positive or negative utility for you? Aurini clearly wants a world with fewer “meat-fuckers”. So would I. That’s utility too.
As far as I can tell, your entire argument consists of playing reference-class tennis with the categorical imperative. I’m not inclined to join you.
[To be less obscure: you have made arbitrary (and unnecessarily broad) choices of what constitutes the “idea” or “example” being “promoted” or “encouraged”, without providing any supporting reasons for your choice, vs. other possible choices of what to abstract from the specific example. That’s reference-class tennis. You’re also assuming that the categorical imperative is the reference standard for ethics, vs. say, strict consequentialism or some sort of deontic ethics.]
This is a revealing story about the double-binds in influence and persuasion. To me, your hypothetical form of influence probably would have been “Dark Arts,” but not just for the reasons you describe.
Actually, I’d prefer to taboo the term “Dark Arts” for a while, because I feel it’s poisoned the well on the subject of influence and persuasion (like the word “manipulation”). The problem with the term “Dark Arts” is that it conflates both ethical and unethical forms of influence, and creates an “Ugh Field” around the subject of influence in general. Let’s talk about how influence and persuasion work, and then later we will decide what’s ethical and what’s unethical.
The irony of a post criticizing the Dark Arts is that this post is full of rhetoric and persuasion itself, whether intentionally or unintentionally. I’d like to examine some of the language you use in this post (“Dark Arts”, “Jedi Mind Tricks”, “hacking”, “Murder Pill”). In general, I consider these forms of emotive language about influence and persuasion to inhibit understanding these topics.
Yes, changing what people want is an important component of influence and persuasion. But it is not inherently unethical, so I don’t think this entire category of behavior deserves the epithet “Dark Arts.”
I think the “influence as brain-hacking” analogy is revealing in some ways, but confusing in others. People engaging in social influence rarely have powers over others comparable to root access to a computer. Perhaps a better analogy is someone giving you an account in their brain with limited privileges.
Brains have firewalls. Unless the brain is in an altered state where its defenses are disabled, there is only so much that an intruder has access to.
There is no such thing as Jedi Mind Tricks.
In the Gandhi thought experiment, the Murder Pill changes someone’s higher order preferences. Gandhi know that the Murder Pill will make him want to start killing people, so he refuses to take the Pill.
Yet most examples of influence probably act on a much lower order of preferences. If you influence someone to buy your product, or go out with you, I guess you are changing their preference to engage in those actions with you in particular, from “no” to “yes.” But you are really influencing their preferences about what behavior they engage in, rather than their preferences about what values they hold, unlike the Murder Pill scenario.
At such a behavioral level of taking a specific action, people’s default preference is generally “no”, because it has to be. People can’t buy every product in the world or go out with every person. For them to want to buy something or go out with someone, their default “no” must change to “yes” at some point (except, perhaps, for “love at first sight” situations). If you succeed in influencing someone to change their mind in this way, you haven’t actually changed someone’s fundamental values; rather, you have shown them that you are offering something that satisfies their values.
That doesn’t sound like a Murder Pill, unless by “Murder Pill” you mean anything that might change what someone wants to do in any way.
Another important part of the Gandhi thought experiment is that the preference-changing pill is offered to him, and he is given a choice. The thought experiment is to show that if the pill is so counter to his own preferences, he won’t choose it. That’s pretty similar to what happened in your actual case: she refused the method of influence you were attempting.
You are quite correct that it’s difficult to distinguish various supposedly controversial forms of influence from regular social interaction, like pickup.
Yes. But the interesting thing about sexuality is that it’s a practically universal want. You can start with pretty high priors that someone is interested in sex and relationships than you can for assuming that they like to donate to charities.
I don’t think the time frame makes a difference. If a married person chooses to expose themselves to people who want to seduce them, and allow that kind of interaction to occur, then it doesn’t matter whether it’s for one night, or over a period of months. If anything, the more time the seduction takes, the more ethical the seducer is being, because of removing the argument that the married person was just acting out of impulse. I don’t buy the idea of people having consensual sex with someone they normally don’t want merely out of eventual vulnerability.
In this post, you recount an example of hypothetical influence which would probably be unethical, and you also acknowledge that other forms of influence are ethical. If you frame this in terms of the “Dark Arts,” then you indeed end up with the somewhat contradictory conclusion that the “Dark Arts” are a tool that can be used in ethical ways. But if it can be used in ethical ways, we are we calling it Dark? If we translate back to neutral terminology and say that social influence is a tool that can be used for good or evil, that makes more sense.
So what distinguishes ethical influence from unethical influence? I think that’s the secret ingredient that’s missing from your analysis.
That ingredient is consent to being influenced. In your story, the other person attempts to withdraw consent. (“Oh God, don’t show me these. I’m such a Rescuer” … “I’m sorry – I can’t look at these children,” she handed back the photographs, “Not right now. I’ve been crying all day and I just can’t deal with those emotions...”). She clearly didn’t want to take your influence pill, and you didn’t force it on her. Even though she wasn’t in a state where she could be assertive enough to slam the door in your face, she was clearly attempting to withdraw consent.
There have been some other arguments in the comments that comforting her and soliciting the donation would have been a win-win. I think that would be the case of she was merely distraught and not also protesting your influence. But the combination of those two things should have caused too much doubt that proceeding was the right thing to do.
Influence is powerful yes, but brains aren’t so malleable and defenseless as a lot of the metaphors in this post would suggest. That means that if you are influencing someone, you should have a higher prior that they are letting you. For the specific example of your story, you were actually dealing with someone who’s mental defenses were down, and I think it’s correct that you held back from using your full arsenal of influence… even though such influence could be ethical with someone who wasn’t emotionally distraught.
How do we evaluate evidence that someone is consenting to influence? What evidence for consent is required for different levels of influence? That’s a subject for another time.
There was a step somewhere in here I missed. You’re clear that these tactics violate your morals, and you specifically do not allege that learning about them will help us avoid them (and nor will anything else). What’s your purpose in teaching them?
Clarification: if you walk into the leather shop, with $300 burning a hole in your pocket, and see a nice jacket—and the guy behind the counter knows what he’s doing (and gives enough of a shit) you will walk out of their, happy with your new jacket.
Saying there’s no defense whatsoever was a bit of an exagerration; it’s true that some people are more resistant than others (though not infinitely resistant), but even if you’re weak you can always avoid the situations in the first place.
Having a blanket policy of “I don’t do anything at the door” is highly effective.
It’s sort of like a bullet; you can’t resist it, but you can avoid it [assuming for the sake of analogy you ignore kevlar].
You know what? I will write a piece on defence, once I’m done with the other pieces in the series. Hopefully what I was trying to convey will be clearer at that point.
As for my purpose − 1) it’s fun; this is stuff I’ve thought a lot about, and the community here is smart enough to understand it, 2) I get LW points and ego validation for writing it, and 3) the information’s already out there, I figure it might as well be in the hands of the Good Guys.
Are you assuming you walked into the leather shop with the intention to buy something? Or does walking in with a friend/partner who is shopping there count, but you wanted to spend that money on something else?
Do you have any evidence to back up these success rates on people who actually don’t want to buy the thing (prior to manipulation)? I am highly skeptical that people are on average so easy to manipulate. I am not skeptical that there are many people who are marginal on whether or not to buy a jacket, and hence easy to talk into it. However, you seem to be assuming a level of irresistibility to manipulation that does not correlate with my normal experience. The world is full of manipulative sales pitches—and most of the people resist them most of the time, otherwise most people couldn’t afford to eat.
I’m not denying there is a large contingent that is so vulnerable, merely claiming they are less than 50% of the population. If I had to put a confidence interval on it, I’d say the percentage was 1% − 30%, with the higher percentages assuming they are segregated into socioeconomic groups that I’m unlikely to encounter in my daily life.
To address both these points, “[amount of money] burning a hole in your pocket” is a colloquial phrase which indicates an intent to spend the money.
It seems like there might be some kind of strategic direct defense. The possible victim could identify remembered triggers in (for example) a compliance situation, and then have a policy about how to act after such observations. You don’t have to disengage; you could decide that you want to continue with the process. The extra level of reflection simply gives you more control.
Cialdini advocates this kind of meta-strategy in Influence.
I hate these tactics, but want to read what Aurini has to say, because I enjoy knowledge about such things and you never can tell when it will turn out useful. For example, “Games People Play” helps me recognize and escape bad situations.
This is a followup to the thread you linked, but I’ll mention it here because it may be relevant to the current conversation.
I spent a little while with TheValliant (offsite) trying to pin down a definition of “manipulation”—specifically, to divide it from acceptable forms of nonverbal communication. The facets which came to mind immediately were “trying to get someone to do something they otherwise wouldn’t have,” which is clearly too broad, and “displaying exaggerated emotion,” which is sometimes valid (when you’re trying to communicate an emotion and your natural expression is too subtle). We got from there to “trying to make someone feel an emotion they otherwise wouldn’t have,” which still isn’t right, because it covers gestures of affection.
What we eventually settled on was this: “Using emotion to bypass someone’s normal decision-making process.” That is, creating emotions in someone else for the purpose of getting them to do something. This phrasing also makes it pretty clear why we find it abhorrent: it’s opening a back door into someone else’s brain, and about as invasive as that makes it sound.
The reason I felt a need to pin it down is that TheValliant, like you, has a policy of not tolerating it under any circumstances, and it seemed to me that that required understanding what it was. So now I’m curious—does the above definition match the thing you hate?
Here’s my understanding of manipulation.
Physical coercion forces you to do something that you don’t want to do, don’t enjoy while you’re doing it, and regret doing afterwards.
Emotional manipulation causes you to do something that you didn’t want to do before, and regret afterwards—but you may like it in the meantime.
For example: violent rape causes you to have sex against your will. You don’t want to have sex ahead of time, you don’t want to have sex while you’re being raped, and you aren’t glad you were raped afterwards.
Manipulating someone into sex means that she didn’t want to have sex before, and she regretted it afterwards, but you got her to want to have sex while she was doing it. It’s not strictly speaking coercion, but you did get her to do something that’s out of character and not in keeping with her usual desires.
The third option is “try it, you’ll like it.” The person didn’t intend to take the action before, but she wanted to do it at the moment she acted, and she was glad she did it afterwards. I don’t see a moral problem with this. It’s influence, but it’s not harmful. To continue the sex example, if the woman’s initial impression is negative, but the man gets her to want to have sex and afterwards she’s glad she did then he’s just good at attracting women, not a harmful manipulator.
Influencing someone to take an action that you know she will regret afterwards is manipulative.
To illustrate another example where the “avoid buyer’s remorse” principle is overbroad (which may or may not be the principle you are advocating), let’s talk about cookies.
You’re having a dinner party, and everyone is stuffed. You bring out some freshly baked cookies for dessert. Guest: “Oh no, I’m stuffed.” You: “They are warm and soft!” Guest: “Well, that does sound good, I’ll have just one.” Guest eats a cookie. Later, Guest looks a bit queasy and is obviously regretting eating so much.
This is another case that falls afoul of the “avoid creating buyer’s remorse” principle, but doesn’t deserve such a negative term as “manipulation.”
The fact is that a lot of people enjoy taking actions that they may later regret. It’s not a moral requirement to protect people from themselves. As long as they are making the decision with free will (or the closest thing to it that humans have), and informed consent exists, then it’s valid for people to take responsibility for the risks of their behavior. We should assume that people can assess their best interests, unless we have reasons to believe otherwise (for instance, if you know that Guest is on a strict diet that they committed to, offering them a cookie would be unethical). But in the absence of such information, you should assume that Guest knows what’s best for them.
There isn’t a perfect duty to protect Guest from himself, but of course empathy is a different subject. You might decide as a matter of empathy that you don’t want to risk Guest being queasy from dessert, even if Guest would be willing to take that risk.
The problem with this principle is it judges the ethics of taking an action in the present, based on the consequences in the future. That’s a similar temporal error to Roissy who said it was OK to slap his girlfriend because she turned out to be massively turned on by it and hump him. You can’t know whether someone will regret sex with you, just as Roissy couldn’t have known that his girlfriend would like being slapped without any prior discussion of consent.
The intuition I would like to extract from your quote is that whether someone regrets sex is potential evidence that the person who initiated the sex did so unethically. This is true, but it’s weak evidence: there are all sorts of other things that cause people to regret sex other than miscreancy by their partner, such as conservative parents, physical discomfort during sex, judgmental/jealous friends, or reflecting on STD/pregnancy risk.
On the other hand, the fact that someone gives consent to something is evidence that they believe that they won’t regret it (or that it is worth a try even if they might). As long as that consent is sufficiently informed, given with a sound mind, not forced, and not coerced by some unethical threat of punishment, that’s good enough… right? To create stricter perfect duties gets into infantilization territory. However, there are a lot of extra measures that are good for people to take, even though we can’t actually require them all the time:
If someone gives enthusiastic consent, that’s extra strong evidence that they have shouldered the risk of buyer’s remorse. Attaining a standard of enthusiastic consent is what I personally aspire to in seduction; I don’t want to just be the tasty cookie that someone regrets eating later. It’s probably not a perfect duty that people absolutely must go for (if someone who is lukewarm about sex with you decides to go for it with you, you don’t have an absolute obligation to stop it), but it’s an imperfect duty that’s a good thing to attain if you can manage it.
The primary responsibility of avoiding buyer’s remorse must fall on the buyer in typical situations that involve informed consent; this is still true in situations where someone’s mind has changed during a relatively short period of time, because people do change their minds.
If the moral principle of protecting others from buyer’s remorse were followed as a perfect duty, it would destroy a lot of the types of spontaneous purchases and sex that many people enjoy (including, but not limited to guilty pleasures); this fact needs to be weighed against the need to protect people from the types of purchases and sex that they might regret.
Making people scrutinize their decisions more to avoid the possibility of buyer’s remorse is not necessarily more ethical; you can probably psych someone out of taking any action with you that they might enjoy, if you try hard enough. At some point we need to step back and let adults make decisions for themselves. And yes, adults can still make valid decisions under the influence of sales hype or sexual arousal; I think the folk concept of “free will” is good enough here. Badgering people with “are you sure you don’t want to?” is the other side of the same coin of badgering people with “are you sure you actually want to?”; both of them fail to accept an adult’s stated preference.
If someone knows that they are vulnerable to influence leading them to do things they will regret, then perhaps they should take measures to avoid situations where they will encounter such influence, instead of walking into them, getting tempted, and then blaming the other person. For instance, if you are trying to diet and find cookies hard to resist, the burden shouldn’t be on cookie-offerering friends to prove that you aren’t on a diet before they can ethically offer you cookies; such a principle would simply destroy the practice for offering cookies to friends for everyone who gives a crap about the ethics. The solution is to tell people that you’re on a diet and that they shouldn’t offer you anything; you have a comparative advantage at making this information known. They should respect your wishes until you give them reason to believe that you’ve changed your stance.
Overbroad moral standards are actually very dangerous, because they will cause the scrupulous people who follow them to fail and get selected out for no good reasons. Furthermore, shifting all the moral calculations to one party in social transactions is both infantilizing and unfair (exception: certain transactions with massive informational asymmetry, like financial services). Even worse, the results are corrupt, because when the moral calculations and burden of proof become impractical (e.g. having to prove that friends aren’t on diets before offering them cookies), people have an incentive to cut corners on their moral calculations and be biased, otherwise they will get outcompeted by people who ignore the morals.
In my next reply, I will propose an alternative demarcation criterion between ethical and unethical social influence in the area of sex.
What if you also “manipulate” so that it will also not be regretted afterwards?
In marketing at least, there is the concept of customer retention through anticipating and countering buyers’ remorse… which mostly, AFAICT, consists of providing a customer with arguments to use to explain to co-workers, friends, relatives, spouses, or whomever why their purchase decision was a good one. This strongly implies that at least in the purchasing arena, the main reason people come to have buyers’ remorse (besides crappy products) is that the purchase makes them look bad in the eyes of others.
Hence the marketing adage that the primary function of facts and logic in a sales pitch is to provide the customer with a rationale that lets them prove to themselves and others that they made the right decision… but only after they’ve been convinced to make that decision based on emotion.
I believe there’s some discussion in the PUA field of similar “buyer’s remorse” issues and providing the same sort of supporting rationale, except that such rationales are more to allow those women who are uncomfortable with casual sex to rationalize that it e.g. “just happened” (i.e., neither she nor the PUA truly intended it, so she is neither a slut nor a victim) or that it falls into the category of “once-in-a-lifetime encounter”, or something else that can be reconciled with her belief system or social image in a non-harmful way.
In either case, a wise “salesperson” takes care to find a way to allow a person to reconcile their actions with their self-perception of “character” or their “usual self”.
Incidentally, my personal model of an “ethical PUA” is Founding Father Gouverneur Morris, who told his lover, “I never lost respect for those who consented to make me happy on the principles of affection.”
Isn’t that kind of a low standard? I’m glad he doesn’t do slut-shaming, but I’m curious about what the women involved think about his effect on their lives.
I expect he’s said other things you’ve liked, but what you quoted is actually that unusual, it might enough by itself to explain why women are apt to be nervous around men.
If it helps, he lived in the 18th century—he was one of the framers of the U.S. Constitution.
Thanks—I didn’t bother to google, and your phrasing left it unclear about whether he was a Founding Father of the US or PUA.
Still, how common do you think it is for men to see women as lowering their own status by consenting to sex?
Still pretty common, unfortunately.
But I wouldn’t single out a contemporary man for praise just because he didn’t slut-shame. It’s more remarkable 250 years ago.
Depending on the social group somewhat less common than women lowering the status of others via judgement on that criteria—for most part it is women who are the more direct rivals with other women.
I wasn’t clear—I meant the specific case of a man lowering the status of a woman for consenting to sex with him.
Hamlet, Act IV, Scene V:
(Ophelia is singing bawdy songs)
hmm, that’s interesting.
I’m still inclined to think that I don’t have a problem with it, if, in the clear light of day with plenty of time and space to think it over, the rationale still holds water. If my best reasoning can accept it, then fine. If I know that I was influenced by the “salesman” but I still don’t really have much to regret, it’s okay. He just swayed me to try something that turned out to be good for me (or, at least, within my range of acceptable outcomes.)
Example: suppose you try to convince someone that sex before marriage is okay and give them reasons that they shouldn’t feel guilty afterwards. I’d say you’ve still helped that person rather than hurt them.
It’s a borderline issue for me, though.
Or, to put it in more objective terms: If you get someone to do something by improving their decision-making process, or if you cause someone to do something that is instrumentally rational relative to their values, then it’s not manipulation. If you believe that premarital sex is okay and you think you can persuade someone to feel the same way, then that’s okay.
Aurini’s hypothetical example with Mary doesn’t pass this criterion. Aurini didn’t really believe that donating to the charity would be a good decision for Mary to make.
Yay, ethics of influence and seduction, one of my favorite subjects. I’m still figuring out my thoughts, so I would appreciate it if people tell me if I’m making incorrect assumptions.
First, I’ll state that I prefer the terms “unethical influence” and “unethical influence” to “manipulation” and “non-manipulation,” because people use “manipulation” to mean too many different things.
Second, in ethical discussions, we should distinguish between things that would be a good things to do that aren’t morally required, and things that are morally required (Kant called these “imperfect duties” vs “perfect duties”). Also, we should distinguish between ethical courses of action, and empathetic courses of action. There are lots of cases where there’s an action that’s a good and empathetic thing to do, but actually requiring it as a perfect duty would screw everything up.
There is clearly a perfect duty against physical coercion, and that duty applies even if someone happens to enjoy what you are doing. An example of that case up when Roissy (a pickup blogger who built up a large following, but isn’t actually typical of the seduction community) condoned slapping women in arguments because it could “turn them on.” I argued that even if a certain percentage of women are turned on in that case, slapping without consent is unethical on both deontological and consequentialist terms. That thread has since been deleted, or I would link to it.
Hitting someone nonconsensually violates their rights from a deontological perspective. From a consequentialist perspective, the expected value of a positive reaction occurring is negative.
Is it empathetic to avoid people regretting your forms of influence? Yes. Is it an imperfect duty to avoid buyer’s remorse? Probably yes. Is it a perfect duty to avoid influence that could create buyer’s remorse? Probably not.
This moral principle (as I understand it) is a bit too broad. It catches many things that we would actually consider ethical. A bit part of the problem is that it doesn’t account for uncertainty about whether buyer’s remorse will occur; I’m not sure what levels of uncertainty you are imagining, so the following analysis may reflect my interpretation of what you wrote better than it reflects your actual views.
Let’s say you want a friend to come to a movie with you. Your friend initially protests, but then you hype up the movie. Your friend decides that it sounds cool, and comes with you. Afterward, the friend enjoys the movie, but afterwards decides that it wasn’t worth the cost of admission.
By your criteria, this was “manipulation,” but I don’t think anything untoward has occurred here. Agree/disagree? We are, of course, assuming that your friend comes out of interest, not merely to make you happy. You don’t know in advance that your friend will decide that the movie wasn’t worth it.
But this sort of influence can sometimes result in people doing things that they regret; I’m having trouble reconciling it with the other principle you articulated.
Now, what if you knew that your friend probably wouldn’t find the movie worth it, and you dragged them along anyway? That’s falling into the category of unethical social influence, and perhaps that’s the scenario you are proscribing. But if you believed that they would enjoy it, and you turn out to be wrong, then arguably you’ve done a good thing by taking your friend to a movie that they might plausibly like, and that they believed that they would like based on information that you provided them… even though you were both wrong.
When influencing people, we must assume by default that people know their preferences better than we do. You must give primacy to the version of your friend that wants to see the movie now after you’ve told them why you think it will be great, rather than to the version 5 minutes ago that didn’t want to go out of the house.
People usually know more about what they will or won’t regret, and what risks they are willing to take, than we do. So if someone is willing to engage in a certain activity with you, that’s pretty strong evidence that it’s a good risk for them to take from their perspective. The fact they initially didn’t want to in the past is evidence that they might have buyer’s remorse, but that evidence is much weaker than the evidence of their preferences in the present.
Of course, if you have some information that someone might end up getting buyer’s remorse that they might not be aware of (like the product you are selling is defective), and it’s information that you believe the other person doesn’t have access to, and shouldn’t be expected to figure out for themselves, then influencing them starts becoming unethical. Perhaps that’s what you mean when you say:
Yet it’s actually rare that you “know” in advance whether or not someone will regret sex; in most cases, you really only have a guess, and so do they. By default, the informational asymmetry about whether the other person will experience buyer’s remorse favors the other person; their guess is probably better than yours about how they’ll feel later, and you shouldn’t try to do their thinking for them unless you know something big that they don’t.
I think you have a much better handle on this than I did; by comparison my comment was pretty flip.
What I was trying to get at was that it isn’t nice to tempt people to do things that are bad for them. (In my opinion it’s outright wrong to use physical force to coerce people except in self-defense; not nice is a weaker statement than wrong.) I don’t think it’s nice to influence people by selling a product you know is defective, or trying to get them to buy something you know they can’t afford. I don’t think it’s nice to talk your friend into going out to a party the night before his big test.
Yes, in some of these cases, the “victim” should have known better, but strong influence can bypass willpower. The better you are at tempting people, the more responsible you are for the results of the temptation. It may even be fair to say that it’s mostly because of you that your friend flunked or your client went broke.
On the contrary, I don’t think I’d mind being influenced to do something that turned out to be good for me, except in a few cases where I have a strong commitment against it and a second-order desire to retain that commitment.
I think we’re saying something similar here. It’s a failure of empathy to tempt people to do things that might be bad for them, but I’m not convinced it’s always a failure of ethics in the sense of perfect duty. There are plenty of forms of influence where I would be comfortable saying that the influencer is being a jerk, but where I wouldn’t be comfortable saying that they are doing something unethical.
This is unethical under my analysis, also. It falls into the category of where you know something big that the other person doesn’t which destroys informed consent.
If you know this for sure, that would also be unethical under my analysis. My analysis only applies to situations where people are capable of watching out for their own interests. This is the default situation, but if someone gives you strong evidence that they can’t watch out for themselves (e.g. they are trying to buy things you know they can’t afford), then you should scale down the influence. Unfortunately, most real-life situations of influence are less clear-cut.
This one seems a bit less clear-cut. If you get your friend to miss out on sufficient sleep, or get hung over, then yeah, that’s unethical. But if the problem is that he might skimp on his studying? That’s less clear. Your friend knows better than you do whether they’ve done enough studying for their test. If you invite your friend, you know that he can refuse if he needs to study. On the other hand, maybe he’s done enough studying, and would benefit from relaxing. Unless you have some evidence either way, you’re not doing anything wrong by inviting him and letting him decide.
Could you give an example of such a scenario? I’m not seeing it in either of the three in your post. Can you give an example of influence that is morally suspect, and (a) involves adequate informed consent, unlike the defective product scenario, (b) involves someone capable of watching out for themselves, unlike the scenario of buyers trying to buy things you know they can’t afford, and (c) is actually strong, unlike talking your friend into going to a party?
I tried to come up with an example, and the first thing I thought of was running an unhealthy fast food business and influencing people to eat your food. Then I realized that this example fails the test I proposed, because it would be known that many people don’t watch out for their health. While eating at your restaurant every once in a while might not be a health risk, your advertising will pull people in on a regular basis (in contrast, if you are running an ice cream shop, you aren’t trying to get people to eat there every day, and they won’t be eating ice cream instead of healthier food). There is something inherently wrong with supplying tasting food that is unhealthy, but if you do it in a way that influences people to substitute healthier food for what you are providing, and eat your food regularly enough that it’s a health risk, that’s unethical.
The best way to tempt people is to offer them what they want (in a way that they identify it as satisfying one of their wants). I think your claim implies that people who can offer things that other people want more should have higher responsibility for the results of influencing people. That principle may be a little to broad, and lead the people who actually have good things to offer (which is influential) to have a moral panic and become too shy.
In terms of sales and products, there are four simplified categories:
Bad products with bad salesmen
Bad products with good salesmen
Good products with bad salesmen
Good products with good salesmen
The principle you suggest would make good salesmen with bad products more inhibited… but it will also make good salesmen with good products more inhibited! Actually, many of the most influential products may be good products with good salesmen… should these salesmen with the good products really be fretting the most about responsibility just because they are good at sales?
I think we need to articulate a principle that makes good salesmen with bad products scrutinize themselves more than good salesmen with good products.
The degree that people want what you offer isn’t quite the right index for the degree of responsibility you should shoulder for the result. People who want something really badly might be more likely to throw caution to the winds and make mistakes, so degree of want is weakly related to the responsibility you should take to protect them. But there are other variables that are much better indexes for the level of responsibility you should take:
Your estimate of the expected value to them of accepting your offer
Your estimate of how well they are watching out for themselves
Your confidence that they have enough information to be giving informed consent
The reasons that someone is consenting to whatever you are influencing them towards, and whether those are the type of reasons that are well-known to be correlated with buyer’s remorse
How do these criteria sound to you?
Unfortunately, people who influence you can’t predict the future and know whether the results will turn out good or bad for you. Of course, they should make guesses, and refrain from influencing you if they think the expected value of the result will be negative for you, and you haven’t given informed consent to that risk. But when there is uncertainty about the result, the ability of people to make those sort of guesses varies depending on the domain.
In financial services, your broker really does know better than you about whether their product will be good for you, and the criteria for satisfying your preferences is objective and easily understood by them. As a result, a lot of responsibility falls on their shoulders.
In social interaction and dating, it’s a lot harder to know whether you are a good match with someone better than they do. You don’t know exactly what their preferences are and how you fit into them. Responsibility for protecting them from buyer’s remorse falls primarily (but of course, not completely) on them. Since they have the higher quality information about how you fit into their preferences, it’s their responsibility to communicate their preferences to you to help protect them from being unhappy later.
“Second, in ethical discussions, we should distinguish between things that would be a good things to do that aren’t morally required, and things that are morally required (Kant called these “imperfect duties” vs “perfect duties”). ”
That’s a good distinction, but it’s not Kant’s. For Kant, a perfect duty is prohibitive (“don’t lie”) while an imperfect duty is one which demands the pursuit of some end, like “develop your talents”. The ‘perfect’ and ‘imperfect’ refer to a grammatical distinction: perfect duties are complete when you’re not murdering people, etc. while imperfect duties are never complete and always ongoing.
The distinction you’re drawing is between the supererogatory and the obligatory, where the latter is what you have to do just to be decent, and the former is some extra good stuff you can do. Kant not only never makes this distinction, by his lights it’s impossible to make it.
I don’t know (or care much) about what Kant said but wikipedia seems to be closer to Hugh’s usage than yours.
Well, in case a fit of curiosity about Kant strikes you, the relevant discussion is in Metaphysics of Morals (not the Groundwork) 6:390ff. Neither Hugh’s interpretation nor mine is obviously correct and I think the matter is up for discussion. Hugh is right that a failure to pursue imperfect duties isn’t vicious or a transgression by Kant’s lights, but I think it’s neverless wrong to attribute to Kant the possibility of supererogatory action. imperfect duties are flexable because they can be trumped by other dutoes, not because they can be passed over. Failure to pursue imperfect duties is still moral failure, just not a failure of the same species as moral transgression.
You should give Kant a glance though. Perhaps his work is not strictly relevant to the question of machine morality (I think he might argue against the possibility of a moral machine), but it’s still very interesting stuff, even to argue against.
Where would you place trying to influence people out of feeling buyer’s remorse when they’re feeling it?
I place it at “closing the stable door after the horse has bolted”. ;)
The part of the game that involves reducing buyer’s remorse is for most part before the sex, not after.
I’d consider that an imperfect duty.
An imperfect duty to do it, or to not do it?
To do it.
Wouldn’t it be better to influence them not to buy similar products again?
In the case of sex, I propose a different demarcation criterion between ethical and unethical social influence than yours. The dimension I’m most concerned with is not remorse or lack thereof after the fact, but rather the reasons for consent at the moment of consent. In the past, I proposed the following definition for ethical seduction on a pickup blog:
The “anticipation of intrinsic enjoyment of the experience” criterion is important, because it gets rid of cases where people consent to sex out of feelings of obligation, pity, merely because the other person wants it, or because they had trouble putting on the brakes. This notion is similar to the notion of “enthusiastic consent,” but without the confusing connotations that “enthusiastic” may hold.
I’m not sure whether this is a perfect or merely an imperfect duty. The argument for this principle being a perfect duty is that if someone has sex with you for reasons other than anticipating inherent enjoyment of the experience (which includes emotional enjoyment, not just physical), there is too high a chance that they won’t enjoy it and feel buyer’s remorse later. As a result, the argument would be that you should refrain from having sex with people in such circumstances, unless you know that they are aware of the potential negative consequences and are willing to risk them.
I’m not convinced either way about this argument, but it’s plausible. It’s actually really hard to think about ethical principles around influence that aren’t or overly broad, and that don’t contradict our existing moral intuitions. If someone thinks they have one, they should try as hard as they can to poke holes in it.
P.S. Sorry about the triple-post brain dump.
Interesting—so part of the definition would be the regret afterwards? If they don’t regret it, you convinced them, but if they do, it was manipulation? In that case, can you manipulate someone accidentally (if you didn’t expect them to regret it but they did)?
Or, he’s a very good manipulator.
Or he’s very good in bed!
Or both of these are the same thing in his case!
Third option would include drug addiction.
Are addicts generally glad to be addicts?
My definition of manipulation: an attempt to bypass or bias a person’s normal decision-making process. Attempting to counter an already present bias doesn’t count. In the case where the sum of the facts clearly favor a specific decision, presenting all the facts doesn’t count as bias.
Manipulation says little about ethics, since people can be manipulated into doing things that are good for them, though it reflects poorly on the decision-making abilities of the one being manipulated and the honesty of the person doing the manipulating. It could both be argued that manipulation is always wrong, or that the intent of the manipulation matters more. My ethics say the destination is more important than the path, but that a path must be composed of a series of temporary destinations.
I’m not quite sure what you mean by “normal decision-making process,” if countering an existing bias doesn’t count as bypassing that process. What you seem to be referring to is some kind of idealized decision-making process that we hypothesize is what the person’s decision-making process would be if they were somehow not subject to any cognitive biases… is that right?
By “bypassing” someone’s normal decision-making process, I mean something such as massive misdirection and emotional tinkering, or verbal threats of some kind (I’ll never talk to you again, I’ll tell your secrets, etc), or something else such that it is ambiguous whether the victim can be credited with making the decision.
By “biasing” someone’s normal decision-making process, I mean minor emotional appeals or simply providing biased data, focused on all the good points of one option and/or the bad points of the other, or activating other biases such as by privileging a hypothesis. It’s still clear that the subject made their own decision, by thinking about and weighing the facts, although due to your manipulation their decision is more suspect than it would otherwise.
The reason I’m saying that it isn’t manipulation to counter biases or provide facts even when the facts clearly favor a decision is that these don’t feel like manipulation. Do these sound manipulative? “Don’t use your lighter, this place is full of fumes” “When you’re in jail for killing your cheating girlfriend, don’t drop the soap” “You’re so drunk you can barely stand. I’m not going to let you leave with that guy, because I know you will regret it later.”
I grow more uncertain, rather than less, as I read your explanation. It sounds like you’re simply working backwards from your intuitions about what feels like manipulation: if I direct your attention in ways that feel manipulative, for example, you class it as “massive misdirection,” otherwise you don’t.
Human definitions aren’t nice, simple, logical, syntactically correct meanings for words—often they’re defined more as “I’ll know it when I see it”. When I want to figure out my own definition for a word, I have to carefully analyze different scenarios where the word applies and doesn’t apply, and try to figure out a definition that fits. Even this is incomplete as it doesn’t account for connotations and subtext. (Once I saw a TV show where they made a reference to sex using the words “play parcheesi”. The meaning was entirely clear despite that I’ve never heard of such a connotation before. Similarly, in the sentence “It’s cold outside, let me go grab my goat”, the word “goat” usually means “a misspelling of ‘coat’” even if that is not what the dictionary says.)
So, my intuition tells me that it is not manipulation to bias someone’s decision-making process towards their normal state if their current state is highly unbalanced due to drugs or emotion. The connotation of manipulation as being negative tells me that helping people make an obviously good decision shouldn’t have a negative connotation, and so shouldn’t be categorized as manipulation.
Ah, I see. So, sure, you’re welcome to your lexical intuitions, and you’re welcome to talk about “manipulation” while referring to the fuzzy concept your lexical intuitions point to. That’s what most people do in casual conversation. And when talking to someone whose intuitions differ from ours we either get derailed into discussing what “manipulation” really means, or we find some other way to talk about the concepts in question, or we fail to communicate at all, and that works more or less OK for our purposes much of the time.
Your original comment made it sound like you were trying to be more rigorous than that… sorry to confuse the issue.
Oh,boy.
Let me suggest that the set of “obviously good decisions” is much narrower than you seem to think. And that is even ignoring the elephant of an observation that other people’s ideas of “good decisions” are likely to be significantly different from yours.
Your definition is too broad—for example, it applies to women using makeup. Maybe amend it to “creating negative emotions in someone else for the purpose of getting them to do something”.
Specifying negative emotions is too narrow; that wouldn’t apply to any strategy that leaves its target marginally happier but at a resource cost considerably greater than the marginal increase in happiness could’ve been obtained with elsewhere. Of Cialdini’s 6 “weapons of influence,” all of which I’d classify as manipulative, only “authority” seems to cause negative emotions with any consistency. PErhaps the metric is orthogonal to the quality of emotion?
“Negative emotions” certainly isn’t right—the example in the post was about making the woman feel better.
I’m not sure I agree with your exception (I don’t equate “making a good impression” or “living up to a social expectation” with “creating emotion”), but perhaps we could make it clearer by adding “for a specific decision” to the end? i.e. the manipulation must have a specific goal.
The example in the post is not okay because it’s piggybacking on an existing negative emotion, and if the woman refused, that emotion would’ve been reinforced. Like a guilt trip.
So do you not think it’s possible to manipulate through positive emotion? What about flattering and pampering someone ’til they fall for you, then robbing them blind?
It’s possible, but I don’t have the same aversion to it.
Wait, what? Really? You don’t find that example scenario objectionable?
Hmm. To me it’s kinda “bad in theory”, like killing kittens. The strong hate is reserved for the things I actually did a lot and then decided to cut out.
Heh. Okay, we think about those things very differently, but that’s fine. :P
I’m assuming this is a definition for a specific kind of manipulation.
You can, of course, use things other than emotion to manipulate people. It is the bread and butter of board game strategy, for example.
EDIT: Come to think of it does this definition cover people whose ‘normal decision-making process’ is essentially emotion? The more I look at it the more it seems like what I consider manipulation is an extremely messy concept.
That is a very good definition.
Heh, I thought so, but apparently several other people have problems with it. Any thoughts about the objections that’ve been raised in this thread?
They’re giving me ideas for articles I hadn’t planned on writing; which is awesome.
Still letting things percolate—I want to do the pieces on the basic tools first.
Normally I would assume that the purpose in teaching them is indeed to help the reader avoid them, but this is explictly refuted in the post. That’s what confused me.
I’m thinking that the author, like most of us, really believes that if you know what’s happening you might have some marginal chance to avoid the effects of these tactics, even if he claims otherwise.
That’s my guess as well—and seems to be part of the answer, judging by the above. But I wasn’t sure enough to assume it without requesting clarification.
Where in particular do you perceive the lie in the situation you described? Would her donation not have sponsored the child? Would the donation not have made her feel better? Or is that you do not believe she should feel better for sponsoring a child; that in your mind it would be dishonest for her to displace her current grief with this balm?
Of course if the donation would not have sponsored the child, then it would be dishonest to claim that it would.
I can also imagine the donation not actually making her feel better. It would be possible to simply overwhelm her (e.g., intimidate them with a whirlwind of emotional stimuli) into doing something she didn’t want to do. This is in the direction of being mean/bullyish. People can push boundaries and this is being pushy...but we don’t often describe it explicitly, perhaps because it requires such a high emotional intelligence to identify and name it.
Finally, the third case is that you don’t believe sponsoring children is a feel-good thing. Which would be a strong indication that that wasn’t the right job for you, but wouldn’t mean that sponsoring a child wasn’t the right thing for her.
My objection: forcing myself onto somebody unexpectedly (door-to-door), and using techiniques they’re not prepared for, with an ulterior motive.
I believe in people’s right to self-determination; that includes the right to be depressed.
I meet someone standing on a bridge, thinking about jumping off. Can I try to persuade them not to jump?
Does the answer depend on whether or not I’m being paid by a suicide-prevention group?
I believe in not physically assaulting people. What does that have to do with being honest with them? Is it the ulteriority of your motive? I think it was clear that you were working for the charity, no? If you find a goal that benefits both you and me, are you interfering with my right to self-determination by telling me about it?
In my view, the biggest problem with continuing to influence her is one you having mentioned: she withdrew consent to be influenced. More in my longer response.
The bit about Sheila from accounting prompts a fun question for those LWers who subscribe to specific theories about gender relationships (PUA, feminism or whatever): within a couple, do you predict the events “husband cheats” and “wife cheats” to be independent, correlated or anticorrelated? To me it sounds like it could just as easily go one way or the other, and each answer has many rationalizations that immediately spring to mind. (Won’t spell them out to avoid spoiling your fun.)
Correlated, because time spent apart causes infidelity for both. Also, cultural norms make a difference, and both are exposed to the same culture.
Anti-correlated, because differences in libido causes a higher probability of infidelity on the part of whoever wants more sex, and a lower probability on the part of whoever wants less.
I’m not sure which of these I find more convincing.
Anticorrelated: Sheila’s attraction to Hubby implies Hubby has qualities which make him appealing to women, which makes Wifey less likely to cheat.
Correlated because cheating by one partner implies less mate-guarding by the other. A partner who doesn’t bother to mate guard is less concerned about losing that partner, and thus more likely to cheat.
Also both partners are more likely to cheat if the passion has died down in the relationship, or if they’re in the midst of a conflict.
Interesting that given the choice between leaving the woman to cry and an obvious route to make her feel better, you felt that the correct, moral thing to do was to leave her miserable.
There would certainly seem to have been a third option: comfort her without selling her anything...
FWIW, I second Relsqui’s curiosity about why you think this sequence is a good idea.
From a consequentialist standpoint, I agree that leaving her happy is better than leaving her crying.
However, it’s not definitely right to comfort people using religious ideas you know are wrong, unless you intend to correct them later or something. Then its just hazy.
Where it gets really muddled though, is where nothing you’re saying is necessarily false, but it wouldn’t seem true without your mentioning it. Like, the link between reasserting her “caring nature”, donating to the charity, and dealing with her father’s death. And how she can reconstruct her life and assert her values by combining them.
She would not come up with that on her own, but once she starts thinking about it/herself that way, I think there are compelling arguments that it actually becomes that way. Or at least feels like that way, and lacks objective evidence to the contrary. Which might be enough in self-concept stuff?
Sorry if the last paragraph (or two) is a bit mysterious or hard to follow—I’m confused about the subject.
If she were upset, went onto the website, and donated there, she might feel better. She might not think of it, because humans are not automatically strategic.
However, if you show up at her door, you not only have the ability to suggest that she reaffirm her caring nature and ability to overcome adversity, you give her a witness.
That means that her signals of “I am caring” and “I can do this” are going straight into the face of someone she’s never met, someone who, if she meets them again, will know nothing about her except for this moment of pure altruism. It’s an extremely strong signal, which makes it very satisfying.
I’m pulling this out of my ass for the most part but I think having a witness there in person greatly enhances the service of signalling her caring to herself.
Additionally, not everyone’s conscience is set up to allow actions like “donate money to convince myself I’m a good person so that I’ll feel better”. On a certain level it seems like a total sham, a selfish attempt to restore an undeserved or inappropriate level of happiness.
I’m not saying this is a rational viewpoint, mind you. Only that I expect plenty of people would have (something like) it.
But for an opportunity to just happen to come along, where you can prove your worth without having set forth just to prove your worth so that you can feel better, might seem more legitimate. Which in a way it might be. If someone says to you “I’m going to test right now whether you’re a good person. Donate 5 bucks to that charity”, your doing it wouldn’t be very compelling evidence that you are actually a good person. It may just be that you want to be thought of as such, and you will be keenly aware of this motivation even as you give up the money. This hidden motive is something it’s easier to hide from yourself, even if it’s true, if you had given 5 bucks because you were walking by and thought “why not?”.
The strange thing is that the OP’s example seems more like the former case. I think this is where your “witness” idea comes into play. The donation was his idea, and he already believes you’re a good person. It doesn’t feel like trying to prove it to yourself, it feels like a friend trying to help remind you of it.
It is muddy. To me a sort of deontological approach makes sense of this (I’m sure anyone who prefers utilitarianism can convert this somehow). Basically, I wouldn’t want to live in a world where salesmen were encouraged to take advantage of your vulnerable emotional state in order to push products on you, even if they make you feel better (yes, I already live in this world, but still).
There’s still a non-dirty-feeling option, which might be to find some other charity that saves kid’s lives and help the woman donate to that. It might seem like, all things considered, this is the same result but requiring more work, making it a questionable choice from a utility point of view. But ultimately I just don’t trust human beings, even myself, to not act selfishly, and to distinguish at all times between rationality and rationalizing.
In short, I can see how an argument could be made that taking advantage of her emotional state to help save lives is a morally acceptable option. I just don’t trust anyone making that argument that has a stake in it. The safe bet is to always look for the third option, just in case.
Thank you for writing a post in a normal, conversational, non-esoteric philosophy tone.
Thank you for reading.
I really can’t help but correct your math: 12! would be if you could choose how to order the 12 toppings, but that’s not what you’re doing—you’re choosing a subset of them—that’s 2^12 options. Don’t worry, you were only off by five orders of magnitude.
Sorry about that.
Would we expect those who consciously learned the Dark Arts to be more or less responsible with them (that is, using them ethically) than those who are naturally skilled?
~12!
Damnit, you’re right! I suppose you could demand that the sandwich artist put your pickles on top of your tomatoes rather than vice versa, but that’s not a way to engender good customer service. I’m so rusty with my mathematics that it’s not even funny.
~Acquired vs Innate
Great question—I never thought of it like that before—and the answer is a resounding yes. One of the guys I worked with was dumb as a brick, and a conspiracy theorist to boot (he honestly believed that 2Pac was assasinated by the Bush government for criticizing Dubya on MTV… seriously) but he was absolutely amazing at the door. I never saw him at work, but some of the people who did were revolted by his methods—methods that worked.
He was like some sort of preternatural killbot, who only had one setting. He’d go on and on about what a moral person he was, but in his daily interactions he was a snake in the grass, manipulating people for his own benefit, with never an ounce of guilt for what he was doing—probably because he lacked the ability to be consciously aware of what he was doing.
Imagine an AGI which was extremely personable and ‘caring’ but ultimately brought everyone it interacted with into a deep depression. That’s him in a nutshell. I would have loathed this guy if not for the pity; he was irredemably stupid, unable to re-order his life to take advantage of his talent, and when he’s sixty years old all he’s going to have is the Canadian Pension and whatever he can mooch off of unsuspecting people—people who will love him despite the mooching.
Compared to him, somebody smart enough to learn the skill is probably going to be less destructive; the exceptions are serial killers and UAIs, but they’re going to figure this stuff out anyways.
Well order is certainly an option, at least around here.
I’ve seen how they make sandwiches; the process is approximate enough that the order doesn’t really matter. At least, not nearly as much as the choice to add or omit any particular topping would matter.
Location matters too. Around here, customers usually request toppings one at a time and watch them being added. In the UK (or at least in Leicester), I believe the typical interaction goes something like, “Salad?” followed by a negative or affirmative response.
2^12? Isn’t it 12C2 (= 66), rather than 2^12 (= 4096)? It’s 12P2 (=132) if we care about order (since there are two different ways to order any two toppings.)
Only if you’re only allowed to have two toppings, no more and no less.
Why should I believe that the way you describe the hypothetical meat-fuck in your head is how it would have really turned out? (I imagine Bricky could have pulled it off)
Because assinging a 1 or a 0 to the statement “Aurini was good enough to pull that shiznit off” won’t change anything else in your belief system; it presents no significant consequences to your ability to understand the universe. However there are emotional effects.
Believing the story to be generally true, and my claims to be highly accurate, means that you live in a slightly more interesting universe. Disbelieving means that things are slightly more grey and tawdry.
So you might as well believe in God, because what do you have to lose?
Is it bad I imagined you were talking about some alternative to Clippy? Or was that the intention?
Ok, I don’t get this. Is this an allusion to something?
In Ian M Banks’ novel ‘Excession’, the term meatfucker is used as an derogatory term for AIs that violate a cultural taboo again reading human minds.
http://en.wikipedia.org/wiki/GCU_Grey_Area
Well, for a specific AI. No other AIs seems to do this.
Don’t mindreaders have better information than Banks does on whether people want their minds to be read or not?
thanks :D
Bang on. I’ve appropriated the term for any of the brain-raping classes: psychiatrists, politicians, marekting gurus, journalists, etc.
Yeah. Real brain rape starts with the Black Knife Kiss* and goes downhill from there.
*No, you don’t want to know.
I was mainly trying to avoid indescriminate use of the word fuck; now I will say, given that the individuals in question are in the habit of greasing up their thin, pliable members before sliding them down some petitioner’s ear canal to excitedly jab and thrust and the pre-existing grey matter, and then leave behind the sticky, congealing remnants of their half-formed ideologies putrefying in the victim’s brain—given all that, it’s true that fuck would be a more appropriate term than rape.
But this is LW, and I’m trying to be all formal and shit.
[P.S. I think you’re the bees knees Alicorn!]
This is the only comment I’ve ever seen here where I’ve vacillating between voting up or down, rather than just leaving it alone. I can recognize that it’s either really good or really bad, but I can’t decide which one, so congratulations, I guess.
Folks around here will react more positively to “fuck” (which is just profanity) than “rape” (which is not a joking matter to some.) Honestly I think we could use a bit more of your writing style.
“Rape” is not more formal than “fuck.”
The ideal situation here from my perspective would be if you made “Meat Fucker” a link to a relevant wikipedia page, then avoided posting disgusting passages so that people (like me) who collect Less Wrong comments like they’re rare pokemon won’t have to read them.
related comic
Hit ‘Next’ to see the followup, also related.
May I suggest using the term “mindfuck” instead? It seems to already have a relatively well established base of usage and was defined eloquently by terryzz on UrbanDictionary.com as follows:
“Meatfucker” is good too, but I think “mindfuck” or “brainfuck” is more evocative of the concept.
The first use of it I encountered had a positive connotation. Apparently there are very conflicting schools of thought on the matter.
Looks like the viewpoint-changing, not necessarily a bad thing meaning might have been the original one.
It could be a reference to GCU Grey Area, an “eccentric” spaceship/AI from the novel Excession.
Google for “meat fucker” (in quotes) turns up your comment, and an obscure 30-minute movie. It can’t be that common a phrase if your comment is already one of the highest-ranking Google results for it, so I’m guessing it’s just a relatively new insult.
Out of interest, how many of the people who listened to your pitch subsequently gave donations?
The market was saturated at the time, the biggest difficulty was finding people who A) gave the slightest shit about children, and B) weren’t already sponsoring 2 or 3 kids already.
Generally speaking, just knocking on doors and talking would get you 1 a day. Having enough product knowledge to bounce common complaints gave you 1 − 2 a day. Learning to Impulse, Bounce, Relate, recognize Buying Signs, and—the hardest of all—Neg the people, would get you 2 or more.
(So they’re sitting on the fence, eh? “You know what? We’ll get you set up with little Jimmy here, and tonight—I promise you—you’re going to sleep like a baby.”)
The real challenge with all of it was that it was a marathon, not a sprint; you had super-positive, happy fun time for 6 hours straight, 5 days a week, four weeks a month.
Personally, I tend to be a hater. This fact made the job difficult.
Dark Arts done right… people need to get used to being manipulated a little to at least get the feel of it happening, or to learn a bit of it if that doesn’t work. Experience as a net troll does wonders to keep me from getting riled up. But honesty does not have the least collateral damage trolling does.
This is true for me, as well. The groups who are easiest to troll are never who I think they will be.
I’m intrigued. Which groups have low troll-thresholds?
1) Any group of mostly men dedicated to any sort of hardware—guns, cars, for all I know chainsaws. For some reason men don’t just form attachments to the specific tool they actually have (Cars, Guns, Hammers, scapels are all tools), they have to defend the BRAND and/or model (Mopar, 1911, Macintosh are the most vociferous defenders).
2) Any group that thinks unreasonably highly of itself. Atheists, Mensa, Vegetarians/Vegans, Hippies, Mac Users.
3) Any group that feels under threat, either from the march of technology, a conspiracy, or just reality damaging their world view like Socialists, Vegetarians/Vegans, 1911 owners and Mac Users.
I salute your ability to troll all of these groups in a post about what kind of groups are easy to troll. I almost started to argue on some of these points before I saw your game.
I don’t troll. I never have. I will attempt to reasonably argue a point of view, but (as an example) I will not go to www.ar15.com and start a post containing my (slightly negative) opinion of it, nor discussing my preferences to that device (Sig 55x, AK etc.).
However I have been on the intarwebs since 1993 and have engaged in some fairly vigorous debates. I have also noticed who responds and why. This is not building interplanetary transport devices.
I will admit I should have put an etc. on the end of points 2 and 3.
In retrospect I should have put Linux Users in there as well.
I used to think I trolled all the time, but then I realized I wasn’t looking for anyone else’s anger or frustration, just trying to find their exact train of thought and get to reasonable arguments. Have you had worthwhile experiences probing for information in ways that could be misinterpreted as trolling or just by asking?
Even when there’s no trolling going on, the discussions I’ve seen online are predominantly shouting matches where no one leaves happy. When there’s a troll, at least someone wins. But when debates become rational, everyone wins.
There are times when I’ve made deliberately provocative statements to attempt to get someone to think and realize, but when you’re dealing with mental children this doesn’t work. Meaning it rarely works.
I can’t decide whether our culture has created a great “under class” of people who don’t want to deal rationally with the world, or whether it’s just a ‘feature’ (or bug if you will) of the great masses of humanity. Either way I’ve come to the conclusion that the vast majority of people are basically peasants with varying degrees of technical training, willing and able to operate at fairly high intellectual levels within narrow areas, then going home, popping a pabst and watching prime time cop buddy shows for decades at a time. you know, as opposed to operating at high intellectual levels, then going home and getting on the internet and arguing about peoples intellectual levels.
There are places online where discussions happen, but these are often places where discussion of one category of contentious issues is disallowed by fiat, and forum owners and/or moderators screen or are very active in preventing those sorts of shouting matches. Whether you’d think these were rational discussions or not is another matter :)
He almost got me, too.
For some reason technology websites are very easy to troll. Atheism-themed websites are so easy to troll it’s slightly embarrassing.
I have found websites devoted to physical fitness and bodybuilding are much harder to troll than average. I would have thought that “smarter” (maybe just more nerdy?) groups would recognize trolling more or at least have more experience dealing with trolls.
Do you think it might be something to do with how eager the people on those sites are for a fight?
Certainly. Upon a moment’s reflection, it seems obvious that websites are easy to troll if they have lots of people with self-identities that can be assaulted. I guess what is strange to me is that people form such a sensitive identity-affiliation with their preferred tech brand and non-belief in deities.
Wired.com is a great example. All you have to do is mention the words “Microsoft” or “Apple” and within 5 minutes the comment threads become a ridiculous flame war between entrenched camps making the exact same arguments they’ve made a million times before.
I’m trying to popularise the term Pro-Skub to generally refer to someone on the other side of a trivial but heated dispute.
Unfortunately I’m pretty confident that somewhere out there is someone trying to popularise Anti-Skub for the same purpose.
sixes and sevens, your comment is unnecessarily inflammatory for this forum; please use a less heated example. Nobody here wants to get into a skub debate—leave that for the bars.
You can’t make the Skub issue go away by simply ignoring it. That’s exactly the sort of tactic Pro-Skub folk use to protect their otherwise indefensible positions.
Uhm, looks good as invertible fact, by the way.
If there were solid data on the trollability of the various forums, that would probably qualify. As it is, I’d bet the bodybuilding.com forum is hard to troll because they’re well-innoculated; the place is like a secondary /b/. Where the norm is more serious debate, violating that norm in a provocative way will be easier.
When you manipulate a person’s distressed emotional state to convince them into an action, even when that action is a positive one (sponsoring a child) there is always the potential that the person who was manipulated will later realize the manipulation.
Essentially—If he had convinced the crying woman to sponsor the child she very well have felt better on an ongoing and permanent basis (a positive result for everyone involved), however there is also the potential that at some later date she would have realized the manipulation.
My understanding of being manipulated is that the later realization has a highly negative outcome—I.E. making the woman more depressed than she was before the manipulation, more likely to believe that all people are manipulators (less trusting), etc.
Whenever you piggy-back a distressed emotional state to manipulate a person into an action you risk breaking that person (Causing them to stop trusting people).
I’m pretty sure I don’t like being Sold. I generally have a pretty good idea of what I do or don’t want, and (to the extent that I recognize it), I dislike being manipulated.
You might be right about people in general, but it seems like a hard thing to check on whether people like being Sold as distinct from whether it’s possible to Sell them.
I don’t like feeling that I’m being Sold. Nevertheless, I’ve bought many things. Those things were sold to me. What’s the difference between Sold and sold?
I think I have a pretty good idea of what I want, but I don’t always know what I want to do to fulfill those wants. My default want for any product is zero, until I’m shown how it could fulfill one of my higher order wants. That’s a sales process, no?
In some of the most successful forms of sales, we never feel like we are being Sold.
There’s a difference between seeking a product, being invited to investigate a product, and being Sold a product. Examples:
I drink a couple of gallons of milk every week. I go to the part of the grocery store which contains milk, and select the kind I know that I prefer (skim) in a brand selected for price, familiarity, attractive packaging, or incidentally as part of sorting by expiration date. No person has to interact with me to cause me to choose the milk; if it’s not available at one store I will arrange to get it elsewhere. I have sought this product.
On my way to check out of the store with the milk, there is a display of free samples of goat cheese, of a type which I haven’t eaten before. I find taste relevant in my cheese selection, so I try some. It tastes good, so I buy a round of the cheese. No person has to interact with me directly to cause me to choose this cheese, but if it hadn’t been there I wouldn’t have bought any. I may continue to buy the same cheese regularly. I have been invited to investigate this product.
Between the goat cheese and the cash register, I am stopped by a salesperson who entices me to buy a specific brand of yogurt. I have neither a preexisting interest in yogurt, nor any reason specific to the brand to select this and not another yogurt even if I did. The salesperson says nice things about eir brand of yogurt: the company that makes it does good things for the environment, the yogurt contains ingredients I don’t recognize which are reputed to have health benefits, the container is recyclable, the yogurt is flavored with an exotic tropical fruit, if I buy this yogurt I can enter a drawing to get a free tote bag, etc. Note that none of these things were involved in my choice of milk or cheese, above: they aren’t the kind of information I find relevant to my selections of dairy products. They’re things I find weakly positive but not enough to look for them actively. And they’re presented by a person, not by a label. While I might buy this yogurt, and might even continue to buy it in the future, it has entered my life by channels via which I did not mean to admit yogurt when I walked into the store.
I don’t feel like you’re giving a good marginal case—for example:
I go to the store looking for orange juice. The store is out of the brand I usually buy, but there are free samples of passion fruit juice being handed out nearby. I stop to try some, at the behest of the person pouring the little cups; partly because I want them to feel like they’re doing something worthwhile, partly because I’m thirsty. Drinking it, I realize that passion fruit juice is the greatest tasting thing that I’ve ever encountered. I immediately buy ten gallons and update my favorite things on facebook.
Was this manipulative? What if the juice stand is on the side of the road instead of in a supermarket?
Even if it was manipulative, is it a bad thing?
That seems like a blend between all three (you sought juice, but not this kind; you tried a sample and enjoyed it on the basis of approved criteria; a person beseeched you to take the sample). If wanting people to feel like they are doing something useful were a means by which you independently chose juices, there are probably better ways to optimize for it. The strength of your response to the taste of passion fruit juice in this example doesn’t seem like the kind of thing that the sample-pourer could have easily influenced, though, so I’m inclined to process it more like the goat cheese example.
(Incidentally, I learned to like interesting cheeses via a sample of goat cheese in a grocery store, and continue to spend significant amounts of money on interesting cheese years later. The sample table was attended but the person spreading cheese on little toasts was not very interested in making people eat the samples. I had to ask her where to find retail-sized amounts of it; she didn’t disclose the information unprompted.)
I see parallels to the clever arguer in being Sold something.
Here’s what I think of as an optimal situation: For a very long time I’ve thought of ginger ale as a better idea in theory than in fact. Recently, a little gourmet shop opened nearby. The owners make some efforts to Sell me things, but don’t apply so much pressure I feel rolled over. Sometimes I like what I buy, sometimes I don’t, occasionally it seems wildly overpriced.
Fevertree ginger naturally light ginger beer is the ginger ale that’s been lurking in the back of my mind. Nobody had to tell me that I was going to like it. The packaging isn’t especially notable. I have actual preferences.
Perhaps someday I will find onion rings which are as good as my idea of onion rings.
Please take my word for it that I don’t want the kid at the counter to make me love the sandwich. This is not my kink. I can appreciate presentation. I like dealing with pleasant people. Now that I know something about Baysianism, if I have no idea what I want, I might ask for the most popular selection.
I can’t imagine wanting or trusting the opinion of a sales clerk about clothing—their job is to sell it, not to have good judgment about how I look. I suppose that if I were more interested in clothes, I might find some sales clerks whose opinions I respected, but that’s still not the same as wanting them to sprinkle pixie dust.
Hugh, my paranoid reading is that you don’t think I should have boundaries about what I want. It’s possible that there’s just something about the way I write about it that sets you off. The non-paranoid reading is that you don’t think anyone should have boundaries about what they want.
I suppose it’s a boundary thing, but I’m content to let you have your personality structure. I really don’t think this aspect of mine is doing me any harm.
I expect to be Sold, because I understand that That’s Business, but it bothers me when we’re talking about a major purchase, because I’m scared of accidentally parting with a dangerous amount of my money. I’m not sure salesmanship for big purchases is immoral, but it’s at least dangerous—a successful car salesman can destroy your life.
So, is the story real, and why did you include the spider (I reckon that is not real, too perfect)?
I notice that I am confused. Manipulation may have occurred if someone’s free will has been overridden. There was a nice post about the compatibility between free will and determinism, which I agree with, but I find the issue of manipulation shows I still have a fuzzy idea about what free will means.
The work on Friendly AI gives me a fairly good idea about how it’s possible to avoid being manipulative to a person, but the answer—following their extrapolated volition—is much too high a standard. What if you want to be neutral rather than selflessly and altrusticly benevolent towards them?
Still, their extrapolated volition seems to be relevant.
Fascinating post!
I agree with your reasoning here. Along these lines, I’m curious about something:
Do you think learning this stuff made you better or worse as a rationalist?