Quirrell, Dumbledore, Snape, Harry, and (increasingly) Draco have something in common. They are all creepy. These characters are intentionally inauthentic—acting as though they posses the specific beliefs, preferences, and abilities that they want others to attribute to them.
I feel unusually strong revulsion about this kind of deception—more than toward someone hiding their faults to manage their appearance, much more than toward someone being tactful and withholding or biasing sensitive claims to avoid conflict.
When I try to unpack “creepy”, my mind suggests it has components of outrage at violations of close interpersonal social norms, distrust of unfamiliar thought patterns, fear of people with motivations that need to be hidden, and a special kind of disgust related to fears of idols, photographs, glassy eyed dolls, humanoid robots, and other simulacra. - the disgust toward an exemplar that doesn’t fall clearly in or out of the human-mind category, toward a soul that has been captured in the depiction of a face and deprived of its intelligence and agency.
Are very intelligent people generally creepy like that? If I were a standard deviation smarter, would my peer group consist of people strategically concealing their identities and mutually modelling their mutual modelling up to the nth order of meta? Or is that inauthenticity just an abnormal personality type that doesn’t correlate much with intelligence, but does fit nicely into a rationalist literary drama?
As a sidenote, intelligent people may seem creepy to the general population even if they don’t try to deceive anyone. The mere fact of being more intelligent makes them more difficult to model for the average person. Then, when their actions violate the (wrong) model, the author of the model may feel deceived. The basic human irrationality: “if things don’t work according to my model, the problem is not with my model but somewhere else”.
Another contributing factor may be the illusion of transparency, when the more intelligent person thinks they made their intentions obvious to the people around them, but the average people don’t get the message, and then they are surprised when the intelligent person does the (unclearly) announced thing.
EDIT: Even the rational thing of “changing your opinion when faced with evidence in the opposite direction” may feel like a dishonesty to a person not used to this. (“Yesterday you believed X, and today you believe non-X; were you lying to me?”)
I’ve traditionally avoided seeming ‘creepy’ by blatantly violating various unimportant norms early in relationships. People don’t like feeling deceived, and they don’t like it when they are shown to be wrong after assembling a model. The trick is to make it clear from the outset that ‘this person is extremely hard to model’, or ‘this person doesn’t fit into any of my prefab models’. That way they don’t get irritated about being wrong, they just assume it’s par for the course.
Examples of norm violation:
I am male and maintain a prominent 3 foot long braid which I often wrap around my neck
I make childish faces at people and things
My default vocabulary is apparently pretty exotic
I don’t flinch and easily participate in sexual topics when they come up
I squeeze in comments about how the human race is going to fix this ‘death’ problem whenever I can
When asked occupation, I say that I started as an engineer, then started a video game company (wtf?)
When asked belief (common in the US), I state ‘true nihilist/atheist’ and that I’m quite possibly the least spiritual person they will ever meet
All this stuff combined generally jams up people’s predictors to the point where they give up on being correct, which largely fixes the creepy problem. Most people have stuff like this available, the trick is to emphasize and export the pieces which generate the most conflict with existing models.
Be aware that there’s also ‘sexually creepy’, which is a whole different ball of wax. Openness and being comfortable with sexual topics helps tremendously. It’s also a very good idea to focus your attention and eyes on faces, not reproductive hardware, when you interact with someone.
Conscious control over social presentation is a learned skill; it doesn’t come in the same box with intelligence or rationality, although either or both might make it easier to pick up. I suspect it’s prominent in Methods mainly because it serves the particular type of wheels-within-wheels plotting that Eliezer seems fond of.
We could have a conversation at this point about whether constructed social presentation is unethical or “creepy”, but I don’t think it’d get us anywhere. Some people have the squick response, some don’t.
(Incidentally, I don’t feel like Snape’s got this in-story. He’s certainly got a facade, but it’s the sort you build semi-involuntarily when you hate parts of yourself and desperately want to hide them, not the kind you consciously build to optimize social outcomes. Harry does have it, but shouldn’t have had the opportunity to develop it; it may be part of his Mysterious Dark Side/possible Harrymort package.)
For Snape, I was specifically thinking of the scene in Dumbledore’s office where Harry reveals that he knows about the prophecy and Snape reacts without hesitation as though he hadn’t heard of it. Snape was also a double agent during the war, and continues to maintain close relationships with Dumbledore and Lucius Malfoy. His actions do seem crude, awkward, uncontrolled or mostly defensive in other scenes such as in the bullying arc or his conversation with Quirrell in the forbidden forest in Chapter 77. But then, one can act with false impulsiveness too.
I suppose the characters are in a cold war and in the shadow of a hot war. That circumstance makes “offensive” deception in one’s social presentation more useful.
They’re also in an environment where creating an artificial persona is not merely useful, but a fundamental survival tactic: each of these characters would be dead by now if they were not Occlumens (and Lord Mafloy is an Occlumens, see chapter 47), which requires the person to also juggle a separate and highly artificial persona.
And of course, McGonagall acts similarly, even though her reaction is not similarly perfect, possibly because she’s not a perfect Occlumens (and had just dealt with a full day of Rationalist!Harry).
There’s some correlation between IQ and various measures of social awareness, despite the stereotype to the opposite, but it is a learned ability and many experts either don’t learn it, or harness it to different ends. Feymann’s writings on public speaking and instruction suggest that he, for one example, was highly aware of how he made his words and how his public face appeared. At the same time, I’m not sure how much of that was foresight and how much was later introspection—and artificially controlling the flow of a Congressional inquiry may be less ‘creepy’ than artificially controlling the flow of information to an eleven-year-old.
((Yudkowsky’s style may be part of the issue, as well. As in /Three Worlds Collide/ and /Sword of Good/, character dialogue, even from ‘normal’ characters, comes across as artificial at times. That more than the complexity of thought may trigger the creepy vibe.))
Quirrell, Dumbledore, Snape, Harry, and (increasingly) Draco have something in common. They are all creepy. These characters are intentionally inauthentic—acting as though they posses the specific beliefs, preferences, and abilities that they want others to attribute to them.
I feel unusually strong revulsion about this kind of deception—more than toward someone hiding their faults to manage their appearance, much more than toward someone being tactful and withholding or biasing sensitive claims to avoid conflict.
When I try to unpack “creepy”, my mind suggests it has components of outrage at violations of close interpersonal social norms, distrust of unfamiliar thought patterns, fear of people with motivations that need to be hidden, and a special kind of disgust related to fears of idols, photographs, glassy eyed dolls, humanoid robots, and other simulacra. - the disgust toward an exemplar that doesn’t fall clearly in or out of the human-mind category, toward a soul that has been captured in the depiction of a face and deprived of its intelligence and agency.
Are very intelligent people generally creepy like that? If I were a standard deviation smarter, would my peer group consist of people strategically concealing their identities and mutually modelling their mutual modelling up to the nth order of meta? Or is that inauthenticity just an abnormal personality type that doesn’t correlate much with intelligence, but does fit nicely into a rationalist literary drama?
As a sidenote, intelligent people may seem creepy to the general population even if they don’t try to deceive anyone. The mere fact of being more intelligent makes them more difficult to model for the average person. Then, when their actions violate the (wrong) model, the author of the model may feel deceived. The basic human irrationality: “if things don’t work according to my model, the problem is not with my model but somewhere else”.
Another contributing factor may be the illusion of transparency, when the more intelligent person thinks they made their intentions obvious to the people around them, but the average people don’t get the message, and then they are surprised when the intelligent person does the (unclearly) announced thing.
EDIT: Even the rational thing of “changing your opinion when faced with evidence in the opposite direction” may feel like a dishonesty to a person not used to this. (“Yesterday you believed X, and today you believe non-X; were you lying to me?”)
I’ve traditionally avoided seeming ‘creepy’ by blatantly violating various unimportant norms early in relationships. People don’t like feeling deceived, and they don’t like it when they are shown to be wrong after assembling a model. The trick is to make it clear from the outset that ‘this person is extremely hard to model’, or ‘this person doesn’t fit into any of my prefab models’. That way they don’t get irritated about being wrong, they just assume it’s par for the course.
Examples of norm violation:
I am male and maintain a prominent 3 foot long braid which I often wrap around my neck
I make childish faces at people and things
My default vocabulary is apparently pretty exotic
I don’t flinch and easily participate in sexual topics when they come up
I squeeze in comments about how the human race is going to fix this ‘death’ problem whenever I can
When asked occupation, I say that I started as an engineer, then started a video game company (wtf?)
When asked belief (common in the US), I state ‘true nihilist/atheist’ and that I’m quite possibly the least spiritual person they will ever meet
All this stuff combined generally jams up people’s predictors to the point where they give up on being correct, which largely fixes the creepy problem. Most people have stuff like this available, the trick is to emphasize and export the pieces which generate the most conflict with existing models.
Be aware that there’s also ‘sexually creepy’, which is a whole different ball of wax. Openness and being comfortable with sexual topics helps tremendously. It’s also a very good idea to focus your attention and eyes on faces, not reproductive hardware, when you interact with someone.
Conscious control over social presentation is a learned skill; it doesn’t come in the same box with intelligence or rationality, although either or both might make it easier to pick up. I suspect it’s prominent in Methods mainly because it serves the particular type of wheels-within-wheels plotting that Eliezer seems fond of.
We could have a conversation at this point about whether constructed social presentation is unethical or “creepy”, but I don’t think it’d get us anywhere. Some people have the squick response, some don’t.
(Incidentally, I don’t feel like Snape’s got this in-story. He’s certainly got a facade, but it’s the sort you build semi-involuntarily when you hate parts of yourself and desperately want to hide them, not the kind you consciously build to optimize social outcomes. Harry does have it, but shouldn’t have had the opportunity to develop it; it may be part of his Mysterious Dark Side/possible Harrymort package.)
For Snape, I was specifically thinking of the scene in Dumbledore’s office where Harry reveals that he knows about the prophecy and Snape reacts without hesitation as though he hadn’t heard of it. Snape was also a double agent during the war, and continues to maintain close relationships with Dumbledore and Lucius Malfoy. His actions do seem crude, awkward, uncontrolled or mostly defensive in other scenes such as in the bullying arc or his conversation with Quirrell in the forbidden forest in Chapter 77. But then, one can act with false impulsiveness too.
I suppose the characters are in a cold war and in the shadow of a hot war. That circumstance makes “offensive” deception in one’s social presentation more useful.
They’re also in an environment where creating an artificial persona is not merely useful, but a fundamental survival tactic: each of these characters would be dead by now if they were not Occlumens (and Lord Mafloy is an Occlumens, see chapter 47), which requires the person to also juggle a separate and highly artificial persona.
And of course, McGonagall acts similarly, even though her reaction is not similarly perfect, possibly because she’s not a perfect Occlumens (and had just dealt with a full day of Rationalist!Harry).
There’s some correlation between IQ and various measures of social awareness, despite the stereotype to the opposite, but it is a learned ability and many experts either don’t learn it, or harness it to different ends. Feymann’s writings on public speaking and instruction suggest that he, for one example, was highly aware of how he made his words and how his public face appeared. At the same time, I’m not sure how much of that was foresight and how much was later introspection—and artificially controlling the flow of a Congressional inquiry may be less ‘creepy’ than artificially controlling the flow of information to an eleven-year-old.
((Yudkowsky’s style may be part of the issue, as well. As in /Three Worlds Collide/ and /Sword of Good/, character dialogue, even from ‘normal’ characters, comes across as artificial at times. That more than the complexity of thought may trigger the creepy vibe.))