Scott Aaronson’s “On Self-Delusion and Bounded Rationality”
Poignant short story about truth-seeking that I just found. Quote:
“No,” interjected an internal voice. “You need to prove that your dad will appear by a direct argument from the length of your nails, one that does not invoke your subsisting in a dream state as an intermediate step.”
“Nonsense,” retorted another voice. “That we find ourselves in a dream state was never assumed; rather, it follows so straightforwardly from the long-nail counterfactual that the derivation could be done, I think, even in an extremely weak system of inference.”
The full thing reads like a flash tour of OB/LW, except it was written in 2001.
Around the middle of the story, the protagonist started to remind me of myself in elementary school. I had not yet learned the word “atheist”, and so when classmates asked after my religion, I had nothing to tell them. This was uncomfortable, so I decided to make something up. It involved unicorns, and thankfully, I was never able to quite push myself to the point where I believed any of it.
LOL! Awesome :-)
I have to say I’m surprised by the amount of praise this story is getting.
The main character seems convinced that the difficulty she experiences in interacting pleasantly with members of the opposite sex and possibly starting a relationship with someone less rational than she is, is due to her inability to delude herself, or even to compartmentalize.
But it’s not. It’s due to her inability to shut up once in a while. Instead of working on changing her entire psyche, couldn’t she have simply made an effort to, you know, control the way she behaves?
Epistemic rationality has nothing to do with extreme honesty towards other individuals, or with showing contempt for irrationalists, or even with feeling contempt for them. The greatest epistemic rationalist on Earth could have a happy relationship with a Young Earth Creationist; all s/he’d have to do is either refrain from criticism, or be very polite and gentle about it.
Also, I wasn’t very impressed with the classification of Richard Dawkins (and those like him) as a “a Type-1-and-higher retard”. What he is is a good Type-1-and-higher thinker who cares about the truth and therefore to whom avoiding self-deception is advantageous.
I think you’re misreading the story. It’s not an argument in favor of irrationality, it’s a horror story. The catch is that it’s a good horror story, directed at the rationalist community. Like most good horror stories, it plays off a specific fear of its audience.
You may be immune to the lingering dread created by looking at all those foolish happy people around you and wondering if maybe you are the one doing something wrong. Or the fear that even if you act as rationally as you can, you could still box yourself into a trap you won’t be able to think your way back out of. But quite a few of your peers are not so immune. I know I’m not, and that story managed to scare me pretty effectively.
The protagonist isn’t an ideal rationalist, and the story isn’t trying to assert that this is what the ideal rationalist does. Instead, the protagonist is an adolescent proto-rationalist, of a type many of us are familiar with, with her social instincts sucking her into a trap that a lot of us can understand well enough to dread.
And so there’s a reason she thinks and acts like a Hollywood stereotype of an intelligent person is that, especially when they’re just barely at the age of being able to really think at all. Where do you think Hollywood got the idea for the stereotype in the first place?
I submit that the reason so many of the average people think intelligent people act that way is because they lose social contact with the geniuses in high school, which is when they do think and act like that.
For a lot of the smartest people, being socially functional is a learned skill that comes late and not easily.
Agreed. I went in expecting a parable against rationality, and about halfway through I realized I was reading existential horror (the best kind).
The writing isn’t great and the points are made hamhandedly, but there is the core of a good story here.
I’ve upvoted this comment, but I disagree.
What should make this an effective horror story, as you put it, is that it’s based on the very real possibility that there are people whose brains are wired in such a way that they can’t be happy and rational at the same time. In order to more effectively ‘scare’ the reader, the author attempts to convince us that this is more than a possibility by making an argument by fictional example, the example being the main character.
My beef with the story is that this example is way too unlikely to be convincing as an argument (and therefore scary as a horror story). If there are people who can’t possibly be rational and happy, I’m pretty sure it’s not because they’re incapable of keeping their tongues under control in order to start a relationship on the right foot.
I dunno. I mean, a lot of horror stories that are famous for being good talk about stuff that can never be and should never be, but that nonetheless (in-story) is. I think it’s that sense of a comforting belief about the world being violated that makes a good horror story, even if the prior probability of that belief being wrong is low.
Least convenient possible world for your objection: the protagonist couldn’t change her liability to shut up, but could change her rationality.
That’s possible, but so unlikely that it strains credulity (haha). Like I said in my reply to Kaj Sotala, this story is trying to make a point about some humans who live in the real world. Have you ever heard of someone for whom changing from a ‘hyper-rationalist’ to a deluded fool is easier than to learn to keep one’s mouth shut every now and then?
Yes.
It is also curious to observe that in humans extroversion is strongly correlated with conformism.
Has a study found such a correlation, wedrifid, or are you going by personal observations here?
Yes, more than one study has found such a correlation. My personal observations agree but not to the extent that I would rule out the fairly obvious potential biasses in my perception even if I had noticed it.
(No, I don’t have the reference on hand.)
Thanks! Any idea how the studies you mention measured conformism? I’m guessing they administered questions like “The conventional wisdom is usually right.”
Who said that the characters in good stories must never be mistaken?
Sure, but this isn’t just a story, it’s a story that tries to make a point about the negative consequences of rationality for some humans in the real world. Her belief isn’t merely her belief, it’s also a step in an argument.
I suspect you may be reading too much into it: I thought the “negative consequences of rationality” were more for the sake of pure comedy than for making any kind of a point.
Of course, you may also be right and I might be the one who’s reading too little into it.
false in general, false as a statistical statement too.
The greatest epistemic rationalist on Earth is still made out of meat.
I was making a point about human thought processes, not human desires. I agree that it’s unlikely that the greatest epistemic rationalist would want to have a relationship with a YEC, but if s/he did want to, s/he could.
If I were otherwise unattached, I would totally have a relationship with a YEC, if she was from a world which had actually been created 6000 years ago. Otherwise no.
What if she was just from a world where lots of evidence pointed to it having been created 6,000 years ago, but it was really created last Thursday?
Is she hot?
...okay, that was a bit out of character, but I think that at that point in the thread I basically had no choice but to say that.
have you ever been in a long-term relationship?
Downvoted as thoroughly unhelpful.
Downvoted as uselessly vague.
If it is false they could not. What would prevent them?
Unless you have actually tracked down and interviewed the greatest epistemic rationalist on earth, how do you know? Maybe (s)he is very tolerant of such things. (When does intolerance win on a personal scale?)
This sounds like scientism.
Because I have experience with good rationalists, and the kind of people they have relationships with, and I am a bayesian so I can assign degrees of belief to propositions that I haven’t tested directly. In this case, it seems reasonable that similar people have similar relationship-behaviors, and so my existing knowledge is relevant.
Rather like “how do you know that the fastest dog in the world can’t outrun a formula one car?”—I know this with high certainty because I believe that similar animals behave in similar ways.
I’d like to remind everyone that at one point this strategy was the de facto foreign policy of the United States.
Obligatory link to our Schelling thread.
If only Eric had had a better response to this:
then maybe tragedy would have been averted?
This is an instance of Hollywood-style rationality: it sounds like she’s being “rational,” but she takes a single implication and just stops thinking. This is doubly true because she then cites strong evidence that pretty much all of the guys she’s around want to fuck her, so merely looking at her is a pretty good sign of this fact, so it seems likely that his asking her out is probably indicative of something extra. If the guy in question had been sharp enough to beat her at her own game, I wouldn’t be surprised if she’d then wanted to go out with him.
But this hits a broader point: people with the mental sharpness needed to break a fairly intelligent Hollywood Rationalist are few and far between. Her misery seems to stem more from a lack of like-minded peers than it does from anything about the nature of her position. If she’d found a very sharp rationalist who’d taken interest in her and given her approval, she’d probably have been happy that way. This may say more about the importance of finding a like-minded peer group than about being hyper-rational.
Agreed. Eric displayed so little ability to react to oddness. Perhaps, “Why would you react with revulsion if I expressed the desire to fuck you? If it’s because of cultural expectation or because you would interpret such an expression as an indication that I’m generally abrupt and crass, then the analogy fails to hold.”
Uh, … I’ve wondered since about four years before this post was published why women seem so universally offended by any signs that a man would like to have sex with them, with the possible exception of a man they’re already very very very much into. Especially considering those signs seem to be redundant, since most of the time a man and a woman meet, he’ll want to have sex with her, unless, of course, his sex drive is being satisfied to the point of exhaustion elsewhere.
I’ve never been able to discuss this rationally with anyone. Women usually imply it should be obvious to everyone why they’re offended and there’s nothing else to say about it. Besides, everyone knows I’m talking to the woman only because I’d like to have sex with her, which makes the topic, and me, doubly disgusting.
I find it therefore amazing, though still puzzling, to see a woman suggesting that question.
On the other hand, does anyone here actually frequent environments where you can get away with such a long reply, in an ordinary conversation, without the other person interrupting you, walking away, slapping you or anything like that? I find it hard enough to believe among mature adults, let alone a teenage boy talking to a haughty and exceptionally attractive and smart female peer. That’s yet another reason for me to deeply envy all of you.
(Wow, I was commenting on LW thirteen years ago...) I didn’t suggest saying this out of the blue! My recommended riposte borrows the story protagonist’s vocabulary and tone. If a woman asks you:
then, it may be appropriate to discuss, optionally using the word “fuck”, why she’d react that way if you’d asked that question, which you didn’t, having instead (as in the story) made a much more innocuous suggestion, neither culturally inappropriate nor abrupt and crass.
I understood that much the first time.
I think the protagonist makes an excellent point. In fact, she understates it. She mentions plenty of reasons to expect essentially any heterosexual male classmate to be interested in having sex with her if given the chance, so if she’s going to be revolted by that, she might as well go ahead and be revolted without waiting for anyone to tell her anything.
The obvious way out of this conundrum, as you pointed out, is to question the premise that she should be revolted by her male peers’ sexual desires toward her. This doesn’t seem to stop people in general from agreeing with that premise and deeming it self-evident.
As Ilyssa herself argues, how is focusing on the first domino any more innocuous than admitting the hundredth will fall, too? Is there anything to what you call socially appropriate behavior, not abrupt or crass, beyond a lame attempt to deceive her? In fact, since he’s probably well aware she won’t be fooled, I don’t even know which simulacrum level he’s trying to operate at; likely 4.
I wish I’d been asked that kind of questions, including the ones in the few following paragraphs, while being both young enough for them to make sense and the situation not to be too disgusting, and mature enough to understand them and know my true answers. At Eric and Ilyssa’s age, I’m sure I would have enjoyed and come to crave an unbounded amount of physical intimacy with almost any of my female classmates, provided we got along and could trust each other, but I didn’t really know it. I just knew I wanted to get to know them, necessarily slowly, much like I found out, half a decade later, that women tend to emphasize. Funny; in my case, it probably comes from being perpetually isolated and vulnerable. Also, I didn’t have the vocabulary to know what “if I offered myself to you” means, so I’d just be puzzled by the question and probably look like a hypocrite to everyone. Also, my parents would have put a stop to that kind of relationship sooner rather than later, and I guess hers would have, too. Also, I was no “one of the more desirable seniors”. Okay, I’ll shut up.
A quite good if depressing story. But he ruins the brilliant Flowers for Algernon parallel by pointing it out explicitly.
Very good story. The change in grammar and writing style may be condescending towards a certain group of people, and the character(s) are perhaps a little too much like caricatures, but it’s overall quite interesting and well-written.
It also highlights a rather massive potential disconnect between rationality and utilitarianism. It seems hard to argue that she’s not hedonically happier at the end, but I still find it rather depressing. This isn’t exactly news, but I feel the issue gets swept under the carpet rather often in these parts.
Hedonically happier, huh? A Roissy quote seems appropriate:
I would’ve amended it to say “likely to have furthered your genes’ goals in the ancestral environment”, but the original formulation fits Aaronson’s story like a glove.
Blatantly obvious counterexamples include forced reproduction and rape. Less blatantly obvious counterexamples include masturbation and bacon double-cheeseburgers, though these don’t apply to your amended version. The fact that we have the capacity to experience hedonic pleasure because it was adaptive does not mean we experience hedonic pleasure as a direct result of acting to increase our inclusive genetic fitness.
It may very well fit the story like a glove, but that doesn’t make it any less egregious of a misunderstanding of evolutionary biology. She’s hedonically happier, and she seems to have increased her odds of reproducing; direct causality does not seem present. If her amazing new boyfriend is sterile, it isn’t really going to change the equation that much.
Being raped wasn’t likely to further your genes’ goals in the ancestral environment, because it decreases your chances to reproduce consensually with a better mate and maybe have him provide resources for the kids.
You’re ignoring spousal rape, where a better mate is already only going to be had through the spouse’s death or hazardous subterfuge, and resource support for the kids can indeed be expected.
I’m skeptical of Roissy’s claim, but I’m not sure your examples are as obvious or convincing as you seem to think.
Unless I’m extremely atypical (which is possible), it is incorrect to say that superstimulant foods and masturbation bring happiness. They bring contentment, which is different and less satisfying. To the extent that this is a definitional dispute, the happiness they bring is far weaker and far more evanescent compared to e.g. feeling like part of a group, feeling like you’ve “made a difference”, etc.
Next, regarding forced reproduction, if it had to be forced, it was a kind of reproduction that the victim’s evolved architecture did not judge to be genetically optimal.
Rape is a bit trickier, for a number of reasons, but it’s clear why it would have questionable, not stricktly good impact on genes. As cousin_it mentioned, someone who rapes you is unlikely to stick around to provide resources for the kids. And like with forced reproduction, the fact that it had to be forced is evidence your body doesn’t regard it as furthering your genes.
Now, to play devil’s advocate, there is the Sexy Son Hypothesis, WHICH I DON’T ENDORSE, which says that the actual father’s willingness to commit is irrelevant because you can just rook a productive male into thinking the baby’s his. So being raped means you get a son (if a boy) who can successfully rape other women, who then rook another man into raising that son, etc. But again, against the clear disadvantages of being raped, this at best proves, under Roissy’s theory, why the woman would feel conflicted, not happy.
By the way, this isn’t necessarily the case. Depending on the species, it could also be the case that force is part of the selection process for determining a mate’s suitability. Evolution isn’t a kind and loving god, alas. (Some species’ mating rituals involve the death of one or both partners, for example.)
I mean “forced”, as I think PH did, in the sense of “the being that was forced did not want to do it”. The cases of beetles that get their heads bitten off after mating with a female wouldn’t be a relevant example, since, in the appropriate sense, the male seems to want that.
But then, if resistance is part of some test, than PH’s example wouldn’t be relevant anyway.
Endorse meaning, “consider true”, or endorse meaning, “consider morally good”? I could understand the latter (so few things about evolution can be considered “good”, after all), but don’t get the former. Do you think it’s untrue?
Enough people who read that post won’t be able to tell the difference.
You can figure it out from there, I think.
The only clear implication I can take from that is that you believe the people in this environment are poor at distinguishing between statements of fact and moral assertions when it comes to potentially sensitive topics.
Oh. Hang on. Took me a while but I can figure it out from there after all. I’ve reversed my downvote. Reasonable answer.
My response is to the Roissy quote specifically, which says, “direct result of actions taken that further your genes’ goals” NOT “would have furthered your genes’ goals in the ancestral environment.” Thus, in the case of forced reproduction, if you were locked in a small room and given just enough food to survive, but then given the opportunity to reproduce with a fertile partner every single time that you were capable of causing conception, you would (I’d hope) not be happy, even though you would produce many, many more times the surviving offspring (if you’re male, literally thousands of times the surviving offspring). It seems almost trivially obvious that objectively maximizing inclusive genetic fitness will not make you happy.
The amendment “tend to maximize… in the ancestral environment” changes the equation completely. First, it’s very prone to “just-so” stories; one can invent a decent explanation as to why practically any action increases genetic fitness (i.e. celibacy is for aunt/unclehood). Second, because the cause is tremendously indirect, it isn’t realistic to say that the cause of someone’s increased happiness is the fact that they are (or would be in the AE) more genetically fit, as outlined in the EY post I linked. There’s a great deal of truth to it, but it’s both indirect and tremendously imprecise. And, more generally, if someone asks “How can I be happier?” the answer is seldom, “Well, think about what would maximize your inclusive genetic fitness in the ancestral environment, and then do that.”
On a related note, my problem with a lot of Roissy’s reasoning is that he views people in terms of rather binary “objective” Platonic ideals of “male” and “female” and does not account for either innate or learned differences, instead dismissing such as trivial, which is quite problematic, because a great diversity of preferences actually exist.
Edit: Also, wikipedia’s phrasing of the sexy son hypothesis contradicts what I’ve read elsewhere. The idea is that if females generally prefer trait X, then females should choose males with trait X, even if trait X is completely superficial, because then their sons will be more successful with future females. That doesn’t make non-superficial trait Y irrelevant, and there is no basis in reason nor a citation claiming it should. The hypothesis is simply an explanation of how an expensive superficial trait can become dominant; it has nothing (explicitly) to do with cuckoldry or anything like that.
Few quotes from the article:
“Bananas were created by God for human enjoyment, for why else would they come in such convenient cases?”
″...Am I to hope that, in the hereafter, a rationalist God will reward me for having the intellectual integrity not to believe in Him?”.....
...Right, but if the only reason it works is that you believe it works, then how can it work if you know it only works because you believe it works?”...
...”Richard Dawkins, the biologist, was once asked about a study claiming that the devout live longer on average than atheists. He replied that, even if that were so, he’d rather know the truth about where he came from and die early than live longer under a fantasy.”
. …. “In other words: the stupider, more ignorant, more irrational you can prove you are, the better the chance you have of winning”.
When Douglas Yates wrote that “people who are sensible about love are incapable of it,” he might have added a footnote: “the Darwinian explanation for this fact arises from certain paradoxes of rationality in games played by agents known to each other to have bounded computational capacity.”
Oldie but goodie. Maybe emphasize the link a bit harder, took me a moment to find it.
Done.