Scott Alexander

Karma: 42,078

Scott Alexander 17 Oct 2021 22:08 UTC
390 points
on: My experience at and around MIRI and CFAR (inspired by Zoe Curzi’s writeup of experiences at Leverage)
I want to add some context I think is important to this.
Jessica was (I don’t know if she still is) part of a group centered around a person named Vassar, informally dubbed “the Vassarites”. Their philosophy is complicated, but they basically have a kind of gnostic stance where regular society is infinitely corrupt and conformist and traumatizing and you need to “jailbreak” yourself from it (I’m using a term I found on Ziz’s discussion of her conversations with Vassar; I don’t know if Vassar uses it himself). Jailbreaking involves a lot of tough conversations, breaking down of self, and (at least sometimes) lots of psychedelic drugs.
Vassar ran MIRI a very long time ago, but either quit or got fired, and has since been saying that MIRI/CFAR is also infinitely corrupt and conformist and traumatizing (I don’t think he thinks they’re worse than everyone else, but I think he thinks they had a chance to be better, they wasted it, and so it’s especially galling that they’re just as bad). Since then, he’s tried to “jailbreak” a lot of people associated with MIRI and CFAR—again, this involves making them paranoid about MIRI/CFAR and convincing them to take lots of drugs. The combination of drugs and paranoia caused a lot of borderline psychosis, which the Vassarites mostly interpreted as success (“these people have been jailbroken out of the complacent/conformist world, and are now correctly paranoid and weird”). Occasionally it would also cause full-blown psychosis, which they would discourage people from seeking treatment for, because they thought psychiatrists were especially evil and corrupt and traumatizing and unable to understand that psychosis is just breaking mental shackles.
(I am a psychiatrist and obviously biased here)
Jessica talks about a cluster of psychoses from 2017 − 2019 which she blames on MIRI/CFAR. She admits that not all the people involved worked for MIRI or CFAR, but kind of equivocates around this and says they were “in the social circle” in some way. The actual connection is that most (maybe all?) of these people were involved with the Vassarites or the Zizians (the latter being IMO a Vassarite splinter group, though I think both groups would deny this characterization). The main connection to MIRI/CFAR is that the Vassarites recruited from the MIRI/CFAR social network.
I don’t have hard evidence of all these points, but I think Jessica’s text kind of obliquely confirms some of them. She writes:
“Psychosis” doesn’t have to be a bad thing, even if it usually is in our society; it can be an exploration of perceptions and possibilities not before imagined, in a supportive environment that helps the subject to navigate reality in a new way; some of R.D. Liang’s work is relevant here, describing psychotic mental states as a result of ontological insecurity following from an internal division of the self at a previous time. Despite the witch hunts and so on, the Leverage environment seems more supportive than what I had access to. The people at Leverage I talk to, who have had some of these unusual experiences, often have a highly exploratory attitude to the subtle mental realm, having gained access to a new cognitive domain through the experience, even if it was traumatizing.
RD Laing was a 1960s pseudoscientist who claimed that schizophrenia is how “the light [begins] to break through the cracks in our all-too-closed minds”. He opposed schizophrenics taking medication, and advocated treatments like “rebirthing therapy” where people role-play fetuses going through the birth canal—for which he was stripped of his medical license. The Vassarites like him, because he is on their side in the whole “actually psychosis is just people being enlightened as to the true nature of society” thing. I think Laing was wrong, psychosis is actually bad, and that the “actually psychosis is good sometimes” mindset is extremely related to the Vassarites causing all of these cases of psychosis.
Unless there were psychiatric institutionalizations or jail time resulting from the Leverage psychosis, I infer that Leverage overall handled their metaphysical weirdness better than the MIRI/CFAR adjacent community. While in Leverage the possibility of subtle psychological influence between people was discussed relatively openly, around MIRI/CFAR it was discussed covertly, with people being told they were crazy for believing it might be possible. (I noted at the time that there might be a sense in which different people have “auras” in a way that is not less inherently rigorous than the way in which different people have “charisma”, and I feared this type of comment would cause people to say I was crazy.) As a consequence, the people most mentally concerned with strange social metaphysics were marginalized, and had more severe psychoses with less community support, hence requiring normal psychiatric hospitalization.
Jessica is accusing MIRI of being insufficiently supportive to her by not taking her talk about demons and auras seriously when she was borderline psychotic, and comparing this to Leverage, who she thinks did a better job by promoting an environment where people accepted these ideas. I think MIRI was correct to be concerned and (reading between the lines) telling her to seek normal medical treatment, instead of telling her that demons were real and she was right to worry about them, and I think her disagreement with this is coming from a belief that psychosis is potentially a form of useful creative learning. While I don’t want to assert that I am 100% sure this can never be true, I think it’s true rarely enough, and with enough downside risk, that treating it as a psychiatric emergency is warranted.
On the two cases of suicide, Jessica writes:
Both these cases are associated with a subgroup splitting off of the CFAR-centric rationality community due to its perceived corruption, centered around Ziz. (I also thought CFAR was pretty corrupt at the time, and I also attempted to split off another group when attempts at communication with CFAR failed; I don’t think this judgment was in error, though many of the following actions were; the splinter group seems to have selected for high scrupulosity and not attenuated its mental impact.)
Ziz tried to create an anti-CFAR/MIRI splinter group whose members had mental breakdowns. Jessica also tried to create an anti-CFAR/MIRI splinter group and had a mental breakdown. This isn’t a coincidence—Vassar tried his jailbreaking thing on both of them, and it tended to reliably produce people who started crusades against MIRI/CFAR, and who had mental breakdowns. Here’s an excerpt from Ziz’s blog on her experience (edited heavily for length, and slightly to protect the innocent):
When I first met Vassar, it was a random encounter in an experimental group call organized by some small-brand rationalist. He talked for about an hour, and automatically became the center of conversation, I typed notes as fast as I could, thinking, “if this stuff is true it changes everything; it’s the [crux] of my life.” (It true, but I did not realize it immediately.) Randomly, another person found the link, came in and said, “hi”. [Vassar] said “hi”, she said “hi” again, apparently for humor. [Vassar] said something terse I forget “well if this is what …”, apparently giving up on the venue, and disconnected without further comment. One by one, the other ~10 people including besides her, including me disconnected disappointedly, wordlessly or just about right after. A wizard was gracing us with his wisdom and she fucked it up. And in my probably-representative case that was just about the only way I could communicate how frustrated I was at her for that.
[Vassar explained how] across society, the forces of gaslighting were attacking people’s basic ability to think and to a justice as a Schelling point until only the built-in Schelling points of gender and race remained, Vassar listed fronts in the war on gaslighting, disputes in the community, and included [local community member ZD] [...] ZD said Vassar broke them out of a mental hospital. I didn’t ask them how. But I considered that both badass and heroic. From what I hear, ZD was, probably as with most, imprisoned for no good reason, in some despicable act of, “get that unsightly person not playing along with the [heavily DRM’d] game we’ve called sanity out of my free world”.
I heard [local community member AM] was Vassar’s former “apprentice”. And I had started picking up jailbroken wisdom from them secondhand without knowing where it was from. But Vassar did it better. After Rationalist Fleet, I concluded I was probably worth Vassar’s time to talk to a bit, and I emailed him, carefully briefly stating my qualifications, in terms of ability to take ideas seriously and learn from him, so that he could get maximally dense VOI on whether to talk to me. A long conversation ensued. And I got a lot from it. [...]
Vassar has had, I think about 6, transfems gravitate to him, join his projects, go on his quests, that I’ve heard. Including Olivia, and Jessica, and I think Devi. Devi had a mental breakdown and detransitioned IIHC. Jessica had a mental breakdown and didn’t detransition. Olivia became an agent of mental breakdown, compulsively breaking others via drug trips because they went through gates they shouldn’t’ve. And didn’t detransition. This all created an awful tension in me. The rationality community was kind of compromised as a rallying point for truthseeking. This was desperately bad for the world. [Vassar] was at the center of, largely the creator of a “no actually for real” rallying point for the jailbroken reality-not-social-reality version of this.
Ziz is describing the same cluster of psychoses Jessica is (including Jessica’s own), but I think doing so more accurately, by describing how it was a Vassar-related phenomenon. I would add Ziz herself to the list of trans women who got negative mental effects from Vassar, although I think (not sure) Ziz would not endorse my description of her as having these.
What was the community’s response to this? I have heard rumors that Vassar was fired from MIRI a long time ago for doing some very early version of this, although I don’t know if it’s true. He was banned from REACH (and implicitly rationalist social events) for somewhat unrelated reasons. I banned him from SSC meetups for a combination of reasons including these. For reasons I don’t fully understand and which might or might not be related to this, he left the Bay Area. This was around the time COVID happened, so everything’s kind of been frozen in place since then.
I want to clarify that I don’t dislike Vassar, he’s actually been extremely nice to me, I continue to be in cordial and productive communication with him, and his overall influence on my life personally has been positive. He’s also been surprisingly gracious about the fact that I go around accusing him of causing a bunch of cases of psychosis. I don’t think he does the psychosis thing on purpose, I think he is honest in his belief that the world is corrupt and traumatizing (which at the margin, shades into values of “the world is corrupt and traumatizing” which everyone agrees are true) and I believe he is honest in his belief that he needs to figure out ways to help people do better. There are many smart people who work with him and support him who have not gone psychotic at all. I don’t think we need to blame/ostracize/cancel him and his group, except maybe from especially sensitive situations full of especially vulnerable people. My main advice is that if he or someone related to him asks you if you want to take a bunch of drugs and hear his pitch for why the world is corrupt, you say no.
EDIT/UPDATE: I got a chance to talk to Vassar, who disagrees with my assessment above. We’re still trying to figure out the details, but so far, we agree that there was a cluster of related psychoses around 2017, all of which were in the same broad part of the rationalist social graph. Features of that part were—it contained a lot of trans women, a lot of math-y people, and some people who had been influenced by Vassar, although Vassar himself may not have been a central member. We are still trying to trace the exact chain of who had problems first and how those problems spread. I still suspect that Vassar unwittingly came up with some ideas that other people then spread through the graph. Vassar still denies this and is going to try to explain a more complete story to me when I have more time.
What links here?

Scott Alexander 11 Jul 2019 5:46 UTC
229 points
on: The AI Timelines Scam
1. For reasons discussed on comments to previous posts here, I’m wary of using words like “lie” or “scam” to mean “honest reporting of unconsciously biased reasoning”. If I criticized this post by calling you a liar trying to scam us, and then backed down to “I’m sure you believe this, but you probably have some bias, just like all of us”, I expect you would be offended. But I feel like you’re making this equivocation throughout this post.
2. I agree business is probably overly optimistic about timelines, for about the reasons you mention. But reversed stupidity is not intelligence. Most of the people I know pushing short timelines work in nonprofits, and many of the people you’re criticizing in this post are AI professors. Unless you got your timelines from industry, which I don’t think many people here did, them being stupid isn’t especially relevant to whether we should believe the argument in general. I could find you some field (like religion) where people are biased to believe AI will never happen, but unless we took them seriously before this, the fact that they’re wrong doesn’t change anything.

3. I’ve frequently heard people who believe AI might be near say that their side can’t publicly voice their opinions, because they’ll get branded as loonies and alarmists, and therefore we should adjust in favor of near-termism because long-timelinists get to unfairly dominate the debate. I think it’s natural for people on all sides of an issue to feel like their side is uniquely silenced by a conspiracy of people biased towards the other side. See Against Bravery Debates for evidence of this.

4. I’m not familiar with the politics in AI research. But in medicine, I’ve noticed that doctors who go straight to the public with their controversial medical theory are usually pretty bad, for one of a couple of reasons. Number one, they’re usually wrong, people in the field know they’re wrong, and they’re trying to bamboozle a reading public who aren’t smart enough to figure out that they’re wrong (but who are hungry for a “Galileo stands up to hidebound medical establishment” narrative). Number two, there’s a thing they can do where they say some well-known fact in a breathless tone, and then get credit for having blown the cover of the establishment’s lie. You can always get a New Yorker story by writing “Did you know that, contrary to what the psychiatric establishment wants you to believe, SOME DRUGS MAY HAVE SIDE EFFECTS OR WITHDRAWAL SYNDROMES?” Then the public gets up in arms, and the psychiatric establishment has to go on damage control for the next few months and strike an awkward balance between correcting the inevitable massive misrepresentations in the article while also saying the basic premise is !@#$ing obvious and was never in doubt. When I hear people say something like “You’re not presenting an alternative solution” in these cases, they mean something like “You don’t have some alternate way of treating diseases that has no side effects, so stop pretending you’re Galileo for pointing out a problem everyone was already aware of.” See Beware Stephen Jay Gould for Eliezer giving an example of this, and Chemical Imbalance and the followup post for me giving an example of this. I don’t know for sure that this is what’s going on in AI, but it would make sense.

I’m not against modeling sociopolitical dynamics. But I think you’re doing it badly, by taking some things that people on both sides feel, applying it to only one side, and concluding that means the other is involved in lies and scams and conspiracies of silence (while later disclaiming these terms in a disclaimer, after they’ve had their intended shocking effect).

I think this is one of the cases where we should use our basic rationality tools like probability estimates. Just from reading this post, I have no idea what probability Gary Marcus, Yann LeCun, or Steven Hansen has on AGI in ten years (or fifty years, or one hundred years). For all I know all of them (and you, and me) have exactly the same probability and their argument is completely political about which side is dominant vs. oppressed and who should gain or lose status (remember the issue where everyone assumes LWers are overly certain cryonics will work, whereas in fact they’re less sure of this than the general population and just describe their beliefs differently ). As long as we keep engaging on that relatively superficial monkey-politics “The other side are liars who are silencing my side!” level, we’re just going to be drawn into tribalism around the near-timeline and far-timeline tribes, and our ability to make accurate predictions is going to suffer. I think this is worse than any improvement we could get by making sociopolitical adjustments at this level of resolution.
What links here?
- [AN #60] A new AI challenge: Minecraft agents that assist human players in creative mode by Rohin Shah (22 Jul 2019 17:00 UTC; 23 points)

Scott Alexander 15 May 2009 11:06 UTC
193 points
on: Cheerios: An “Untested New Drug”
This is a beautiful example of politics at work.

Cheerios claims on its box that it can “lower your cholesterol four percent in six weeks”. This is false. It is based on a “study” sponsored by General Mills where subjects took more than half their daily calories from Cheerios (apparently they ate nothing but Cheerios for two of their three daily meals). No one eating Cheerios in anything resembling a normal way would get close to this effect; therefore, it is false and misleading advertising. If General Mills wants to market Cheerios as a drug, it needs to meet the normal standards for drug evidence, and it doesn’t.

So far, so good. Either you approve of the FDA’s decision and think it’s important to hold cereal companies to a high level of accuracy, or you think they should relax their standards and allow more leeway to food advertisers. Either one would be a legitimate response. But look at what happens:

A few sources correctly title the story, eg “Cheerios Aren’t A Drug, FDA Says”. The majority choose to go the other way and title it something more inflammatory like “Popular Cereal Is A Drug, US Food Watchdog Says”, which is of course the opposite of what it said. It’s the inflammatory outrageous headlines that get put on blogs and Reddit (now reworded further to “WTF? FDA says Cheerios are a drug”)

Then all the usual suspects take the mistitled blown-out-of-proportion story and look to see whether or not this supports their preferred political narrative. For example:

Independent Institute (a libertarian think tank):

“The Obama administration blunders onward in its “progressive” (i.e., authoritarian) absurdities.”

Freedom News Blitz (“The News Freedom Lovers Devour”):

“Do you get the feeling that since we are experiencing a severe recession, the Food and Drug Administration is running out of honest businesses to harass and persecute?...Certainly, it appears the old virtues of free enterprise, hard work, self-discipline, saving money, weighing the short-term consequences against the long-term consequences, etc. are unpalatable to the vast majority of people. So they allow the false promises of politicians, civic leaders and pseudo-intellectuals to mislead them.

So yes, we went from “please don’t use poorly designed made-up studies to make spurious medical claims” to “hard work is unpalatable to the vast majority of people” in two steps. Remarkable, ne?

And then the media goes into one of its periodic vicious feedback loops and started reporting on the reporting, adding a little more liberal bent at each pass. From Reuters, here’s Cheerios, Cereal of Liberty. The article starts with “Disputes over food-label claims are always political” and quickly moves onto “But the current, insane iteration of the American right has walked several steps past the crazy line...For them, wholesome, “American” foods are a-OK. Eurocommie foods are right out.”

Three steps and now we’re at “Eurocommie.”

In the last phase of the decline from “reasonable question about cholesterol lowering properties of cereal” to “complete proof humankind as a species is doomed”, someone opens the floodgates to Mordor and a horde of semi-human blog commenters swarm out, ready to add their “opinion” to the “discussion”. From here and here:

“Ha—I guess the makers of the liver corroding, memory erasing cholesterol drugs, must have whined about this to their best friends, (the guys they pay off to legalize these poisons) the FDA.”

“Sorry mister.g but you folks are delusional in the USA. You buy into everything and I mean everything. That is why you are the sickest, poorest industrialized economy going and you can inject GOLD into dog poop but it will still be dog poop. If you think it’s worth eating to get to that golden nugget in the center you go right ahead. Unbelievable.”

“Unapproved though (non-Obama worship) is not approved by your government overlords. Americas are too stupid to understand what is a cereal and what is a drug. You will be eliminated.”

Four steps from “Don’t lie about cholesterol on your cereal box, please,” to “Obama will kill everyone who disagrees with him.”
What links here?

Scott Alexander 3 Nov 2011 23:22 UTC
162 points
in reply to: NancyLebovitz’s comment on: Rational Romantic Relationships, Part 1: Relationship Styles and Attraction Basics
This is a terrible debate and you should all feel bad for having it. Now let me join in.

The research on this topic is split into “completely useless” and “mostly useless”. In the former category we have studies that, with a straight face, purport to show that women like nice guys by asking women to self-report on their preferences. To illuminate just how silly this is, consider the mirror case of asking men “So, do you like witty charming girls with good personalities, or supermodels with big breasts?” When this was actually done, men rated “physical attractiveness” only their 22nd most important criterion for a mate—number one was “sincerity”, and number nineteen was “good manners”. And yet there are no websites where you can spend $9.95 per month to stream videos of well-mannered girls asking men to please pass the salad fork, and there are no spinster apartments full of broken-hearted supermodels who just didn’t have enough sincerity. So self-reports are right out.

Other-reports may be slightly less silly. Herold and Milhausen, 1999, found that 56% of university women believed that women in general were more likely to date jerks than nice guys. But although women may have less emotional investment in the issue than men, their opinions are still just opinions.

The few studies that earn the coveted accolade of “only mostly useless” are those that try to analyze actual behavior. Bogart and Fisher typify a group of studies that show that good predictors of a man’s number of sexual partners include disinhibitedness, high testosterone levels, “hypermasculinity”, “sensation seeking”, antisocial personality, and extraversion. Meston et al typify a separate group of studies on sex and the Big Five traits when she says that “agreeableness was the most consistent predictor of behavior...disagreeable men and women were more likely to have had sexual intercourse and with a greater number of partners than agreeable men and women. Nonvirgins of both sexes were more likely to be calculating, stubborn, and arrogant in their interpersonal behavior than virgins. Neuroticism predicted sexual experience in males only; timid, unassertive men were less sexually experienced than emotionally stable men...the above findings were all statistically significant at p<.01″

These studies certainly show that jerkishness is associated with high number of sexual partners, but they’re not quite a victory for the “nice guys finish last” camp for a couple of reasons. First, men seem to come off almost as bad as women do. Second, there’s no reason to think that any particular “nice” woman will like jerks; many of the findings could be explained by disagreeable men hooking up with disagreeable women, disagreeing with them about things (as they do) and then breaking up and hooking up with other disagreeable women, while the agreeable people form stable pair bonds. Boom—disagreeable people showing more sexual partners than agreeable people.

I find more interesting the literature about intelligence and sexual partners. In high-schoolers, each extra IQ point increases chance of virginity by 2.7% for males and 1.7% by females. 87% of 19-year old US college students have had sex, yet only 65% of MIT graduate students have had sex. There’s conflicting research about whether this reflects lower sex drive in these people or less sexual success; it’s probably a combination of both. See linked article for more information.

The basic summary of the research seems to be that smart, agreeable people complaining that they have less sex than their stupid, disagreeable counterparts are probably right, and that this phenomenon occurs both in men and women but is a little more common in men.

Moving from research to my own observations, I do think there are a lot of really kind, decent, shy, nerdy men who can’t find anyone who will love them because they radiate submissiveness and non-assertiveness, and women don’t find this attractive. Most women do find dominant, high-testosterone people attractive, and dominance and testosterone are risk factors for jerkishness, but not at all the same thing and women can’t be blamed for liking people with these admittedly attractive characteristics.

There are also a lot of really kind, decent, shy, nerdy women who can’t find anyone who will love them because they’re not very pretty. Men can’t be blamed for liking people they find attractive either, but this is also sad.

But although these two situations are both sad, at the risk of being preachy I will say one thing. When a girl is charming and kind but not so conventionally attractive, and men avoid her, and this makes her sad...well, imagine telling her that only ugly people would think that, and since she’s ugly she doesn’t deserve a man, and she probably just wants to use him for his money anyway because of course ugly women can’t genuinely want love in the same way anyone else would (...that would be unfair!) This would be somewhere between bullying and full on emotional abuse, the sort of thing that would earn you a special place in Hell.

Whereas when men make the same complaint, that they are nice and compassionate but not so good at projecting dominance, there is a very large contingent of people, getting quite a lot of respect and validation from the parts of society that should know better, who immediately leap out to do their best to make them feel miserable—to tell that they don’t deserve a relationship, that they’re probably creeps who are only in it for the sex and that if they were a real man they’d stop whining about being “entitled to sex”.

EDIT: But see qualification here
What links here?
- Evan_Gaensbauer's comment on Open Thread, May 25 - May 31, 2015 by Gondolinian (27 May 2015 1:22 UTC; 0 points)

Scott Alexander 22 Mar 2009 21:07 UTC
147 points
0
on: Eliezer Yudkowsky Facts
Ooh, this is fun.

Robert Aumann has proven that ideal Bayesians cannot disagree with Eliezer Yudkowsky.
Eliezer Yudkowsky can make AIs Friendly by glaring at them.
Angering Eliezer Yudkowsky is a global existential risk
Eliezer Yudkowsky thought he was wrong one time, but he was mistaken.
Eliezer Yudkowsky predicts Omega’s actions with 100% accuracy
An AI programmed to maximize utility will tile the Universe with tiny copies of Eliezer Yudkowksy.
What links here?
- Emile's comment on Less Wrong: Open Thread, September 2010 by matt (8 Sep 2010 22:29 UTC; 9 points)

Scott Alexander 14 Oct 2010 19:03 UTC
132 points
0
on: The Strangest Thing An AI Could Tell You
On any task more complicated than sheer physical strength, there is no such thing as inborn talent or practice effects. Any non-retarded human could easily do as well as the top performers in every field, from golf to violin to theoretical physics. All supposed “talent differential” is unconscious social signaling of one’s proper social status, linked to self-esteem.

A young child sees how much respect a great violinist gets, knows she’s not entitled to as much respect as that violinist, and so does badly at violin to signal cooperation with the social structure. After practicing for many years, she thinks she’s signaled enough dedication to earn some more respect, and so plays the violin better.

“Child prodigies” are autistic types who don’t understand the unspoken rules of society and so naively use their full powers right away. They end out as social outcasts not by coincidence but as unconscious social punishment for this defection.
What links here?
- A1987dM's comment on Open Thread, May 1-14, 2013 by whpearson (2 May 2013 18:54 UTC; 9 points)

Scott Alexander 6 Jan 2014 23:53 UTC
131 points
on: [LINK] Why I’m not on the Rationalist Masterlist
Since it has suddenly become relevant, here are two results from this year’s survey (data still being collected):

When asked to rate feminism on a scale of 1 (very unfavorable) to 5 (very favorable), the most common answer was 5 and the least common answer was 1. The mean answer was 3.82, and the median answer was 4.

When asked to rate the social justice movement on a scale of 1 (very unfavorable) to 5 (very favorable), the most common answer was 5 and the least common answer was 1. The mean answer was 3.61, and the median answer was 4.

In Crowder-Meyer (2007), women asked to rate their favorability of feminism on a 1 to 100 scale averaged 52.5, which on my 1 to 5 scale corresponds to a 3.1. So the average Less Wronger is about 33% more favorably disposed towards the feminist movement than the average woman (who herself is slightly more favorably disposed than the average man).

I can’t find a similar comparison question for social justice favorability, but I expect such a comparison would turn out the same way.

If this surprises you, update your model.
What links here?
- satt's comment on New Year’s Prediction Thread (2014) by Thomas (8 Jan 2014 1:21 UTC; 0 points)

Scott Alexander 9 Dec 2019 20:18 UTC
128 points
1
on: Is Rationalist Self-Improvement Real?
I have some pretty complicated thoughts on this, and my heart isn’t really in responding to you because I think some things are helpful for some people, but a sketch of what I’m thinking:

First, a clarification. Some early claims—like the ones I was responding to in my 2009 essay—were that rationalists should be able to basically accomplish miracles, become billionaires with minimal work, unify physics with a couple of years of study, etc. I still occasionally hear claims along those lines. I am still against those, but I interpret you as making weaker claims, like that rationalists can be 10% better at things than nonrationalists, after putting in a decent amount of work. I’m less opposed to those claims, especially if “a decent amount of work” is interpreted as “the same amount of work you would need to get good at those things through other methods”. But I’m still a little bit concerned about them.
First: I’m interpreting “rationalist self-help” to mean rationalist ideas and practices that are helpful for getting common real-life goals like financial, social, and romantic success. I’m not including things like doing charity better, for reasons that I hope will become clear later.

These are the kinds of things most people want, which means two things. First, we should expect a lot of previous effort has gone into optimizing them. Second, we should expect that normal human psychology is designed to optimize them. If we’re trying to do differential equations, we’re outside our brain’s design specs; if we’re trying to gain status and power, we’re operating exactly as designed.

When the brain fails disastrously, it tends to be at things outside the design specs, that don’t matter for things we want. For example, you quoted me describing some disastrous failures in people understanding some philosophy around atheism, and I agree that sort of thing happens often. But this is because it’s outside of our common sense. I can absolutely imagine a normal person saying “Since I can’t prove God doesn’t exist, God must exist”, but it would take a much more screwed-up person to think “Since I can’t prove I can’t fly, I’m going to jump off this cliff.”

Another example: doctors fail miserably on the Bayes mammogram problem, but usually handle actual breast cancer diagnosis okay. And even diagnosing breast cancer is a little outside common sense and everyday life. Faced with the most chimpish possible version of the Bayes mammogram problem—maybe something like “This guy I met at a party claims he’s the king of a distant country, and admittedly he is wearing a crown, but what’s the chance he’s *really* a king?” my guess is people are already near-optimal.

If you have this amazing computer perfectly-tuned for finding strategies in a complex space, I think your best bet is just to throw lots and lots of training data at it, then try navigating the complex space.

I think it’s ironic that you use practicing basketball as your example here, because rationalist techniques very much are *not* practice. If you want to become a better salesman, practice is going out and trying to make lots of sales. I don’t think this is a “rationalist technique” and I think the kind of self-help you’re arguing for is very different (though it may involve better ways to practice). We both agree that practice is useful; I think our remaining disagreement is on whether there are things other than practice that are more useful to do, on the margin, than another unit of practice.

Why do I think this is unlikely?

1. Although rationalists have done pretty well for themselves, they don’t seem to have done too remarkably well. Even lots of leading rationalist organizations are led by people who haven’t put particular effort into anything you could call rationalist self-help! That’s really surprising!

2. Efficient markets. Rationalists developed rationalist self-help by thinking about it for a while. This implies that everyone else left a $100 bill on the ground for the past 4000 years. If there were techniques to improve your financial, social, and romantic success that you could develop just by thinking about them, the same people who figured out the manioc detoxification techniques, or oracle bone randomization for hunting, or all the other amazingly complex adaptations they somehow developed, would have come up with them. Even if they only work in modern society, one of the millions of modern people who wanted financial, social, and romantic success before you would have come up with them. Obviously this isn’t 100% true—someone has to be the first person to discover everything—but you should expect the fruits here to be very high up, high enough that a single community putting in a moderate amount of effort shouldn’t be able to get too many of them.
(some of this becomes less relevant if your idea of rationalist self-help is just collecting the best self-help from elsewhere and giving it a stamp of approval, but then some of the other considerations apply more.)

3. Rationalist self-help starts looking a lot like therapy. If we’re trying to make you a more successful computer programmer using something other than studying computer programming, it’s probably going to involve removing mental blocks or something. Therapy has been pretty well studied, and the most common conclusion is that it is mostly nonspecific factors and the techniques themselves don’t seem to have any special power. I am prepared to suspend this conclusion for occasional miracles when extremely charismatic therapists meet exactly the right patient and some sort of non-scaleable flash of lightning happens, but this also feels different from “the techniques do what they’re supposed to”. If rationalists are trying to do therapy, they are competing with a field of tens of thousands of PhD-level practitioners with all the resources of the academic and health systems who have worked on the problem for decades. This is not the kind of situation that encourages me we can make fast progress. See https://slatestarcodex.com/2019/11/20/book-review-all-therapy-books/ for more on this.

4. General skepticism of premature practical application. It took 300 years between Harvey discovering the circulatory system and anyone being very good at treating circulatory disease. It took 50 years between Pasteur discovering germ theory and anyone being very good at treating infections. It took 250 years between Newton discovering gravity and anyone being very good at flying. I have a lower prior than you on good science immediately translating into useful applications. And I am just not too impressed with the science here. Kahneman and Tversky discovered a grab bag of interesting facts, some of which in retrospect were false. I still don’t think we’re anywhere near the deep understanding of rationality that would make me feel happy here.

This doesn’t mean I think rationality is useless. I think there are lots of areas outside our brain’s normal design specs where rationality is really useful. And because these don’t involve getting sex or money, there’s been a lot less previous exploration of the space and the low hanging fruits haven’t been gobbled up. Or, when the space has been explored, people haven’t done a great job formalizing their insights, or they haven’t spread, or things like that. I am constantly shocked by how much really important knowledge there is sitting around that nobody knows about or thinks about because it doesn’t have an immediate payoff.

Along with all of this, I’m increasingly concerned that anything that has payoff in sex or money is an epistemic death zone. Because you can make so much money teaching it, it attracts too much charlatanry to navigate easily, and it subjects anyone who enters to extreme pressure to become charlatan-adjacent. Because it touches so closely on our emotions and sense of self-worth, it’s a mind-killer in the same way politics are. Because everybody is so different, there’s almost irresistible pressure to push the thing that saved your own life, without checking whether it will help anyone else. Because it’s such a noisy field and RCTs are so hard, I don’t trust us to be able to check our intuitions against reality. And finally, I think there are whole things lurking out there of the approximate size and concerningness of “people are homeostasis-preserving control systems which will expend their entire energy on undoing any change you do to them” that we just have no idea about even though they have the potential to make everything in this sphere useless if we don’t respond to them.

I actually want to expand on the politics analogy. If someone were to say rationality was great at figuring out whether liberalism or conservatism was better, I would agree that this is the sort of thing rationality should be able to do, in principle. But it’s such a horrible topic that has the potential to do so much damage to anyone trying to navigate it that I would be really nervous about it—about whether we were really up to the task, and about what it would do to our movement if we tried. These are some of my concerns around self-help too.

Scott Alexander 22 Mar 2009 21:53 UTC
124 points
0
in reply to: badger’s comment on: Eliezer Yudkowsky Facts
Eliezer Yudkowsky’s map is more accurate than the territory.

Scott Alexander 8 Dec 2014 1:26 UTC
115 points
on: PSA: Eugine_Nier evading ban?
Both accounts’ SSC comments come from the same IP.

Scott Alexander 10 Mar 2022 9:37 UTC
113 points
0
on: We’re already in AI takeoff
I wasn’t convinced of this ten years ago and I’m still not convinced.
When I look at people who have contributed most to alignment-related issues—whether directly, like Eliezer Yudkowsky and Paul Christiano—or theoretically, like Toby Ord and Katja Grace—or indirectly, like Sam Bankman-Fried and Holden Karnofsky—what all of these people have in common is focusing mostly on object-level questions. They all seem to me to have a strong understanding of their own biases, in the sense that gets trained by natural intelligence, really good scientific work, and talking to other smart and curious people like themselves. But as far as I know, none of them have made it a focus of theirs to fight egregores, defeat hypercreatures, awaken to their own mortality, refactor their identity, or cultivate their will. In fact, all them (except maybe Eliezer) seem like the kind of people who would be unusually averse to thinking in those terms. And if we pit their plumbing or truck-manuevering skills against those of an average person, I see no reason to think they would do better (besides maybe high IQ and general ability).
It’s seemed to me that the more that people talk about “rationality training” more exotic than what you would get at a really top-tier economics department, the more those people tend to get kind of navel-gazey, start fighting among themselves, and not accomplish things of the same caliber as the six people I named earlier. I’m not just saying there’s no correlation with success, I’m saying there’s a negative correlation.
(Could this be explained by people who are naturally talented not needing to worry about how to gain talent? Possibly, but this isn’t how it works in other areas—for example, all top athletes, no matter how naturally talented, have trained a lot.)
You’ve seen the same data I have, so I’m curious what makes you think this line of research/thought/effort will be productive.

Scott Alexander 25 Dec 2013 5:49 UTC
112 points
in reply to: HatCloak’s comment on: Open thread for December 24-31, 2013
All right, I’ll look through my old stuff later this week, find a very few embarrassing or controversial things I want to hide, and unlock the rest.

Scott Alexander 25 Sep 2020 7:21 UTC
108 points
on: The rationalist community’s location problem
The Bay Area is a terrible place to live in many ways. I think if we were selecting for the happiness of existing rationalists, there’s no doubt we should be somewhere else.
But if the rationalist project is supposed to be about spreading our ideas and achieving things, it has some obvious advantages. If MIRI is trying to lure some top programmer, it’s easier for them to suggest they move to the Bay (and offer them enough money to overcome the house price hurdle) than to suggest they move to Montevideo or Blackpool or even Phoenix. If CEA is trying to get people interested in effective altruism, getting to socialize with Berkeley and Stanford professors is a pretty big plus. And if we’re trying to get the marginal person who isn’t quite a community member yet but occasionally reads Less Wrong to integrate more, that person is more likely to be in the Bay than anywhere else we could move. I think this is still true despite the coronavirus and fires. Maybe it’s becoming less so, but it’s hard to imagine any alternative hub that’s anywhere near as good by these metrics. *Maybe* Austin.
Separating rationalists interested in quality-of-life from rationalists working for organizations and doing important world-changing work seems potentially net negative.
I think if we were going to move the Berkeley hub, it would have to be to another US hub—most people aren’t going to transfer countries, so even if the community as a whole moved, we would need another US hub for Americans who refused to or coudln’t emigrate.
I don’t think Moraga (or other similar places near the Bay) are worth trying. They’re just as expensive as Berkeley, but almost all single-family homes, so it would be harder for poorer people to rent places there. Although there’s a BART station, there’s not much other transit, and most homes aren’t walkable from the BART station, so poorer people without cars would be in trouble. And it really isn’t much less expensive than Berkeley, and it’s got the same level of fire danger, so we would be splitting the community in two (abandoning the poor people, the people tied to MIRI HQ, etc) while not gaining much more than a scenery upgrade. I think they’re a fair alternative option for people who can’t stand the squalor and crime of the Bay proper, but mostly in the context of those people moving there and commuting to Berkeley for community events.
If we made a larger-scale move, I think it would be to avoid the high housing costs, fires, blackouts, taxes, and social decay of the Bay. That rules out anywhere else in California—still the same costs, fires, blackouts, and taxes, although some places are marginally less decayed. It also rules out Cascadian cities like Portland and Seattle—only marginally better housing costs, worse fires, and worse social decay (eg violence in Portland).
If we wanted to stick close enough to California that it was easy to see families/friends/colleagues, there are lots of great cities in or near the Mountain West—Phoenix, Salt Lake, Colorado Springs, Austin. All of those have housing prices well below half that of the Bay (Phoenix’s cost-of-housing index is literally 20% of Berkeley’s!). Austin is a trendy exciting tech hub, Colorado Springs frequently tops most-liveable lists, Salt Lake City seems unusually well-governed and resilient to potential climate or political crisis, and Phoenix is gratifyingly cheap.
The most successful adjacent past attempt at deliberate-hub-creation like this I know of was the Free State Project, where 20,000 libertarians agreed to create a libertarian hub. They did some analyses, voted on where the hub should be, created an assurance contract where every signatory agreed to move once there were 20,000 signatories, got 20,000 signatories, and moved. They ended up choosing New Hampshire, which means we might want to consider it as well. It’s got great housing prices (Manchester is as cheap as Phoenix!), a great economy, beautiful scenery, a vibrant intellectual scene, it’s less than an hour’s drive to Boston, it’s very politically influential (small, swing state, presidential primaries), and (now) has 20,000 libertarians who are interested in moving places and building hubs.
If people are interested in this, I think the first step would be to consult MIRI, CFAR, CEA, etc, and if they say no, decide whether splitting off “the community” from all of them is worth it. If they say yes, or people decide it’s worth it to split, then make an organization and take a vote on location. Once you have a location in mind, start an assurance contract where once X people sign, everyone moves to the location (I’m not sure what X would be—maybe 50?)
I think this is a really interesting project, but probably am too tied to my group house to participate myself :(

Scott Alexander 1 Aug 2019 17:34 UTC
108 points
on: Mistake Versus Conflict Theory of Against Billionaire Philanthropy
Rob Reich is a former board member of GiveWell and Good Ventures (i.e. Moskowitz and Tuna) and the people at OpenPhil seem to have a huge amount of respect for him. He responded to my article by tweeting “Really grateful to have my writing taken seriously by someone whose blog I’ve long enjoyed and learned from” and promising to write a reply soon.
Dylan Matthews, who wrote the Vox article I linked (I don’t know if he is against billionaire philanthropy, but he seems to hold some sympathy for the position), self-describes as EA, has donated a kidney, and switched from opposing work on AI risk to supporting it after reading arguments on the topic.
And here’s someone on the subreddit saying that they previously had some sympathy for anti-billionaire-philanthropy arguments but are now more convinced that it’s net positive.
I don’t think any of these people fit your description of “people opposed to nerds or to thinking”, “people opposed to all private actions not under ‘democratic control’”, or “people opposed to action of any kind.” They seem like basically good people who I disagree with. I am constantly surprised by how many things that seem obvious and morally obligatory to me can have basically good people disagree with them, and I have kind of given up on trying to understand it, but there we go.
Even if there are much worse people in the movement, I think getting Reich and Matthews alone to dial it down 10% would be very net positive, since they’re among the most prominent opponents.
I was concerned about backlash and ran the post by a couple of people I trusted to see if they thought it was net positive, and they all said it was. If you want I’ll run future posts I have those concerns about by you too.

Scott Alexander 22 Jan 2012 4:51 UTC
106 points
on: How I Ended Up Non-Ambitious
Generic “ambition” is a serious case of putting the cart before the horse. If you have ambition to do something, that’s great; if not, deciding you should be “ambitious” and then trying to figure out what to be ambitious about rarely ends well.

I think this is why I get creeped out by ambitious people sometimes. I’d much rather my doctor be passionate about medicine than be someone who decided medicine was more “prestigious” than nursing. As a personal anecdote, I am currently in medicine because I want to specialize in psychiatry. I am passionate about psychiatry and plan to be an awesome psychiatrist. I am not quite as passionate about organic medicine with its heart attacks and kidney infections, and although I work hard at it and am pretty good, some of my classmates who get up every morning super excited because they’ve dreamed of treating kidney infections their whole lives are better. I don’t begrudge them this and if I ever got a kidney infection I’m going straight to them and not to the doctor who went into medicine as a subgoal of something else; if they ever get depression I hope they’ll come to me for the same reason.

I understand it’s the same in many other fields. Paul Graham writes that successful startup founders start with a problem they want to solve, eg Larry Page and Sergey Brin were frustrated with terrible online search; unsuccessful startup founders decide they would really like to earn fantastic amounts of money and only worry about what business they’ll do it in as an afterthought.

The only problem here is charity: I do think it may be morally important to be ambitious in helping others, which might even include taking a lucrative career in order to give money to charity. This is especially true if the Singularity memeplex is right and we’re living in a desperate time that calls for a desperate effort. See for example Giving What You Can’s powerpoint on ethical careers. At some point you need to balance how much good you want to do, with how likely you are to succeed in a career, with how miserable you want to make yourself—and at the very least rationality can help clarify that decision.

Scott Alexander 17 Oct 2021 23:18 UTC
104 points
in reply to: jessicata’s comment on: My experience at and around MIRI and CFAR (inspired by Zoe Curzi’s writeup of experiences at Leverage)
I don’t remember the exact words in our last conversation. If I said that, I was wrong and I apologize.
My position is that in schizophrenia (which is a specific condition and not just the same thing as psychosis), lifetime antipsychotics might be appropriate. EG this paper suggests continuing for twelve months after a first schizophrenic episode and then stopping and seeing how things go, which seems reasonable to me. It also says that if every time you take someone off antipsychotics they become fully and dangerous psychotic again, then lifetime antipsychotics are probably their best bet. In a case like that, I would want the patient’s buy-in, ie if they were medicated after a psychotic episode I would advise them of the reasons why continued antipsychotic use was recommended in their case, if they said they didn’t want it we would explore why given the very high risk level, and if they still said they didn’t want it then I would follow their direction.
I didn’t get a chance to talk to you during your episode, so I don’t know exactly what was going on. I do think that psychosis should be thought of differently than just “weird thoughts that might be true”, as more of a whole-body nerve-and-brain dysregulation of which weird thoughts are just one symptom. I think in mild psychosis it’s possible to snap someone back to reality where they agree their weird thoughts aren’t true, but in severe psychosis it isn’t (I remember when I was a student I tried so hard to convince someone that they weren’t royalty, hours of passionate debate, and it just did nothing). I think if someone has mild psychosis and you can guide them back to reality-based thoughts for a second, that is compassionate and a good thing to do in the sense that it will make them feel better, but also kind of useless because the psychosis still has the same chance of progressing into severe psychosis anyway—you’re treating a symptom. Analogy to eg someone having chest pain from a heart attack, and you give them painkillers for the pain but don’t treat the heart attack.
(although there’s a separate point where it would be wrong and objectifying to falsely claim someone who’s just thinking differently is psychotic or pre-psychotic, given that you did end up psychotic it doesn’t sound like the people involved were making that mistake)
My impression is that some medium percent of psychotic episodes end in permanent reduced functioning, and some other medium percent end in suicide or jail or some other really negative consequence, and this is scary enough that treating it is always an emergency, and just treating the symptom but leaving the underlying condition is really risky.
I agree many psychiatrists are terrible and that wanting to avoid them is a really sympathetic desire, but when it’s something really serious like psychosis I think of this as like wanting to avoid surgeons (another medical profession with more than its share of jerks!) when you need an emergency surgery.

Scott Alexander 22 Jul 2014 4:21 UTC
104 points
on: Politics is hard mode
“Hard mode” sounds too metal. The proper response to “X is hard mode” is “Bring it on!”

Therefore I object to “politics is hard mode” for the same reason I object to “driving a car with your eyes closed is hard mode”. Both statements are true, but phrased to produce maximum damage.

There’s also a way that “politics is hard mode” is worse than playing a video game on hard mode, or driving a car on hard mode. If you play the video game and fail, you know and you can switch back to an easier setting. If you drive a car in “hard mode” and crash into a tree, you know you should keep your eyes open the next time.

If you discuss politics in “hard mode”, you can go your entire life being totally mind-killed (yes! I said it!) and just think everyone else is wrong, doing more and more damage each time you open your mouth and destroying every community you come in contact with.

Can you imagine a human being saying “I’m sorry, I’m too low-level to participate in this discussion”? There may be a tiny handful of people wise enough to try it—and ironically, those are probably the same handful who have a tiny chance of navigating the minefield. Everyone else is just going to say “No, I’m high-enough level, YOU’RE the one who needs to bow out!”

Both “hard mode” and “mind-killer” are intended to convey a sense of danger, but the first conveys a fun, exciting danger that cool people should engage with as much as possible in order to prove their worth, and the latter conveys an extreme danger that can ruin everything and which not only clouds your faculties but clouds the faculty to realize that your faculties are clouded. As such, I think “mind-killer” is the better phrase.

EDIT: More succintly: both phrases mean the same thing, but with different connotations. “Hard mode” sounds like we should accord more status to politics, “mind-killer” sounds like we should accord less. I feel like incentivizing more politics is a bad idea and will justify this if anyone disagrees.
What links here?
- wedrifid's comment on Politics is hard mode by Rob Bensinger (23 Jul 2014 6:44 UTC; 7 points)

Scott Alexander 18 Oct 2021 19:30 UTC
103 points
in reply to: jessicata’s comment on: My experience at and around MIRI and CFAR (inspired by Zoe Curzi’s writeup of experiences at Leverage)
Yes, I agree with you that all of this is very awkward.
I think the basic liberal model where everyone uses Reason a lot and we basically trust their judgments is a good first approximation and we should generally use it.
But we have to admit at least small violations of it even to get the concept of “cult”. Not just the sort of weak cults we’re discussing here, but even the really strong cults like Heaven’s Gate or Jamestown. In the liberal model, someone should be able to use Reason to conclude that being in Heaven’s Gate is bad for them, and leave. When we use the word “cult”, we’re implicitly agreeing that this doesn’t always work, and we’re bringing in creepier and less comprehensible ideas like “charisma” and “brainwashing” and “cognitive dissonance”.
(and the same thing with the concept of “emotionally abusive relationship”)
I don’t want to call the Vassarites a cult because I’m sure someone will confront me with a Cult Checklist that they don’t meet, but I think that it’s not too crazy to argue that some of these same creepy ideas like charisma and so on were at work there. And everyone knows cults can get weird and end in mental illness. I agree it’s weird that you can get that far without robes and chanting or anything, but I never claimed to really understand exactly how cults work, plus I’m sure the drugs helped.
I think believing cults are possible is different in degree if not in kind from Leverage “doing seances...to call on demonic energies and use their power to affect the practitioners’ social standing”. I’m claiming, though I can’t prove it, that what I’m saying is more towards the “believing cults are possible” side.
I’m actually very worried about this! I hate admitting cults are possible! If you admit cults are possible, you have to acknowledge that the basic liberal model has gaps, and then you get things like if an evangelical deconverts to atheism, the other evangelicals can say “Oh, he’s in a cult, we need to kidnap and deprogram him since his best self wouldn’t agree with the deconversion.” I want to be extremely careful in when we do things like that, which is why I’m not actually “calling for isolating Michael Vassar from his friends”. I think in the Outside View we should almost never do this!
But you were the one to mention this cluster of psychotic breaks, and I am trying to provide what I think is a more accurate perspective on them. Maybe in the future we learn that this was because of some weird neuroinflammatory virus that got passed around at a Vassarite meeting and we laugh that we were ever dumb enough to think a person/group could transmit psychotic breaks. But until then, I think the data point that all of this was associated with Vassar and the Vassarites is one we shouldn’t just ignore.

Scott Alexander 4 Oct 2009 10:35 UTC
103 points
in reply to: Gavin’s comment on: ‘oy, girls on lw, want to get together some time?’
“You must be an expert Bayesian...cause you’ve got a great posterior.”
What links here?
- Relsqui's comment on (Virtual) Employment Open Thread by Will_Newsome (1 Oct 2010 1:56 UTC; 0 points)

Scott Alexander 13 Mar 2009 2:20 UTC
102 points
0
on: A Sense That More Is Possible
Eliezer, I have recommended to you before that you read The Darkness That Comes Before and the associated trilogy. I repeat that recommendation now. The monastery of Ishual is your rationalist dojo, and Anasurimbor Kellhus is your beisutsukai surrounded by a visible aura of formidability. The book might even give you an idea or two.

My only worry with the idea of these dojos is that I doubt the difference between us and Anasurimbor Kellhus is primarily a difference in rationality levels. I think it is more likely to be akrasia. Even an irrational, downright stupid person can probably think of fifty ways to improve his life, most of which will work very well if he only does them (quit smoking, quit drinking, study harder in school, go on a diet). And a lot of people with pretty well developed senses of rationality whom I know, don’t use them for anything more interesting than winning debates about abortion or something. Maybe the reason rationalists rarely do that much better than anyone else is that they’re not actually using all that extra brainpower they develop. The solution to that isn’t more brainpower.

Kellhus was able to sit down, enter the probability trance, decide on the best course of action for the immediate future, and just go do it. When I tried this, I never found the problem was in the deciding—it doesn’t take a formal probability trance to chart a path through everyday life—it was in following the results. Among the few Kellhus-worthy stories I’ve ever heard from reality was you deciding the Singularity was the most important project, choosing to devote your life to it, and not having lost that resolve fifteen years later. If you could bottle that virtue, it would be worth more than the entire Bayesian corpus combined. I don’t doubt that it’s positively correlated with rationality, but I do doubt it’s a 1 or even .5 correlation.
What links here?
- Is Rationalist Self-Improvement Real? by Jacob Falkovich (9 Dec 2019 17:11 UTC; 258 points)
- Extreme Rationality: It’s Not That Great by Scott Alexander (9 Apr 2009 2:44 UTC; 241 points)