Geoff Anders of Leverage and Anna Salamon of CFAR had a conversation on Geoff’s Twitch channel on October 23, triggered by Zoe Curzi’s post about having horrible experiences at Leverage.
Technical issues meant that the Twitch video was mostly lost, and the audio only resurfaced a few days ago, thanks to Lulie.
I thought the conversation was pretty important, so I’m (a) using this post the signal-boost the audio, for people who missed it; and (b) posting a transcript here, along with the (previously unavailable, and important for making sense of the audio) last two hours of chat log.
You can find the full audio here, and video of the first few minutes here. The full audio discusses a bunch of stuff about Leverage’s history; the only part of the stream transcribed below is the Anna/Geoff conversation, which starts about two hours in.
The chat messages I have aren’t timestamped, and sometimes consist of side-conversations that don’t directly connect with what Geoff and Anna are discussing. So I’ve inserted blocks of chat log at what seemed like roughly the right parts of the conversation; but let me know if any are in the wrong place for comprehension, and I’ll move them elsewhere.
1. (1:57:57) Early EA history and EA/Leverage interactions
[...]
Anna Salamon: All right, let’s talk.
Geoff Anders: All right, all right. Yeah, let’s get to it.
Anna Salamon: I hear that the rationality and Leverage communities have some sort of history!
Geoff Anders: That’s right. Okay, so basically, Anna, as I mentioned to you when we chatted about this, I wanted to talk about in general Leverage history, and then there’s a long history with the rationality community, and right now, it seems like there’s… I’d like to understand what happened with the relations, basically.
Like, earlier on, things were substantially more congenial. More communication. There were various types of joint events. Ben, who suggested he would be called by first name, organized the first solstice for the rationality community. He did that while working at Leverage. That was totally fine. I volunteered at the CFAR workshops. Ben also did that. Then, EA Summit 2013, I thought was sort of a big deal. We had you guys sort of do a half-workshop during that. I thought that was great. Ben reminded me that we actually held a book launch for MIRI somehow. I remember the event very vaguely. Yeah, but it was at Luke Nosek’s house, and I don’t know. Maybe it was a joint… I don’t remember, but there was a bunch of stuff.
Let’s see. All right, yeah, so I just switched my audio. Ali, let me know if that’s better or worse.
But anyway, so there was that, and then I feel like we just had lots of interests in common and so forth, but then somehow, things didn’t end up working out as excellently as possible. So I was interested in just chatting about what happened, getting some common context on the history, maybe trying to...
sowgone: ‘Our Final Invention’, maybe
Geoff Anders: Oh, yeah. Maybe Our Final… Yes! Yes! I think that was the book. At least, that sounds pretty likely.
Yeah, so basically looking a bit at the past, and also trying to understand where things are presently, and maybe figuring out whether there’s anything constructive to be done. Like, in some circumstances it seems like trust is so low that I’d love to know what could make things better, basically.
So anyway, I just talked for a bit. Yeah, so let’s talk about it.
Anna Salamon: Yeah. Are we still trying to end at 1:00, in 20 minutes?
Geoff Anders: I mean, I’m willing to go on longer. I know you had a thing, but you know-
benitopace: I’d be down for a 60 minute conversation, starting now.
Anna Salamon: Oh, I don’t have a thing.
Geoff Anders: Oh, okay. Yeah, so-
Anna Salamon: Yeah.
Geoff Anders: Apologies to all the viewers and so forth, but yeah, let’s go to 1:30.
benitopace: Sg!
Anna Salamon: Great. So yeah, I don’t know. I guess one thing I want to know...
Look, I’m going to try to tell history. I’m going to get it wrong. There’s 50 other people out there who are going to have 50 other histories. Maybe we can get them all out there via mine and Geoff’s to start with, or something.
But, so my perception is—I want to talk about that 2013 summit, where we all… like, 50 people or something stuffed into a giant Leverage house for like, I forget, several days.
Geoff Anders: Yeah, I think it was 60-
Anna Salamon: A week?
Geoff Anders: … I think, and it was a week, yeah. It was about seven days long, I think.
Anna Salamon: And these people were CFAR, and Leverage, and MIRI, and a bunch of EA-ers from overseas?
Geoff Anders: Yeah, we-
Anna Salamon: Who else was there?
anna_salamon: Who here was there, at the 2013 stuff into leverage house EA retreat?
Geoff Anders: Well, we tried to invite basically a selection of everybody. You know, Holden Karnofsky talked there. I think we beamed in Jaan and Singer for talks. Thiel talked, but then we had-
Anna Salamon: Oh yeah, in person.
Geoff Anders: Yeah, yeah. Then, we had-
Anna Salamon: For several hours. I’m remembering as you’re saying the things.
Geoff Anders: Yeah, yeah. And then we… I remember when we designed it, we tried to basically balance it to have people who were interested in the different parts of EA. I have some spreadsheet somewhere, but we tried to… We flew in a bunch of people from the UK, maybe a couple other places, but something like that. And obviously not everybody invited came, but yeah, we had… I feel like we had a mix. Like, for people who are interested, the-
Anna Salamon: Was Toby Ord there?
Geoff Anders: No. I don’t think-
Anna Salamon: I feel like CEA was somehow proto-there. Maybe I’m making that up. Okay. Great. [inaudible] says I made that up. I probably did.
Geoff Anders: I think some CEA-affiliated people were there. I don’t think the main people came, though they were invited.
There’s a video online that, yeah, if you go to YouTube and I guess search 2013 EA Summit-
Anna Salamon: Anyway, great, great, great. I am in a hurry.
Geoff Anders: Yeah, go ahead.
Anna Salamon: Even though we have until 1:30.
So the thing I want to say about that—which I’m not sure how many people would agree with me—but my own narrative is something like that there were a bunch of different people working on and having strands that were kind of related, and we were trying abstractly to think well with each other, because we all knew it was good to be cooperative or something, but we didn’t have that much actual flesh on our cooperation.
And then my narrative is that Geoff or Leverage or somebody was like, “You know what would help? I think it would help if we all spent a week stuck in a house together.”
Geoff Anders: Yeah.
Anna Salamon: And it did, and then we emerged all friends somehow. Like, not exactly friends, but all actually feeling like there was a thing that was in common, that we might call the EA thing, and that it was worth talking to each other. And a bunch of people got the AI risk thing, who came in skeptical about the AI risk thing. And I don’t know, I was very self-centered about the AI risk thing. I’m not sure how many...
sowgone: I was at 2014, not 2013
Anna Salamon: No, no, no, it wasn’t… Oh, you were at 2014. Yeah, yeah. There was a previous one. The 2014 one was much larger, and more disparate. I liked it too, but it was like we were… Sorry, somebody in the chat was mentioning 2014.
Geoff Anders: Well, we did-
Anna Salamon: It was-
Geoff Anders: If I could just jump in with one fact.
Anna Salamon: Okay.
Geoff Anders: We did both the week-long thing at a retreat place, and then we did a two-day event, like the Summit, so the retreat was like-
Anna Salamon: In 2014.
Geoff Anders: Yeah, so we did both in 2014.
Anna Salamon: Yeah, but even the week-long thing to me seemed, in some ways, less good than the week-long thing in 2013, because it was spread out instead of being in one house.
Geoff Anders: I agree. Yeah.
Anna Salamon: Which was maybe necessary because there were more people or whatever.
But anyway, from my perception, there was a lot of camaraderie in the early days, even like 2008, 2007, up through 2013, but the camaraderie sort of congealed and became more of a thing across more of the people that week in the house. And there were always tones of skepticism, but the tones of skepticism were maybe at a local low point right after that week from my perception.
And then we have gradually, not just Leverage and the rationality community—whatever that is, sorry CFAR, LessWrong, I don’t know, MIRI -
Geoff Anders: I… think there’s a community.
Anna Salamon: What? There is a community, but it’s a community that’s… Yeah, there’s a community, but there’s also several different partial centers of the community or something.
Geoff Anders: Agreed, yeah.
Anna Salamon: So I wasn’t quite sure which one I was trying to talk about.
Anyway, so Leverage and various strands of the rationality community were pretty harmonious there in my perception, and were a little less so before, and much less so gradually after.
But also like, I don’t know—CFAR and CEA, for example. In my head, there was more of the congealment there. Like, we were all friends back then.
And now we’re all… maybe friends, certainly trying to be harmonious—many of us, maybe not with Leverage, maybe Leverage is our official “figure out whether we’re supposed to ostracize this month” target, we can talk about that...
But I guess I just wanted to situate the puzzle of how rationality and Leverage drifted apart, with a puzzle of: I think that in general, most of the organizations think that most of the other organizations’ work isn’t that good, and usually do try to be cordial and harmonious, but I think that things somehow started out much more “everybody believing in cooperation with each other”, and are much less that way now, even though there’s quite a bit of cordialness still.
And I guess I want to situate the splitting of Leverage with CFAR or whatever, in that larger context where I think most things split from most things. And I’m kind of curious about what was up with the larger context.
Geoff Anders: Yeah. I mean, I agree with this broad picture. I wouldn’t put the start of EA to the EA Summit, though I think people could argue about that. I think EA started before. But apart from that, I agree.
I think there is something like most of the groups have become more disparate. It’s really hard to figure out how to say this without sounding biased in my own favor, and also while there’s lots of concerns, and a bunch of sort of justifiable things, it’s really hard to say things like this, but in 2012 and 2013 and 2014, we were working hard to try to get the different groups to be able to work together.
Like, 2013 we did the Summit. 2012, we did THINK, the high impact network. Where we wanted to have CFAR, The Life You Can Save, 80,000 Hours, and Giving What We Can and Leverage all collaborate. And that didn’t end up working. But then the EA Summit did.
Yeah. And then, I feel like the cordial relations and the “more on the same team” thing extended through 2014.
For my own story, just to add a part to it, I think that I stopped wanting to… like, stopped engaging as much with CFAR after CFAR stopped doing as much research? Like, CFAR was doing more research early on, and then the research diminished over the course of time. That was something that mattered for me, personally.
But I mean even, I think in 2018 we did a Paradigm workshop specifically for CFAR people to try to, like, something. And I talked to the people about, “Is there room for some sort of alliance or working together?” And that didn’t go anywhere, but who knows?
Anna Salamon: Huh. I don’t remember this. Was I invited?
Geoff Anders: No, no, this was the-
Anna Salamon: Is it specifically for CFAR people...?
Geoff Anders: I think at the time you were… I haven’t always tracked when you’re in charge of CFAR. Like, there’s at some times...
Anna Salamon: But I’ve been working at CFAR continually.
Geoff Anders: Yeah. So-
Anna Salamon: I wasn’t ED, but I was totally working there.
Geoff Anders: Okay. Yeah. So I-
Anna Salamon: I’ve been in a leadership role.
Geoff Anders: Okay. Yeah. So this was a reach out to Eli Tyre et al., I think.
Anna Salamon: Okay. Yeah, yeah.
This is not very charitable of me at all, but I think I had a perception that you were trying specifically to recruit our junior staff and that you weren’t that interested in talking to people like—by, recruit, sorry, I don’t mean to pull them away from CFAR, but I do mean to sort of ideologically recruit them—but that you weren’t that interested in me because you were less likely to get that much of my mind or soul or something. Confirm or deny?
Geoff Anders: Yeah, no, it’s a good pointed question.
How do we draw a distinction between “get you ideologically” and “have constructive conversations, wherein both sides update”?
Anna Salamon: Um. I don’t know. I do think there’s… Well, what do you think there… I think these are quite different. I think they’re both great things, depending on context.
Geoff Anders: Sure.
Anna Salamon: Especially the second one.
Geoff Anders: Yeah, yeah.
anna_salamon: Anja, would love to hear if your perceptions match/differ, what you remember
fiddlemath: Anja: matches up so far
fiddlemath: Anja is carrying the baby, but—she has a pet theory that a lot of the friction / tension traces back to Leverage overstepping boundaries with their recruiting practices.
AgileCaveman: The vibe here is two people talking about a failed relationship, except it’s two large orgs.
habrykas: I can confirm memories of tension that was associated with Leverage’s recruiting practices
habrykas: From the 2014-2016 era
Anna Salamon: I often learned from conversations with you, though. I remember going over to your house a bunch of times with Nate, it seemed good. You seemed less interested.
Geoff Anders: What year was that?
Anna Salamon: That was, I’m going to make this up, my made-up year is 2016 or 2017.
Geoff Anders: Yeah, I-
Anna Salamon: Remember?
Geoff Anders: I was talking - [crosstalk] - I was talking to somebody who jogged my memory on that. And I do think that was less important to me at the time. Like, the...
Anna Salamon: Okay, so my yeah jerk story as to why we drifted apart—I can tell jerk stories that emphasize me as the bad guy, I’m going to emphasize you as the bad guy -
Geoff Anders: Well, y’know...
Anna Salamon: - but you’re welcome to turnabout or whatever.
Geoff Anders: Yep.
Anna Salamon: My jerk story is, you were busy trying to build an empire for reasons, that you thought would be able to build toward good things, or whatever. And we didn’t look like good ways for you to do that. Partly because we related differently to power than you do, or power, closure, how much of your stuff to keep inside in what way. And partly because, as you say, we were not doing that much of the kind of psychological research that you guys were trying to do, so we weren’t particularly project-aligned in that way. And I think it’s cool that you guys were trying to do that kind of thing. And I think in CFAR, a lot of us were curious as to what your stuff was and whether it was cool. So I think that was one of the things.
Anyway, I think you were trying to build a thing with power where you could control it, and I think alternate perspectives were not that interesting to you on a visceral level for that reason. And this is part of why we drifted apart.
Geoff Anders: Okay.
Anna Salamon: And I think I looked more like a… kind-of-like-resource-kind-of-like-obstacle, whereas some of the junior staff looked more like people you could recruit.
Geoff Anders: Yeah. But then, you said “recruit”. So, if I could ask some questions about this. So, you said “recruit, but not hire”.
Anna Salamon: Well, maybe hire, but “hire” isn’t the important part.
Geoff Anders: Okay. Right. So then I guess my question is something like: I do end up forming views on who I’ll be able to work with, more and less and so forth. And then, I mean… there’s, like, a nefarious-themed-ness to the thing you’re saying. But how does the...
Anna Salamon: Should I try to actually spell out what the thing is that I think is bad?
Geoff Anders: Yes! [crosstalk] That would be great. Because I can tell the same story without the nefarious theme. Yeah.
Anna Salamon: Yeah. Sorry for accidentally having it in the connotation.
Geoff Anders: Oh, it’s okay.
2. (2:13:22) Narrative addiction and EA leaders being unable to talk to each other
Anna Salamon: Okay. I’ll take a couple steps back, explain a bit of worldview in which I think it contexts why a thing I think might be bad might have been occurring. And then say the thing, sorry.
So, I guess I’ve been playing around a lot lately, in the last six months or something, with the concept of narrative addiction. Where in my head… I don’t know, I’m gonna talk about stuff I don’t know anything about, nobody should believe me if you think it’s false. If you guys could just figure out which parts are which, that’ll be great.
I tend to think of, for example, anorexia. Which I know nothing about, I’m just going to terribly say bad things about it as a metaphor while violating all rules. I tend to think of anorexia as being something maybe sort of like narrative addiction.
Like, it’s addictive, but it’s not addictive the way heroin is addictive. It’s not physical, but there’s a sense of control or something. And you want to be in control. There’s a narrative, or something, and somehow something will get better in the narrative. In extreme cases, people cling to it even at the expense of life. That’s probably horribly wrong for anorexia, and I apologize to everybody for using it as a metaphor.
But the thing I actually want to talk about is I think, for me, I spent a whole bunch of the last five years or something in what I would call narrative addiction.
Geoff Anders: Hm!
Anna Salamon: Yeah, sorry. Accuse me of a thing and then I’ll go accuse you of the thing. And then you can tell me that it’s false about you.
Where according to me, there was a thing that I wanted, which was something like “to make the world a lot better.” And a story that I had in which I was doing a thing like that. And then various things would happen that were kind of uncomfortable, or didn’t feel good, or whatever. And I would sort of retreat to perching on this narrative and to doing things that would locally reinforce the narrative for me.
And I think basically this was pretty costly and not very good for my happiness or my effects in the world, or my ability to have actual conversations where I was listening to the other party.
I think a thing that happens to a lot of people in positions of relative power and influence in a social graph—similar to the one that I was in, and similar also to the one that I believe you were in—is somehow only doing the things that reinforce the narrative, and having much more trouble than other people have hearing perspectives that aren’t their own.
So, like, in terms of this fragmentation of the initial EA community, it seems… Like, I had an interesting conversation with some people several years ago where they were like, “Yeah, I feel like if we stuck...” I can’t remember who they actually named, but something like, “I feel like if you stuck, for example, Anna and Geoff, or Geoff and Eliezer, or Eliezer and Toby Ord, or whatever, in a room, it would actually be like relatively...” I don’t know about Toby. Maybe Toby’s better. ”… It would be relatively difficult for most of these pairs of people to have a real conversation. But if you take random people from the organizations, they do fine. What’s up with this?”
Geoff Anders: Right.
Anna Salamon: We were like, “Yeah, that’s kind of interesting.”
spiracularity: I like the way this is going, this feels both interesting and I think it hits something valuable.
larissa24joy: +1
AgileCaveman: oh yeah super valuable,
larissa24joy: something like the narrative addiction concept rhymes with some of my EA experience so this is an interesting and useful concept (thanks Anna)
fiddlemath: Geoff, for later: Some of the narrative addiction effect may come from organizations growing larger and more of their members navigating reality via narratives and social cues
Geoff Anders: I definitely just affirm—I definitely think there’s a leader’s thing. In different fights and so forth that happened at different times, my observation was, the leaders were more opposed. The people were like, “Can we get along?”
Anna Salamon: And they could, and the leaders couldn’t; and the leaders also couldn’t talk, which is more the interesting thing. Not just couldn’t get along.
Geoff Anders: Yeah, yeah.
Anna Salamon: Anyway, so I believe you… So, look, the negative thing that may or may not have been the case, is that you wanted people who were “small” enough that you could—and I think these people were great and were on great growth trajectories, but nevertheless, to say terrible things—you wanted people who were “small” enough that you could intellectually dominate them and they would follow your thing, rather than being willing to loosen your grip on narrative addiction or something enough to be able to hear alternate perspectives. That would be the-
3. (2:17:14) Early Geoff cooperativeness
Geoff Anders: Okay. I definitely think that there’s at least… Like, I definitely think I didn’t appreciate really at all the work-life distinction and the value of doing things that weren’t for the mission and so forth. So I think that’s definitely there. But if I could ask a question about the thing, what was the narrative that-
Anna Salamon: Mine?
Geoff Anders: No, no, for-
Anna Salamon: Yours?
Geoff Anders: Yeah, because from my perspective… I mean, you may recall this and this may be another source of tension. Initially, I wanted to just merge the organizations, right? I tried to be hired as research director at MIRI. I don’t know if you recall this, but-
Anna Salamon: I remember you tried to get hired at MIRI, at the very beginning, yeah.
Geoff Anders: Yep. Yep. But that was like, Leverage was super small then. And I’m like, well, we have some differences, but there’s a… There’s the inside-Leverage thing, where there’s like the different projects working in parallel and helping each other. I also thought about that with regard to external organizations. I talked to Will MacAskill in… 2012, I guess. Was it 2012? I think 2012. And I was like, “We should maybe merge Leverage and Giving What We Can.” And he was like, “Yeah, good idea,” or something like that. And then it didn’t happen. And I don’t know if he was… I’m not sure what his attitude was on that.
But, like, even take something like Giving What We Can, which is focused on global poverty, or was focused on global poverty. I always had the perception that it just would be valuable for the other projects to build up a really big Giving What We Can thing. I mean, if you have somebody build up Giving What We Can in a very large, substantial way, that just will in fact pick up more people who are interested in animals and building EA and AI and Leverage’s style of research, and et cetera. So there’s a thing that I’d like you to try to square with-
Anna Salamon: [crosstalk] Rationality is the common interest of many causes. Yeah, we had this in the beginning, then it became rational-
Geoff Anders: Right. But, so square that with the idea of needing people to be in my thing.
Whoops, sorry. Hold on, my phone is beeping-
Anna Salamon: Well, okay. So my story about you and me, and many people, is that we started out not having as much of the narrative addiction and ended up more in the narrative addiction. So I agree that at the beginning you didn’t seem that much like you were doing this, compared to my stories about you later on.
Geoff Anders: Okay.
Anna Salamon: I have the same story about myself and several other people.
4. (2:20:00) Possible causes for EAs becoming more narrative-addicted
Geoff Anders: Do you have any idea what would have caused us to become more narratively addicted? Because I find this to be really an interesting hypothesis. I think there’s some ways that I may not have had as much perspective on things, especially near the end before dissolving Leverage. And so I’m really interested in this hypothesis, but I haven’t been able to… It’s really hard to see how you lose perspective.
Anna Salamon: Yeah, it is hard to see how you lose perspective. You almost… Yeah, you need to look from inside and from outside at once, or something.
Geoff Anders: Yeah. But maybe you have leads on… or, what...
Anna Salamon: Yeah. I mean, I don’t know. One story—this is a very, very generic story, I think it would also be nice to try to track this one with Leverage and CFAR specifically.
Geoff Anders: Sure. Yeah.
Anna Salamon: But sort of as a case study in this larger thing, almost. In addition to being a thing in itself.
But, yeah, I have a story that’s something like… Many, many actions run partly on promissory notes. (Borrowing a lot of this, by the way, from Bryce Hidysmith, so if anybody likes these ideas, I think they should give some credit to him.)
Many, many actions run partly on a promissory note. I make myself tea, and this is because I can visualize the tea which is located in the future, and it’s calling to me, and I believe that if I put my cup in the microwave I’m going to successfully get some tea, and it’s going to be great. Right?
But there’s the question of how much of the thing is a promissory note that hasn’t been cashed yet? Like, what the ratio is of “expectation of the future” compared to “realized stuff right now”. And what ratios those have in pulling our actions.
Geoff Anders: Right.
Anna Salamon: The EA movement was very, very high in “we’re going to have a whole bunch of glorious stuff in the future, it’s going to be amazing”, compared to how much we had actually done so far in terms of lives saved, or whatever.
Geoff Anders: Right.
Anna Salamon: One story I have is that we got together, and initially we all hoped to do the best thing. And the best thing was basically, we were all going to cooperate and it was going to be great. And then various bits of data came in, and the data was not super coherent with the best thing that we were hoping for. And so we retreated farther and farther and farther into smaller and smaller scope of what we were taking in kind of narrative addictions. And less and less willingness and ability to listen and bridge. But I mean, that’s an abstract story and then we can tell a more mundane story.
habrykas: Huh, at the time, it felt like EA was almost only focused on doing things right now
larissa24joy: From my own experience of something that might be like “Narrative addiction,” I feel like I lost some perspective out of fear/responsibility. I felt hugely responsible for EA (more than makes sense on reflection) and I feel like that led me to tunnel vision focus on EA and trying to make that go well.
habrykas: Like, the movement was primarily about global poverty things and high-standards of evidence
5. (2:22:34) Conflict causing group insularity
Geoff Anders: No, but that’s interesting, because I think the… I’ve been tracking a different aspect that maybe fits with the narrative… I want to think about the narrative addiction part. I definitely think there’s truth to that, so that seems like a good part, and I want to think about it.
The thing that I’ve been tracking is the ways that groups become more insular as a result of conflict. Where, so, really simple examples like: Leverage had a lot of information on its website back in 2011. And then we got a bunch of critique, and we weren’t getting a lot of positive engagement. And so then we eventually, by 2015 I think we’re down to a splash page.
But then there’s something similar. So right now I think one of the potentially really negative, I would imagine unintended, side-effects of the current thing happening right now, is when there’s fighting it sort of scorches the earth between organizations. Where it’s something like… Yeah, go ahead.
Anna Salamon: I guess… sorry. In my opinion… No, no, I had a thing to say, but maybe I should hear the end of your sentence.
spiracularity: Someone also appears to have petitioned to get the old Leverage website taken down from Internet Archive, too.
spiracularity: …why did you guys feel like you needed to do that, anyway?
habrykas: My memory of the reasoning (when I asked people about this many years ago) was something like: “People keep digging up a bunch of stuff I said and keep taking it out of context. Like, that one time when I said ‘Connection Theory is a complete theory of psychology’, by which I meant ‘Connection Theory is trying to be a bold stick-out-your-neck type theory that aims to be complete, but look, it obviously doesn’t explain everything, but this is the thing it’s aiming for’”
habrykas: “And people kept accusing us of saying wrong things in annoying ways that seemed pretty confused, and it seemed like overall when people looked at the archives, they got more confused than enlightened”
habrykas: But someone should feel free to correct me
larissa24joy: Something like Habryka’s account seems right to me but that’s based on similar conversations with people about their recollection, not from any knowledge at the time
larissa24joy: (so people around at the time should totally fill in gaps instead)
Geoff Anders: I mean, I think that there are… Basically, I think there’s lots of forces that produce insularity. And I think something like continued unresolved-but-still-real conflict is one of them. I’m not sure that that’s true, but it’s like, if you have like… Let me just add, it’ll just take a second. You’ve got red team and blue team on different sides, right? And each has a negative thing about the other that’s not being expressed, but it’s being held on to. And I’m not saying it’s not—it could both be real. It’s not meant to not be real. But I think if that happens, then it’s really natural for the things to sort of curl in and not… Yeah, basically become more insular. But yeah, go ahead. You were going to say a thing.
6. (2:24:51) Anna on narrative businesses, narrative pyramid schemes, and disagreements
Anna Salamon: Yeah. I was going to say, I think there’s different kinds of… Sorry. A different word that I want to introduce besides narrative addiction is “narrative pyramid schemes”.
Geoff Anders: Okay.
Anna Salamon: Where you get a bunch of people buying into a narrative or something, and this is how you’re able to… Sort of a group-level version of something like narrative addiction. But you get a bunch of people buying into a narrative. Because they bought into the narrative, they’ll do different things depending on what you say. So they’ll work for CFAR, they’ll work for Leverage, they’ll go take a bunch of actions in the world that you want.
Geoff Anders: Yeah.
Anna Salamon: But they’ll do it because they’re expecting this narrative to pay off in some fashion. And the thing… yeah, I could just say “narrative business” or something rather than “narrative pyramid scheme”. The thing that makes it a narrative pyramid scheme is if it only works insofar as the narrative is able to get other people to buy into the narrative, is able to get other people to buy into narrative, and in this way-
Geoff Anders: Sure. Well, yeah. I mean-
Anna Salamon: - whatever, the movement can grow, even though the movement doesn’t really have an actual business model that produces value in the sort of… Anyway, you could agree or disagree with that, but I want to connect it back to what you were just saying.
Geoff Anders: Sure.
Anna Salamon: And then you can reply. It seems to me that there’s lots of different ways that groups A and B could be in conflict. One way we could be in conflict is by, I don’t know… well, maybe there aren’t. Maybe we just have different research agendas and we’re just trying to do our different research agendas. And we just think each other’s research agendas is going to be dumb. And so we’re most [inaudible] to do the research agendas. And in the end it’ll go to the glory of group A, I mean of group B. Okay, that’s a kind of conflict. It’s not a very active kind, because we’re not really disputing what to do with a resource. But anyway.
Geoff Anders: I do think that can be an actually real thing. Different research projects are based on different assumptions about the world. And I think you can-
Anna Salamon: You can have pretty important disagreements that way. I’m not sure whether they’re conflicts, exactly.
Geoff Anders: There’s going to be some reason why they’re not being expressed or being discussed fruitfully, right?
Anna Salamon: Well, you could have… in particular, I could imagine a set of people who have really interesting intellectual disagreements while disagreeing about which research are going to be fruitful or whatever, and it’s just really fun. I don’t know if that’s a conflict, it’s like conflict...
Geoff Anders: Oh yeah, no, no. Yeah. I agree, you can have it without the conflicty part. Yeah, basically.
Anna Salamon: But the interesting thing to me is that if you have two groups that are each pulling on their own people via narrative, and if the narrative is in some important sense false—I don’t mean it’s 100% false, but it’s one of those things that could be destroyed by the truth, in some sense...
Geoff Anders: Yeah.
Anna Salamon: Maybe by pointed, carefully selected truths designed to destroy it, or whatever. But it’s not robustly “just go notice this stuff”. It’s, like, a narrative.
Geoff Anders: Right.
Anna Salamon: Then conflict between the two groups I think tends to create dead zones of non-conversation. I don’t know, because… sorry, maybe you could just actually flesh this out into a model that I’m not articulating yet.
7. (2:27:51) Geoff on narratives, morale, and the epistemic sweet spot
Geoff Anders: No, that’s super interesting. I want to add a piece, though, that I think relates to maybe disagreement, but I think is important, and also I think relates to the rationality/Leverage divide.
I think there’s a bunch of dangers associated with narrative. The narrative addiction thing, I think, is real. I want to think about it and understand it better, et cetera, but I’ve noticed some things that are similar—the thing I’ve been thinking of is in terms of “fantasies” basically, and people getting trapped in them. That’s my own language.
But there’s a way in which narratives are really, really valuable. Like, narratives can help people to maintain morale in circumstances that are really bad and really hard. And then I think this is one of the flash points between Leverage and rationality, where I think of it as a very careful navigation. If narratives are misleading and wrong, people become epistemically less good; but I also think that if your morale gets too low, you also become epistemically less good.
And so I think the epistemic sweet spot is—the best thing is “you know how to do things and you can accomplish them, and you can see all the truths and everything works”. But then, as a person’s ability to accomplish things, or the quality of their position goes down, then I think frequently narratives are useful, and sometimes necessary, to maintain morale. And that if you graphed narrative versus epistemics, it’s not “epistemics are the best when narratives are the least”. I think that there’s frequently—like I said, morale. Morale sometimes helps people to look for paths for hope. You can have a person who just doesn’t think about how things could be better. So I think narrative is dangerous, but also useful. And so it has to be used correctly, roughly.
spiracularity: You know what else is often surprisingly adaptive to negative environments, then becomes maladaptive but hard to deconstruct once you’re in other circumstances? A lot of trauma responses!
329zwydswr: what’s morale, other than a belief that a plan will work?
spiracularity: (The tone is not intended to be as bad as it sounds, though. I just think they’re both STICKY.)
WaistcoatDave: @spiracularity exactly. and as everyone is human, they come into these situations with preexisting identities’e
329zwydswr: that seems to conflict with the thing about morale
WaistcoatDave: identities, experiences and often trauma
larissa24joy: Can you elaborate @spiracularity ? (I feel like there might be an important thing here but I don’t quite grokk it yet)
8. (2:30:08) Anna on trying to block out things that would weaken the narrative, and on external criticism of Leverage
Anna Salamon: Yeah, so I guess the thing I would say is narratives… are great. Like, even something like “I’m going to put the tea in the microwave and then it’ll be warm and I can add a tea bag and then I’ll have tea” sort of a narrative. It’s a carefully selected small portion of a thing that I can use as a storyline. Larger-scale narratives are great too...
Narrative addiction I would like to distinguish from narratives, and to say that it has to do with the specific motion whereby I’m trying to block out things that would otherwise weaken the narrative.
And I am sure there are local benefits that can be obtained by blocking out things that will weaken the narrative. But my current position is that we just shouldn’t do that. Like, it can help locally, but it creates something like technical debt. Let’s not do it.
Geoff Anders: I roughly agree. I roughly agree. I mean, I think there can be cases, but roughly I agree with that.
Anna Salamon: Okay, I… don’t believe you that you agree.
Um. Or, sorry, I’m sorry, I believe—I’m sure you tried to tell the truth and so on. But I would like to bring up things that I think you also think, that I think are in tension with what you just said, and see what you say about it. I mean...
Anna Salamon: So, I think… So, look, I—mm. It’s hard to say all the things in all the orders at once. I’m going to say a different thing and then I’ll [inaudible], sorry.
So, once upon a time I heard from a couple junior staff members at CFAR that you were saying bad things to them about me and CFAR.
Geoff Anders: [I] believe it.
Anna Salamon: I forget. They weren’t particularly false things. So that I don’t accidentally [inaudible]-
Geoff Anders: Okay.
Anna Salamon: Whatever. Personal view is that it was probably partly silly that I was upset about this. It was not my view at the time; we can talk about it. But anyway, the reason I was upset about it is basically because I did not hold the attitude… I don’t know. Sorry, I could say...
I think it was partly that it didn’t reach me very naturally. I think that was more of a legitimate objection. But, whatever, we can talk about it, I probably am getting the facts of the situation wrong while accidentally maligning you on Twitch TV.
Geoff Anders: I’m sorry if I… did an incorrect thing there. So I’ll just let you know that-
Anna Salamon: Well, let me-
Geoff Anders: Go ahead. Go ahead. Go ahead.
Anna Salamon: Thanks. I appreciate it. I should skip you for the story. Once upon a time there were various past contexts where I would be upset by people seeming to me to weaken CFAR’s narratives in its staff. I currently think that that was a mistake on my part. And that it violated this heuristic that I currently think we should have, where you don’t use narratives to try to block stuff out.
I think there were also past situations where you, Geoff, were upset at people saying bad things about Leverage in ways that made it harder for your staff members to have, as you would put it, morale. And I’m not sure how that’s consistent with the thing you just said about how you basically agree that one shouldn’t use narratives to try to block things out.
Geoff Anders: Well, do you have cases in mind? That’d be helpful.
Anna Salamon: I have only generic cases, but I bet you can fill in the specifics.
Geoff Anders: Yeah. Try a generic case.
Anna Salamon: I think that you’re all, like… I don’t know, the straw Geoff in my head is all like, “Man, the rationale of the community was mean to us, because they kept saying bad things about Leverage in ways that made it harder for our staff members to have morale. I wish they would stop it. They kept saying it specifically to staff members and so on, and I don’t like it. They shouldn’t-”
Geoff Anders: I mean, I do think that that’s true. I do think that Leverage—this is one thing that I think is a really unappreciated aspect of this whole situation. It’s like, a whole bunch of Leverage staff just got tons of negative stuff from people. And it was, like, as soon as you joined.
Anna Salamon: I know, I know. I would be afraid… just to put it out there, I would be afraid to say positive true things about Leverage because then people would get mad at me.
Geoff Anders: Right. So I-
Anna Salamon: And then I would say fewer positive true things about Leverage than I thought, even though… Like, whatever, I’ve thought various true bad things and various true good things. Well, of course I think my things were true. I thought various good and bad things. But like-
Geoff Anders: But it’s super nice to hear you say that. I mean, wow. Okay. All right. Well, good. I feel like we’re having a good conversation, at least from my side. Hopefully this will turn out good for you also. Okay. I...
AgileCaveman: it was somewhat surprising that some really vocal critics of leverage later applied to join. There was certainly an under-current of envy
habrykas: @AgileCaveman : Huh, do you have an example?
habrykas: None come to mind for me, but I am of course not omniscient
z1zek100: I think there are cases the other way. People wanted to work at Leerage, didn’t for whatever reason and then were very upset.
AgileCaveman: yes, i have a couple people in mind, but i’d rather not name for privacy sake
habrykas: Yeah, seems sensible
AgileCaveman: z1zek100 yes, true as well
spiracularity: To be fair, you’re taking Rationalists who love ’ferretting out weird rare elusive dangerous truths,” and then do stuff like taking down your website. Some of them being envious and then joining doesn’t surprise me. Some of them assuming you had skeletons doesn’t surprise me, but I agree they could be pretty brutal sometimes.
9. (2:34:24) More on early Geoff cooperativeness
Geoff Anders: Definitely one thing that was a practice, a part of Leverage culture, at least as I understood it—I’m going to say a pro-Leverage thing, but I’m going to say then a negative thing after that. The pro-Leverage thing is: As I understood it and as I experienced it, we had a practice of trying quite hard to understand the value of different projects as part of assessing potential collaboration. And so it was really important to us, something like “how much original research was happening at CFAR”. And the more original research, the more it would be like, “Okay, good,” because that’s something we could conceivably learn from. And also, at least in my… It’s just better to have groups that are… If they’re research groups doing more research, then that’s good.
I think that that’s the positive thing. I mean, there is just a lot of “try to say the positives and negatives”, and then a way that this may have… One of the things that’s been coming to my attention as I’ve been thinking about it, is the way that something like culture is not always transmitted. Like, some of the things about narratives, some of the things that have been said I was really surprised to hear that people were using narratives in that way. But it’s quite possible that people were using the relevant negatives [sic] - something like, trying to paint an overly negative story. But, I mean… Yeah, I don’t want to, again, be overly pro my own thing because there’s obvious biases and so forth. But it’s like, if, I mean… yeah, it’s… anyway.
Anna Salamon: Maybe best to just display your own thing with obvious biases mixed in, and then anyone can complain about them.
Geoff Anders: Yeah. I mean, okay. Because invited, and in the name of transparency:
I think we tried hard to cooperate with the other groups. I think we reached out. I think we ran events. I think we did, like, a whole bunch of things. We, like… see, look, now I’m going back into the “We did many things wrong,” which I do think is true, but… I think that it just would be nice to not be excluded or cut out of the histories, or et cetera. So it’s like, you look at tellings of… Look at the EA Global Wikipedia page and see what you think about whether or not we’re all, like, being included and so forth.
And so my big update from 2012 was: I had done a whole bunch of reaching out and trying to cause collaboration, and people just weren’t interested.
And so, I thought about it like: I took a hard lesson. Like, the hard lesson was “the other people don’t want to collaborate.”
And then I had all these theories—like, one of the theories was that it had to do with planning horizons, where it seemed like people were very focused on what would be useful for their organization in the short run. And I know I get critiqued to death for having, like, really long-term plans with all of these parts. One of the things that happens if you have that is you just see ways that other people can fit in.
And so it’s like, huh, if there’s like… I mean, this is part of the EA Summit “all the groups sort of aid each other in various ways” thing. And THINK was meant to be a collaboration. So from my—maybe this is biased, but if score it, I just give us great marks on trying, and I don’t give the other groups great marks on trying.
And so, yeah, people can be mad about that; it does seem… Yeah. Okay. Let me just say one other thing. I think this is important.
I think there’s definitely a way in which I and other people at Leverage have been beaten down by the constant narrative sort-of opposition.
fiddlemath: Anja says: Geoff—I think the people in the Rationality community who say negative things about Leverage, are/were in an arms race with an intention on Leverage’s part to withhold details that could be negatively construed.
spiracularity: @fiddlemath EXACTLY.
fiddlemath: yep
spiracularity: Add an extra layer of “you guys were not optimizing for outside legibility”, so the straight words didn’t really convey things very well.
Geoff Anders: And part of why I wanted to have a Twitch channel was just to stop doing that. And this is pointing to an area where I think I-
Anna Salamon: Sorry, stop doing which?
Geoff Anders: To stop, like… The thing I want to say is something like, “I think we valued collaboration and so forth.” Whereas, the thing that I should somehow be able to say was: We tried to collaborate. We offered help. If other people had wanted to collaborate, it would have happened. That it counterfactually, causally depends on the other people’s not wanting to collaborate. If you, Anna, had come to me and said, “Geoff, let’s figure out how to do a joint psychology research program and have a bunch of workshops,” I would have said “Yes,” or I would have said “Yes, if,” and then worked out some complex thing.
I’m sorry that that’s coming across as, like, directive or determinative, or I don’t know—the way I’m saying it. But I’m, like, pushing through-
Anna Salamon: I appreciate that you’re putting your narrative out there in a clearer fashion. It seems nice. It seems good for conversation, to me.
Geoff Anders: Okay. Well, it’s—yeah. I’m just expecting to get critiqued to death on this because-
Anna Salamon: Well, I mean, I think that is one of the things that happens if you say something clear—is people can respond. And that seems good too.
Geoff Anders: It is notable, also: I’m usually okay being critiqued to death, or whatever, but, like—Okay. Useful. Useful. There’s my narrative. I want updates. I… anyway. Yeah, you know. Okay, go ahead, say something.
LuliePlays: One thing I’m finding odd about this conversation about narratives is that there were accusations of demon seances—I’m curious what @anna_salamon thinks about that stuff, or if it seems irrelevant to CFAR’s relationship with Leverage
329zwydswr: hm. sounds like geoff is describe a sort of progressive escalation of narrative warfare. like a feedback loop of perceiving a slight narrative tilt → tilt right back, to correct the scales → other people push back → repeat
Turbowandray: +1
habrykas: My current model is we are talking about Leverage from 2013-2017
habrykas: Which did not involve any demon seances
Turbowandray: (to clearer position/etc)
habrykas: By the time demon seances were happening, I think relationships between CFAR and Leverage had mostly come to a stop
spiracularity: Maybe your narrative was protecting you from feeling critiqued to death, but had costs elsewhere? (speculative, though)
DaystarEld: “One thing I’m finding odd about this conversation about narratives is that there were accusations of demon seances—I’m curious what @anna_salamon thinks about that stuff, or if it seems irrelevant to CFAR’s relationship with Leverage” +1
z1zek100: +1
Anna Salamon: Yeah… Um, sorry, somehow I’m now wanting to read the sentence that has my name in it from the chat.
Geoff Anders: Yeah, it’s really hard to navigate when people are saying interesting things.
Anna Salamon: Oh, yeah. They want us to talk about different stuff from what we’re talking about, but I think I should reply to you first. So I’m going to do that, and then we can go there, or not.
Yeah, so… I agree… Like, my perception… I agree that I remember a bunch of concrete things from the early years, like 2012 through 2015 or something − 2014, I don’t know—in which Leverage was doing a bunch of things that I would call trying to collaborate. Especially things in the vicinity of “trying to collaborate on concrete projects that were meant to bring resources to both groups”, or whatever.
Geoff Anders: Right. Yeah. Yeah.
Anna Salamon: I recall Leverage as being extraordinarily, by local standards, open to and initiative-y in proposing things of this sort.
Geoff Anders: Right.
Anna Salamon: And I believe you, that you would have been receptive to such initiative from other groups [inaudible].
Geoff Anders: Really?! Wow! Really? Okay.
Anna Salamon: My brain really thinks I should be adding some sort of “but” afterward, and I keep looking to see what the content of the “but” is, and my brain’s like, “But… I don’t like the rearrangement of egos that happens when I say that sentence!” And I’m like, “Great, brain. Can you give me something with more content?” And my brain’s like, “I think there’s probably something to say. Didn’t Leverage...”
And now I’m going to try it, because there’s sort of “epistemic status: kind-of-like rationalization”.
10. (2:41:45) “Stealing donors”, Leverage’s vibe, keeping weird things at arm’s length, and “writing off” on philosophical grounds
Anna Salamon: There was that incident at the beginning where everybody was very upset because Leverage was “trying to steal our donors” or something—like, going down some sort of donor list.
Geoff Anders: Yeah-
Anna Salamon: What was that? I don’t remember it.
Geoff Anders: Yeah, yeah. So it’s-
Anna Salamon: Anja or somebody brought it upthread, is why I remembered it at all.
Geoff Anders: Yes.
Anna Salamon: But once she did, I was like, “Yeah, yeah, yeah! That’s how we knew Leverage was bad and out to get us!”
So, I actually don’t think… Sorry. You can reply to that in a second. I’m just going to keep babbling for a second, though.
Geoff Anders: Yeah.
Anna Salamon: Actually I think there’s something different, though. I think Leverage came—and by “Leverage” in the beginning, I mean you, Geoff—came in with a different pattern of something that I think a lot of people had an immune response to.
Geoff Anders: Yeah. I agree.
Anna Salamon: And a different pattern of something—it was partly the particular way that you weren’t into a particular kind of materialism. The “pole through the head” thing—I can say this more slowly for people to follow it. It was partly that you… I don’t know, you told me a story when we first...
Sorry. I guess I’m inclined to just say all the things, but I-
WaistcoatDave: I think there’s a toxic narrative that critiquing is entirely fact based and therefore no time or focus should be put on HOW the points are raised. It’s clear the consequences of that has put Leverage staff on edge about entering conversations
habrykas: Content I remember: “Leverage reached out to a bunch of people on the top MIRI donor list, and some people were upset about that.”
habrykas: I think the people who were upset about that were kind of wrong. But I thought they were right for a while
sowgone: Cartesians vs Dennettheads
WaistcoatDave: and ultimately open conversations in the face of that fear and where that’s supported by the community, is how things are moved forward healthily.
Geoff Anders: I have interjections, but you know, it seems good. The thing I, you know-
Anna Salamon: You told me a story when we first met, I can’t even remember how it went, but it led me to think that you sometimes were, like, sort of more towards manic than most people. It was something about high school. You can fill in the details if you want, or ask me to, or we can skip it.
Geoff Anders: Definitely energetic.
Anna Salamon: It wasn’t—It wasn’t particularly [inaudible].
Geoff Anders: People have accused me of being hypomanic at times.
Anna Salamon: Yeahh… I was like, “Okay, this guy, I feel like he’s...” Anyway, whatever.
That, plus the materialism… I think you came in with a thing that a bunch of us—not so much me, although I went along with it-
Geoff Anders: Can I?
Anna Salamon: But a bunch of us wanted- Mm-hm?
Geoff Anders: Well, so, interestingly—I mean, we’re getting to a bunch of things. So, my experience was that literally within thirty seconds of our meeting, you had written me off on philosophical grounds because I… I visited SingInst, I got off the plane, I came out, I got picked up by you, and we started talking.
And maybe it wasn’t thirty seconds. Maybe it was sixty seconds. But literally the first thing that happened was that I was subjected to, in my perspective, a philosophical litmus test. I failed. And you were like, “Okay, not that one.”
And so that… You know. What do you think about that?
Anna Salamon: What do I think about that? I feel-
Geoff Anders: Do you remember it?
Anna Salamon: I don’t remember it, but I believe you.
Geoff Anders: Okay.
Anna Salamon: Um. Yeah. If you want to fill in more details, it might help me remember it. But probably [inaudible] gotten you from the airport.
Geoff Anders: No, no. It’s- No, I don’t remember.
Anna Salamon: Are you sure it was me, for that matter?
Geoff Anders: Yeah, absolutely. Yes.
Anna Salamon: Great. So, I don’t feel skepticism about it. So I do have feelings about it, despite not remembering it.
Geoff Anders: Yeah.
Anna Salamon: I remember later on, in what I assume was that same visit, in my head, we were sitting.
But what year was this? In my head, we were sitting in the house that I was living in in 2011, 2012 in Berkeley.
Geoff Anders: I do remember talking in that house a couple times. This was 2011. Yeah, this was in March 2011.
sowgone: I think I (Rob Bensinger) wrote you off immediately on philosophical grounds. (basically ‘Geoff is Cartesian + Geoff is’ confident’)
Anna Salamon: Yeah, but Rob Bensinger was not who got you from the airport, even though he’s saying that he remembers something similar.
Geoff Anders: No, I don’t remember Rob Bensinger being at SingInst at the time.
Anna Salamon: Rob wasn’t even around yet.
Geoff Anders: No, this was this right before Luke Muehlhauser came in as the ED of SingInst, which turned into MIRI.
habrykas: I think I docked Geoff a lot of points on philosophical grounds
DaystarEld: “By the time demon seances were happening, I think relationships between CFAR and Leverage had mostly come to a stop”—Ah, this makes sense, but is the point of the conversation then to try to clear the air about Leverage, or to try to help people understand why it diverged from the rest of the community? (I only joined the call in the last 30 minutes)
habrykas: I still engaged with Leverage a lot though, so definitely not “writing off”
spiracularity: On demons: Look, I think Leverage’s internal phrasing around it is toxic BS that is kinda bad for some people. I personally have never had anyone flip-their-shit when I explain social contagion, so feel free to reach out for phrasing-for-normals help if you want.
habrykas: But I think I also stand by that docking of points
sowgone: I wasn’t involved then, I’m just saying I am a MIRI person who wrote you off later
habrykas: Oh, that’s my docking of points
habrykas: Yeah, that’s what I said too
spiracularity: (I also think it’s bad and panic-inducing in ways that aren’t true; analogies with Roko warranted.)
Anna Salamon: Yeah. Sorry, I’m just going to randomly respond to parts of the chat without having read [inaudible]-
Geoff Anders: Fine, fine, all right, well.
Anna Salamon: Ollie Habryka is all like, “But I think I stand by that docking of points.” I haven’t even read his context, so I’m just responding. Personally, I stand by docking of points or something? Maybe?
What I don’t stand by is a “having written off”. And the reason I don’t stand by the “having written off” is to my mind, it resembles this thing I’m calling narrative addiction.
Geoff Anders: Yeah.
Anna Salamon: So, like, to respond to Geoff’s question from a minute ago: Geoff, you were all like, “How do you feel about it now?” How I feel about it now is I try to turn my mental eyes toward this memory, which is somehow located in my chest—maybe this is too much phenomenological detail-
Geoff Anders: Yep-
Anna Salamon: And I feel a cringe. I feel a mental cringe, or something, as I look. And I think the reason for it is that to my mind, there was an element of something like cowardice mixed into my response.
Geoff Anders: Really?
Anna Salamon: Where it wasn’t just my own authentic response to you seeming to me to be philosophically wrong about a thing. Sorry, and I don’t remember the event that you’re talking about-
Geoff Anders: No, no, that’s fine. That’s fine.
Anna Salamon: So I’m thinking of other events that are sort of in the same reference class. But to my mind, it was partly that I didn’t want the group to reject me. And that this was sort of part-
Geoff Anders: You mean, your group?
Anna Salamon: My group. I didn’t want my group to reject me.
Geoff Anders: Okay.
Anna Salamon: And so I was participating in having more of a “that’s weird and I need to keep it at arm’s length” reaction than I would have had in my own person. As opposed to a “That’s interesting! What’s with a smart person like you seeming to believe something dumb! Let’s have a conversation!”
Geoff Anders: Right.
Anna Salamon: Which is an entirely different reaction that doesn’t have the dissociating arm’s-length this-guy’s-a-weirdo component.
Geoff Anders: Okay. That’s-
Anna Salamon: Like, when you say “written off”, it’s more like that, I think [inaudible]-
Geoff Anders: Yeah. No, that’s super interesting because, I mean… Some of the people in the chats are talking about docking points or writing people off on philosophical grounds. I think there’s something like it. I think it can make sense to dock people points. Maybe in some cases writing off? It’s, like, a bit extreme, but I could… But there’s something like… There’s, like, a broader… I’m sort of responding to people in the chat also, but it’s like… There’s just this large corpus of philosophical works and texts from the history… You know, people are going to be writing me off for dualism. Note, I’m not even a dualist. But it’s something like, I think it’s usually not correct to see something like a point-sized view or something like that, and then write something off.
I also think there can be trends, but it’s something like… it would be good to crystallize what exactly is wrong. Like, if you could make a strong argument from… It’s like when people say, “Oh, well, so-and-so believes in God, therefore they’re bad intellectually or something.” And you’re like, “What about Newton?” And they’re like, I don’t know, something.
It’s like, if you can draw a strong link between “believes X” and “has various other features” then, like, sure. But I think that empirically that’s frequently not the case.
And even if there are general trends, you can recognize when there’s something different happening with a person and look into it more. So. Yeah.
But I appreciate the thing you said. I mean, I, yeah. It’s… Why did the group need to write off, like whatever, I mean, we got...
Anna Salamon: Yeah-
Geoff Anders: Robby Beninger said “Cartesian”, so, like, Cartesianism is really bad. And like, I know dualism was a really big deal. Like… why?
I mean, it is cool sort of in a way to have communities differ on the basis of really basic philosophical propositions, because, like, at least aesthetically you’re like, “Oh, you know, there’s deep philosophical commitments.” But I feel like we should be able to go meta and notice them and be like, “Okay, deep, different philosophical commitments-”
Anna Salamon: Well-
Geoff Anders: Yeah, go ahead.
11. (2:49:42) The value of looking at historical details, and narrative addiction collapse
fiddlemath: Anja notes a significant incident at the EA summit, 2014, where Geoff and Anna got into a … pretty heated? … argument about dualism. Seemed very emotional to both parties, given the degree of abstraction of the object-level conversation.
habrykas: Yeah, that was a big deal
habrykas: It caused me to have nightmares, and was a big component of me distancing myself from Leverage
habrykas: (Nightmares because I was interning at Leverage at the time, and it made me feel very alienated from my environment)
habrykas: (And felt like some kind of common ground was pulled out from under me)
DaystarEld: “(Nightmares because I was interning at Leverage at the time, and it made me feel very alienated from my environment)” - /hug
LuliePlays: “On demons: Look, I think Leverage’s internal phrasing around it is toxic BS that is kinda bad for some people. I personally have never had anyone flip-their-shit when I explain social contagion, so feel free to reach out for phrasing-for-normals help if you want.”—yeah I love Leverage’s psych tech from what I’ve experienced of it (workshops), it just seems like it would be odd if those weird-sounding things were not explicitly addressed + I’m curious how Anna views the surrounding context of that post
z1zek100: A bunch of people were obsessed with that conversation for whatever reason. Will McAskill repeatedly cited it as his most definitive critique of Leverage for example.
z1zek100: I was there and I really never followed the strength of the response.
habrykas: I think that’s kind of fair. I do think the conversation exposed some really big philosophical holes.
z1zek100: (This is Kerry btw)
larissa24joy: I’d find it really helpful to understand the emotional charge that seems to be associated with that (dualism) conversation for people?
habrykas: I think naturalistic reductionism is a really major foundation for my epistemology, and I do think it implicitly supports a lot of other common assumptions, and whenever I did into a bunch of my beliefs and plans, I do think a lot of them are pretty based on naturalistic reductionism
LuliePlays: and I’m finding myself wondering why they’re talking about the past stuff (which I find interesting) when there’s questions about the recent stuff and what CFAR’s relationship to Leverage is now
fiddlemath: Anja says she’d like to try to explain...
habrykas: And I think it’s pretty sensible. I do indeed think that an epistemology without naturalistic reductionism feels a lot less grounded, and I feel a lot more scared of having a group of people who “take ideas seriously” without being grounded in that way
larissa24joy: I’d love to hear from Anja on that if she didn’t mind and we had time?
sowgone: I think Cartesianism is kind of a hard-to-escape trap with lots of implications, like a dangerous meme that affects a lot of other stuff. I don’t think it’s morally bad to be Cartesian (you got trapped!), but I think there are so few Cartesians + it’s so bad, that I tend to write people off. Like a smart very devout Christian
AgileCaveman: imo, people tending to write each other off is a common pattern. I feel like i have been written off because of my theories of social contagion. I also wrote people off because of a rejection of biological substrate
habrykas: Agree that people trying and tending to write each other off is a common and kind of bad pattern
fiddlemath: The way I would have known not to reveal myself socially as a cartesian/dualist has to do with the tone and volume of Eliezer’s writing on the topic
sowgone: (BTW as a *separate* thing, I did the ‘Leverage is low-status so I want to avoid them’, which I don’t endorse and is bad.)
fiddlemath: Plus I remember something in the sequences about epistemic spotchecks being good in particular on philosophical topics
zwydswr: “I think naturalistic reductionism is a really major foundation for my epistemology, and I do think it implicitly supports a lot of other common assumptions, and whenever I did into a bunch of my beliefs and plans, I do think a lot of them are pretty based on naturalistic reductionism”—how is it a foundation for your epistemology?
z1zek100: @habrykas: That makes sense to me. I think I had a reaction that was something like “that seems wrong to me, but I don’t understand why Geoff thinks it and there’s probably an interesting story there at least” which involved not being very threatened about it in some way. I guess I generally expected people to have that reaction, but it could be that I don’t have the same emotional valence on this topic or something.
Anna Salamon: Yeah. I would also… Sorry, Larissa and the chat, yeah, I’m also pretty interested if anyone on the chat wants to explain about the emotional charge that it had for you personally—or, sorry, not for you personally but for the person in the chat who might explain.
Yeah, people are wondering about the current stuff. Lulie, I think I’m talking about the past stuff partly because Geoff mentioned in pre-call wanting to focus on the past stuff and partly because it’s genuinely easier.
Geoff Anders: Oh, we also, the thing is...
Anna Salamon: Right?
Geoff Anders: We also—we’re at, like—we’re at the 1:30 mark. I mean, so we did say we’d go sort of this long. I’m willing to go a little bit longer, but maybe we should shift gears and be like, “What’s the way forward?” Because—is there a way forward, you know...
Anna Salamon: I… I’m… yeah. I think we’re all hurtling forward in time. So there’s clearly some ways forward-
Geoff Anders: Love it.
Anna Salamon: I don’t know toward where. I mean, personally, I’m really into history and conversation and sharing all the details. And I think once we do that, we’ll find a real way forward to somewhere good. And if we don’t do that, I don’t trust our ability to reason or get anywhere.
I also think sharing all the details is the opposite of narrative addictions. Or, like, it’ll help dismantle them. And I think it’s in fact compatible with morale or with the good parts of morale, in the long run. I may be wrong.
I could be wrong, but I don’t think I’m wrong.
Geoff Anders: I think I agree with like 80% or 90% of that or something. It’s like, I think there are some things… It’s not like everything needs to be hashed out. I think there are problems other than ones addressed by the history. But I do think history is really undervalued, to put it that way. And I’ve gotten value from looking into some of the things. Yeah, I...
Anna Salamon: I mean, I think we should-
Geoff Anders: Go ahead.
Anna Salamon: Yeah. I guess I’m of the hopeful-to-my-mind opinion that many of the things I’ve been calling “narrative addictions” and “narrative pyramid schemes” and so on are sort of on the point of collapse, or have already begun to collapse. And that maybe we’re heading into a place where we can have real conversations across people, again.
Geoff Anders: I’d love that. I think that’d be great.
Anna Salamon: Time based more in a grounded thing and less in a “I’ll support your crazy scheme if you support mine” kind of deal that I don’t think it’s how we build, in the real world for the actual situation.
12. (2:52:23) Geoff wants out of the rationality community; PR and associations; and disruptive narratives
Geoff Anders: Yeah… Something like that seems generally good. But I do want to say something like, there’s a way that I feel like I’m… You know, I sort of want to say, like, “How do I get out of the rationality community?”
Like, I’ve never considered myself a member. I don’t read LessWrong. People are like, “Did you see the whatever?” and the answer is like, “No, no I didn’t.” And I’m running an organization and we’re going to do all sorts of other stuff. And so it’s not that I don’t see value. There is potential value. But I… it’s like the whole thing is sort of a nightmare in a certain way.
Like, how many people inside the rationality community—like, sum up, do an integral or something, on their attitudes towards me; and it’s like, that doesn’t give them a right to actually interact… Like, the fact that someone’s mad at me doesn’t by itself cause it to make sense for me to engage.
I do think some of the things are my fault. Absolutely. I—theories about various things… Like, sounds like I said some bad stuff about you in the past in a way that you didn’t find toward. Like, I...
Anna Salamon: Well, but sorry. I was mentioning that partly as an example of ways that I used to be doing a thing that I think was not that great.
Geoff Anders: Well-
Anna Salamon: I had mixed feelings about it.
Geoff Anders: No, but I guess what I mean to say is, like, you have a claim or a stake. And then maybe the rationality community has some stake.
Like, definitely one thing I’ve read a little bit about was, seemed like people were doing a memory consolidation sort of event. Like, I think that’s great. I would love the rationality community to do more of X, whatever that is.
And then I feel like it makes total sense for you to want to go into all the details, because you’re much more closely tied to the rationality community. You’re part of it, a leader, et cetera. I don’t know if you would go for leader, but “leader”, definitely.
How do I set up my relationship properly so that, like… I don’t want people thinking that I’m going to, like… Like, there was one comment somebody directed me to—actually, I think, was it Spiracular? I think Spiracular’s here in the chat.
Anyway, there was some comment that was like, the person’s like, “I’m going to make a stand. There’s all this fear, and somebody needs to stand up. And so it might as well be me.” And I’m like, “Well, that’s good.” You know, I like that. But it’s really different than my reality.
And so I don’t really know how to navigate this sort of circumstance. Do you have any thoughts?
spiracularity: If you critique the rationalists, you are one of them. :P
spiracularity: It’s how we roll.
sowgone: BTW I’d love to have a LW thread/post where we can argue about Cartesian (esp. ‘trust first-person stuff a lot’) stuff
AgileCaveman: you used rationality to critique rationality, it’s inside of you now. A classic blunder
sowgone: (‘write off’ for me doesn’t mean ‘war’ / ‘don’t talk’, it means they’re a friend I don’t expect to do amazing things)
benitopace: “you used rationality to critique rationality, it’s inside of you now. A classic blunder”—hahahaha
sowgone: :o
z1zek100: “you used rationality to critique rationality, it’s inside of you now. A classic blunder”—NotLikeThis
Anna Salamon: Sorry. So I think you just said, “How do I get the rationalist to be bored by me, or to otherwise leave me alone?”
Geoff Anders: It’s not “be bored by me”. It’s like—I think Spiracularity just said, “If you critique the rationalists, you’re one of them”! Yeah. That’s a viewpoint. Not one I endorse...
Anna Salamon: Yeah, to my mind you seem very obviously part of this broader community. Partly because, yes, you critique. Partly because you recruit from. Partly because-
Geoff Anders: Well, “critique or recruit from”—like, but when? Like, what years? Like, not Leverage more recently, right?
Anna Salamon: Yeahh.
Geoff Anders: And like- Yeah.
Anna Salamon: I don’t have a good solution for you here.
DaystarEld: To me “Why is dualism bad?” can be answered pretty easily with something like “It can reliably lead to people doing things like demonic seances.” I’m not trying to be fighty by saying that, it just seems like a pretty straightforward relationship in my experience/observations of distinguishing philosophies that are more or less likely to go down dangerous paths
DaystarEld: (I get that “demons” can just be a language handle on psychology stuff, but what was described is much more than just a semantic handle)
z1zek100: “To me ‘Why is dualism bad?’ can be answered pretty easily with something like ‘It can reliably lead to people doing things like demonic seances.’ I’m not trying to be fighty by saying that, it just seems like a pretty straightforward relationship in my experience/observations of distinguishing philosophies that are more or less likely to go down dangerous paths”—That kind of implies that once you believe in dualism all bets are off or something and I don’t quite see why that ought to be the case.
habrykas: “I think naturalistic reductionism is a really major foundation for my epistemology, and I do think it implicitly supports a lot of other common assumptions, and whenever I did into a bunch of my beliefs and plans, I do think a lot of them are pretty based on naturalistic reductionism”—My guess is this would take a bit longer to explain. Short answer is something like “It’s what allows me to have a shared foundation for all the different methods of knowledge gathering that I use, from emotions to scientific experiments to expert deference, and is something like the foundation of how I sanity-check different parts of my map against each other”
329zwydswr: how was it different than a semantic handle and how is that related to dualism?
AgileCaveman: i am SUPER interested in learning more about demonic seanses. I am not at all turned of by the label “demons.” I am interested in “did they work?”
habrykas: Also, like, a bunch of your staff was like working at CEA until 2017
larissa24joy: 2 of the 4 staff (me and Kerry) were at CEA until very early 2019.
spiracularity: I was making a pun off of the fact that we use the term for both the philosophy stance, and the community, though. (But also the community kinda loves its best critics.)
habrykas: So at the very least you shouldn’t be surprised to be understood as part of the community if you are literally sharing staff with one of the central community organizations
habrykas: Sorry, meant 2019
spiracularity: Like, the fact that these are different didn’t escape me.
spiracularity: (end explaining the joke)
larissa24joy: So if you’re ever involved in EA or Rationality you can never leave?
Geoff Anders: Okay. Well, I beseech the powers that be inside of rationality—Anna Salamon, Oliver Habryka, others—Robby Bensinger, et cetera, Ben Pace: I’d like some sort of reasonable exit strategy. I think you guys don’t, like… I just get crazy messages where it’s like, people don’t want me anywhere around and then they don’t want me to leave. That’s not cool. That’s, you know-
Anna Salamon: The same people, or different people?
Geoff Anders: I, you know, I’m not even in it enough to know. Okay. And so-
Anna Salamon: Well, sorry, did you get a message from some person Bob who’s both saying “stay around” and “get out”, or do you get a message from Bob saying “stay around” and Carol saying “get out” or whatever?
Geoff Anders: I think that… I mean, I’m loathe to point to particular individuals...
Anna Salamon You don’t have to name them.
Geoff Anders: … But yeah, I think I’m getting mixed messages from at least one person! And it’s something like… There should just be—I mean, this gets to like a thing that I think is a bigger issue with the rationality community. Like, there’s all this stuff about Leverage managing its PR—like, PR matters in the actual world. Like, we want to do projects with people. They don’t want to—you know, it diminishes the likelihood of their wanting to be associated with you if-
Anna Salamon: Yeah, I… have an important disagreement with you here that would take-
Geoff Anders: No, I agree it would take a while, but something thing like: How do we… I think there’s some way that people want, like… I want something a bit more concrete. I want somebody to figure out what I’m supposed to do, and I want to have something I can interact with.
habrykas: As long as you have power over people, they will want you to be held accountable somehow, and want you to play by their norms. And you’ve had large amounts of power over people for many years.
z1zek100: Over Rationalists specifically?
habrykas: And EAs
z1zek100: That could be true, but it would be a bit surprising
spiracularity: “As long as you have power over people, they will want you to be held accountable somehow, and want you to play by their norms. And you’ve had large amounts of power over people for many years.” - +1
habrykas: I mean, Leverage shared staff with CEA for like 4 years, and was intimately involved with almost all EA Global conferences.
habrykas: So why would it be surprising?
Geoff Anders: Look, if it’s a mob, if there’s no way to stop it, right, like, people, I would appreciate a little bit of clarity on that. We could have you and Habryka and other people just get together and issue an official formal statement that says the rationality community is going to continue doing this until they happen to personally feel satisfied. And then I can let my people I work with know that I’ve got this rival/enemy that is planning to constantly interact with me and there’s really not a way out. And then I can at least be transparent to them about that.
Anna Salamon: Ah. Yeah, sorry, I feel like there’s so many threads and-
Geoff Anders: Yeah, no, I mean it’s—and I don’t mean to say that I wouldn’t put in time to addressing problems or concerns and so forth. But there’s some people for whom the rationality community, like, is the world, and, like, that’s the thing they care about. And I think that’s fine. I think that’s, like—people have communities. Communities seem fine.
Anna Salamon: Yeah, sorry, you’re adding more and more to your thing. [crosstalk]
Geoff Anders: But—Okay, cool. So it’s too hard, alright, alright. Well, it seems like there’s—so, we didn’t manage-
Anna Salamon: Oh, I’m going to try to reply anyway, I was just [crosstalk]-
Geoff Anders: Go ahead. Sure, alright.
Anna Salamon: -how it’s too many threads to reply to you and you were like, “Here’s some more threads!” And I’m like, “Okay, great.”
Geoff Anders: Alright.
Anna Salamon: ”Here my new preface again.” “Here’s some more threads!” And I’m like “Gaahhh!”
Geoff Anders: I’m happy to let you say a thing.
Anna Salamon: I don’t know. It seems to me like there’s a bunch of different things going on. So, personally I feel like Zoe writing that public essay is sort of an invitation for me to take an interest in what happened to that part of Leverage, and an invitation to a bunch of other people.
Geoff Anders: Yeah.
Anna Salamon: An invitation from Zoe. Like, otherwise I’d be like, “Oh, well, that’s not exactly my business.” Like, I don’t know whether I would or wouldn’t be that way; but I feel like somebody who was in Leverage if 2018, 2019, sort of invited me and a bunch of other onlookers in, in my opinion.
Geoff Anders: I agree with that. I accept that.
Anna Salamon: So that’s part of it. Then there’s a different part that is something like: Everybody has the right to free speech and free thought. Particularly LessWrongers. And I’m not going to be trying to rein them in, and probably neither is anybody. And if they do, I’m probably going to oppose them. And that might get in the way of your PR, as you put it. And I actually feel bad about that, but I don’t think it’s my problem. Or, like, I don’t plan to take any interest in it. (Like, I’m interested. I get to watch!)
Geoff Anders: No, I think that’s fine. The-
Anna Salamon: And then there’s a different thing, which is… Okay, here’s my most controversial belief on this topic. Mostly not vis-à-vis you.
So let’s talk about Eliezer for a second. Eliezer has huge communities of people who hang out complaining about him all the time. Not really, that exaggerates slightly—but, like, Sneer Club and so on.
Geoff Anders: I was thinking about this. Yeah. How does Eliezer get out if he wants to?
Anna Salamon: He can’t! Sorry, he could, but he won’t be able to. Here’s my story about why it’s happening to Eliezer.
Geoff Anders: Okay.
Anna Salamon: As I said, this is my own—I don’t know, I haven’t put this take out there before. I don’t know if it’s right.
Like, it seems to me that Eliezer… Let’s talk about narratives. Lots and lots of people have personal narratives that are something like, “I’m going to be a school teacher and then I’ll help a bunch of kids and that’s good. And that makes me a good person and means I did something useful with my life.” Or whatever. People have all kinds of, like, little local narratives.
Eliezer put these sequences out there, and they’re just, like, a massive vortex that disrupts a huge number of people’s local narratives.
Geoff Anders: Right.
Anna Salamon: And then some of those people hang out and get really upset.
Geoff Anders: Yes.
Anna Salamon: And this is the sense in which… And there’s a co-dynamic here… I don’t know. Look, I’m probably going to go to various kinds of Said Something Wrong Out Loud On The Internet In Front Of Fifty People Hell for this remark. But, like, it seems to me that Eliezer’s narratives… I, sorry, I love and admire Eliezer more than almost—like, he’s one of my favorite people in the whole world. But also, I imagine when I read the sequences that they’ve got kind of an edge, and that it’s not just a presentation of facts, that there’s sort of a reinforcement of his own narrative in there; I could be wrong about this.
Geoff Anders: Yeah.
Anna Salamon: And I think the reinforcement of his own narrative is sort of part of why it erodes other people’s narratives so much.
Geoff Anders: Yeah.
Anna Salamon: And it’s part of why a bunch of them then try to get vengeance.
Geoff Anders: Yeah. Yeah, yeah. Totally agree.
Anna Salamon: I think there’s something a little like that about you and the rationality community that’s happening on a somewhat smaller scale. And this is part of what’s up with it. Although I think the thing about “Did Leverage actually harm some people, and were we invited into that room to try to help sort it out and make sure that there’s, like-”
Geoff Anders: Yeah, yeah. Right.
Anna Salamon: ”-restoration of things that those people might want or need,” is a different and pretty legitimate point.
So those are the three things that I have.
Geoff Anders: Okay. Yeah. Well, so, I agree with the harms and being brought in-
Anna Salamon: Oh wait, sorry.
Geoff Anders: Oh, go ahead.
Anna Salamon: There’s a fourth thing I really want to say. I’m sorry. Is that okay?
Geoff Anders: Yeah.
Anna Salamon: The fourth thing is: Rationality was never trying to be an organization in the way that Leverage is trying to be an organization. At least, I don’t think so. Like, I think if I had been like, “Hey, Geoff, your Leverage is messing up my life. How do I get out?” You would have tried to be the responsible person at the front who could have had an answer for me.
Geoff Anders: Right.
Anna Salamon: Rationality is really not trying to be the kind of thing that can have a responsible person at the front who has an answer for people.
DaystarEld: “how was it different than a semantic handle and how is that related to dualism?”—From my understanding of Zoe’s post… if people treat semantic handles for psychological phenomenon as real enough to be affected by using crystals, it starts to leave the realm of semantic handles and enter the realm of irrationality
sowgone: motte: ‘Geoff, stop doing abusive actions X, Y, Z.’ bailey: ‘Geoff, stop being weird. And given that you’ve ever been weird, stop trying to improve the world.’ ?
329zwydswr: “It’s what allows me to have a shared foundation for all the different methods of knowledge gathering that I use”—ah ok this makes sense. hard to discuss here, but i want to note that this i think is actually compatible with some kinds of dualism. eg saying “beliefs aren’t material things” (or maybe easier to agree on would be, “algorithms aren’t material things”) is compatible with “everything that happens, happens in accord with physical law”
habrykas: “ah ok this makes sense. hard to discuss here, but i want to note that this i think is actually compatible with some kinds of dualism. eg saying ‘beliefs aren’t material things’ (or maybe easier to agree on would be, ‘algorithms aren’t material things’) is compatible with ‘everything that happens, happens in accord with physical law’”—Yeah, agree, though with many many caveats. I have a tiny steelman in my mind that’s trying to map between Tegmark 4, Platonic Realism and Cartesianism. Though I’ve never gotten it to actually fully typecheck and make work.
329zwydswr: “as real enough to be affected by using crystals.”—well… this is clearly a mistake, but does this really argue against they (the psychological phenomena) being really real?
329zwydswr: like, is talking about “having beliefs” a form of dualism? seems fairly dualistic, in that beliefs aren’t material things. doesn’t have to be irrational / nonsensical / crazy
DaystarEld: “well… this is clearly a mistake, but does this really argue against they (the psychological phenomena) being really real?”—I think there’s a certain type of thinking that *leads* to those mistakes, and we should know that by now
329zwydswr: “Though I’ve never gotten it to actually fully typecheck and make work.”—this problematic seems to be something in the air, there’s maybe an idea here who’s time is coming.
AgileCaveman: One thing that’s somewhat frustrating about the question of “did leverage harm people” is the misunderstanding of “harm base rates”. I have a couple friends severly harmed by rationality or working at normie tech companies. there is a high baseline of people being harmed in the world and the question of whether Leverage is above this baseline is harder than “did it harm people”
habrykas: Agree with this. I think establishing baselines is super important.
329zwydswr: “a certain type of thinking agreed”—i’m curious wat it is, and it doesn’t seem like dualism fits, but maybe i’m missing the point.
DaystarEld: “One thing that’s somewhat frustrating about the question of “did leverage harm people” is the misunderstanding of ‘harm base rates’. I have a couple friends severly harmed by rationality or working at normie tech companies. there is a high baseline of people being harmed in the world and the question of whether Leverage is above this baseline is harder than ’did it harm peopl’e”—Agree with baselines too. But one of the reasons (for example, sorry to use the C word) cults are bad is that they can do targeted harm in excess to baseline organizations to a few people beyond what a randomly selected control group might
DaystarEld: Even as most people have positive or neutral experiences
Geoff Anders: Okay. I’ve got some thoughts. We still have viewers, so we’re still going.
So, first I should say, I do think that there is a legitimate public interest in the things that Zoe said. I think that those be pursued. I think the people that want to pursue them are doing the right thing, or doing an acceptable thing. And so I’m not trying to push back on that at all.
On the “Eliezer created the narrative and is still reinforcing it, and then the people are stuck inside it and it erodes their narratives thing,” if there’s a thing that’s similar that I’m doing, I want to find it and stop. I hereby offer that I will spend time doing that. And I would love to have one or more interlocutors who could help to try to crystallize what it is, so as to make this process more effective. I don’t know if Eliezer believes that something like that’s happening. I find it plausible that something like this is happening, even though the interactions with me are obviously less thoroughgoing than with Eliezer from the rationality community or from rationalists.
So, but I hereby offer to try to resolve that. And you know, I’m not sure that it’s there, but if it’s… I’ll look, I’ll do this myself. I don’t need other—and I would like help from relevant other people who maybe have a bit more insight into how people are thinking about it, because one of the things that’s harder for me is I don’t in fact track the hive-mind of LessWrong, if it is a hive mind, or the non-hive-mind of many, many individuals who… something, like, I don’t know.
But, yeah, so that sounds good. And then—you said a fourth thing that seemed super important. What was the fourth thing?
Anna Salamon: Not trying to be that kind of thing, that Leverage I think was trying to be, where there’s somebody at the front who can take calls.
Geoff Anders: Ah, so here’s-
Anna Salamon: CFAR maybe is, LessWrong maybe is, but, like, the rationality community, no.
Geoff Anders: Well, but this is where I think we have a disagreement, but one that’s not like a deep philosophical disagreement about Cartesianism or something like that. It’s a disagreement about social technology.
I think that the correct form for communities involves community leaders and involves walls of particular types. I think that that’s actually from the nature of a community. I think communities—I’m not sure about this, but I think communities come together over shared strengths and shared weaknesses. The community offers an opportunity for the people to, like, collect and grow the shared strengths, and to help protect the people from the shared weaknesses.
And then I think that that’s—like, the reason you need walls is because there is the shared weakness and the walls help to protect the community. And then, in terms of leaders, I think that communities, because there are certain types of shared weaknesses and blind spots, they’re really subject to predators.
And so I think something like “Communities need to have leaders,” and the challenge of the leader is to do their best to try to handle the predators.
But this is a claim about the nature of communities. And as a plug for my Twitch stream, I’m going to be talking about coordination research, which includes looking at different forms of social technologies, including communities. So I’m going to be exploring this.
But I think that there’s actually, like, a positive claim here, not a normative claim, not a philosophical-whatever claim, there’s a claim about a particular social form and its correct thing. And so maybe there’s a fruitful line of discussion to have there.
Anna Salamon: Yeah. I’m pretty into this conversation.
Geoff Anders: Great. All right. Well, I feel like we had a really natural breaking point. All right. It took a little bit longer, and I apologize to everybody for the technical difficulties. Those will get better.
Okay. I think this was fine, right? All right. Good. Okay! Awesome. Thanks, everybody.
I will see… Maybe people will be less interested in my philosophical whatever. I don’t know how we’re gonna do better. This was good. Anyway, whatever. [crosstalk] I’ll send out a tweet. All right. I’ll-
Anna Salamon: I appreciate you chatting.
Geoff Anders: Great. All right. I’ll see you later-
Anna Salamon: And all the people in the text.
z1zek100: This seemed good
sowgone: this conversation was amazing, thank you
z1zek100: Love you both
329zwydswr: thanks
habrykas: I appreciate this conversation happening.
larissa24joy: Thank you for doing this Anna and Geoff!
LuliePlays: Thanks Geoff and everyone!
Turbowandray: bye
habrykas: Sorry for not being able to fix the audio things
Transcript for Geoff Anders and Anna Salamon’s Oct. 23 conversation
Geoff Anders of Leverage and Anna Salamon of CFAR had a conversation on Geoff’s Twitch channel on October 23, triggered by Zoe Curzi’s post about having horrible experiences at Leverage.
Technical issues meant that the Twitch video was mostly lost, and the audio only resurfaced a few days ago, thanks to Lulie.
I thought the conversation was pretty important, so I’m (a) using this post the signal-boost the audio, for people who missed it; and (b) posting a transcript here, along with the (previously unavailable, and important for making sense of the audio) last two hours of chat log.
You can find the full audio here, and video of the first few minutes here. The full audio discusses a bunch of stuff about Leverage’s history; the only part of the stream transcribed below is the Anna/Geoff conversation, which starts about two hours in.
The chat messages I have aren’t timestamped, and sometimes consist of side-conversations that don’t directly connect with what Geoff and Anna are discussing. So I’ve inserted blocks of chat log at what seemed like roughly the right parts of the conversation; but let me know if any are in the wrong place for comprehension, and I’ll move them elsewhere.
1. (1:57:57) Early EA history and EA/Leverage interactions
[...]
Anna Salamon:
All right, let’s talk.
Geoff Anders:
All right, all right. Yeah, let’s get to it.
Anna Salamon:
I hear that the rationality and Leverage communities have some sort of history!
Geoff Anders:
That’s right. Okay, so basically, Anna, as I mentioned to you when we chatted about this, I wanted to talk about in general Leverage history, and then there’s a long history with the rationality community, and right now, it seems like there’s… I’d like to understand what happened with the relations, basically.
Like, earlier on, things were substantially more congenial. More communication. There were various types of joint events. Ben, who suggested he would be called by first name, organized the first solstice for the rationality community. He did that while working at Leverage. That was totally fine. I volunteered at the CFAR workshops. Ben also did that. Then, EA Summit 2013, I thought was sort of a big deal. We had you guys sort of do a half-workshop during that. I thought that was great. Ben reminded me that we actually held a book launch for MIRI somehow. I remember the event very vaguely. Yeah, but it was at Luke Nosek’s house, and I don’t know. Maybe it was a joint… I don’t remember, but there was a bunch of stuff.
Let’s see. All right, yeah, so I just switched my audio. Ali, let me know if that’s better or worse.
But anyway, so there was that, and then I feel like we just had lots of interests in common and so forth, but then somehow, things didn’t end up working out as excellently as possible. So I was interested in just chatting about what happened, getting some common context on the history, maybe trying to...
Geoff Anders:
Oh, yeah. Maybe Our Final… Yes! Yes! I think that was the book. At least, that sounds pretty likely.
Yeah, so basically looking a bit at the past, and also trying to understand where things are presently, and maybe figuring out whether there’s anything constructive to be done. Like, in some circumstances it seems like trust is so low that I’d love to know what could make things better, basically.
So anyway, I just talked for a bit. Yeah, so let’s talk about it.
Anna Salamon:
Yeah. Are we still trying to end at 1:00, in 20 minutes?
Geoff Anders:
I mean, I’m willing to go on longer. I know you had a thing, but you know-
Anna Salamon:
Oh, I don’t have a thing.
Geoff Anders:
Oh, okay. Yeah, so-
Anna Salamon:
Yeah.
Geoff Anders:
Apologies to all the viewers and so forth, but yeah, let’s go to 1:30.
Anna Salamon:
Great. So yeah, I don’t know. I guess one thing I want to know...
Look, I’m going to try to tell history. I’m going to get it wrong. There’s 50 other people out there who are going to have 50 other histories. Maybe we can get them all out there via mine and Geoff’s to start with, or something.
But, so my perception is—I want to talk about that 2013 summit, where we all… like, 50 people or something stuffed into a giant Leverage house for like, I forget, several days.
Geoff Anders:
Yeah, I think it was 60-
Anna Salamon:
A week?
Geoff Anders:
… I think, and it was a week, yeah. It was about seven days long, I think.
Anna Salamon:
And these people were CFAR, and Leverage, and MIRI, and a bunch of EA-ers from overseas?
Geoff Anders:
Yeah, we-
Anna Salamon:
Who else was there?
anna_salamon: Who here was there, at the 2013 stuff into leverage house EA retreat?
anna_salamon: Anyone but me and Geoff?
larissa24joy: https://www.youtube.com/user/nnevvinn/videos
fiddlemath: Anja’s here with me, and was there
larissa24joy: Videos from it above
Geoff Anders:
Well, we tried to invite basically a selection of everybody. You know, Holden Karnofsky talked there. I think we beamed in Jaan and Singer for talks. Thiel talked, but then we had-
Anna Salamon:
Oh yeah, in person.
Geoff Anders:
Yeah, yeah. Then, we had-
Anna Salamon:
For several hours. I’m remembering as you’re saying the things.
Geoff Anders:
Yeah, yeah. And then we… I remember when we designed it, we tried to basically balance it to have people who were interested in the different parts of EA. I have some spreadsheet somewhere, but we tried to… We flew in a bunch of people from the UK, maybe a couple other places, but something like that. And obviously not everybody invited came, but yeah, we had… I feel like we had a mix. Like, for people who are interested, the-
Anna Salamon:
Was Toby Ord there?
Geoff Anders:
No. I don’t think-
Anna Salamon:
I feel like CEA was somehow proto-there. Maybe I’m making that up. Okay. Great. [inaudible] says I made that up. I probably did.
Geoff Anders:
I think some CEA-affiliated people were there. I don’t think the main people came, though they were invited.
There’s a video online that, yeah, if you go to YouTube and I guess search 2013 EA Summit-
Anna Salamon:
Anyway, great, great, great. I am in a hurry.
Geoff Anders:
Yeah, go ahead.
Anna Salamon:
Even though we have until 1:30.
So the thing I want to say about that—which I’m not sure how many people would agree with me—but my own narrative is something like that there were a bunch of different people working on and having strands that were kind of related, and we were trying abstractly to think well with each other, because we all knew it was good to be cooperative or something, but we didn’t have that much actual flesh on our cooperation.
And then my narrative is that Geoff or Leverage or somebody was like, “You know what would help? I think it would help if we all spent a week stuck in a house together.”
Geoff Anders:
Yeah.
Anna Salamon:
And it did, and then we emerged all friends somehow. Like, not exactly friends, but all actually feeling like there was a thing that was in common, that we might call the EA thing, and that it was worth talking to each other. And a bunch of people got the AI risk thing, who came in skeptical about the AI risk thing. And I don’t know, I was very self-centered about the AI risk thing. I’m not sure how many...
Anna Salamon:
No, no, no, it wasn’t… Oh, you were at 2014. Yeah, yeah. There was a previous one. The 2014 one was much larger, and more disparate. I liked it too, but it was like we were… Sorry, somebody in the chat was mentioning 2014.
Geoff Anders:
Well, we did-
Anna Salamon:
It was-
Geoff Anders:
If I could just jump in with one fact.
Anna Salamon:
Okay.
Geoff Anders:
We did both the week-long thing at a retreat place, and then we did a two-day event, like the Summit, so the retreat was like-
Anna Salamon:
In 2014.
Geoff Anders:
Yeah, so we did both in 2014.
Anna Salamon:
Yeah, but even the week-long thing to me seemed, in some ways, less good than the week-long thing in 2013, because it was spread out instead of being in one house.
Geoff Anders:
I agree. Yeah.
Anna Salamon:
Which was maybe necessary because there were more people or whatever.
But anyway, from my perception, there was a lot of camaraderie in the early days, even like 2008, 2007, up through 2013, but the camaraderie sort of congealed and became more of a thing across more of the people that week in the house. And there were always tones of skepticism, but the tones of skepticism were maybe at a local low point right after that week from my perception.
And then we have gradually, not just Leverage and the rationality community—whatever that is, sorry CFAR, LessWrong, I don’t know, MIRI -
Geoff Anders:
I… think there’s a community.
Anna Salamon:
What? There is a community, but it’s a community that’s… Yeah, there’s a community, but there’s also several different partial centers of the community or something.
Geoff Anders:
Agreed, yeah.
Anna Salamon:
So I wasn’t quite sure which one I was trying to talk about.
Anyway, so Leverage and various strands of the rationality community were pretty harmonious there in my perception, and were a little less so before, and much less so gradually after.
But also like, I don’t know—CFAR and CEA, for example. In my head, there was more of the congealment there. Like, we were all friends back then.
And now we’re all… maybe friends, certainly trying to be harmonious—many of us, maybe not with Leverage, maybe Leverage is our official “figure out whether we’re supposed to ostracize this month” target, we can talk about that...
But I guess I just wanted to situate the puzzle of how rationality and Leverage drifted apart, with a puzzle of: I think that in general, most of the organizations think that most of the other organizations’ work isn’t that good, and usually do try to be cordial and harmonious, but I think that things somehow started out much more “everybody believing in cooperation with each other”, and are much less that way now, even though there’s quite a bit of cordialness still.
And I guess I want to situate the splitting of Leverage with CFAR or whatever, in that larger context where I think most things split from most things. And I’m kind of curious about what was up with the larger context.
Geoff Anders:
Yeah. I mean, I agree with this broad picture. I wouldn’t put the start of EA to the EA Summit, though I think people could argue about that. I think EA started before. But apart from that, I agree.
I think there is something like most of the groups have become more disparate. It’s really hard to figure out how to say this without sounding biased in my own favor, and also while there’s lots of concerns, and a bunch of sort of justifiable things, it’s really hard to say things like this, but in 2012 and 2013 and 2014, we were working hard to try to get the different groups to be able to work together.
Like, 2013 we did the Summit. 2012, we did THINK, the high impact network. Where we wanted to have CFAR, The Life You Can Save, 80,000 Hours, and Giving What We Can and Leverage all collaborate. And that didn’t end up working. But then the EA Summit did.
Yeah. And then, I feel like the cordial relations and the “more on the same team” thing extended through 2014.
For my own story, just to add a part to it, I think that I stopped wanting to… like, stopped engaging as much with CFAR after CFAR stopped doing as much research? Like, CFAR was doing more research early on, and then the research diminished over the course of time. That was something that mattered for me, personally.
But I mean even, I think in 2018 we did a Paradigm workshop specifically for CFAR people to try to, like, something. And I talked to the people about, “Is there room for some sort of alliance or working together?” And that didn’t go anywhere, but who knows?
Anna Salamon:
Huh. I don’t remember this. Was I invited?
Geoff Anders:
No, no, this was the-
Anna Salamon:
Is it specifically for CFAR people...?
Geoff Anders:
I think at the time you were… I haven’t always tracked when you’re in charge of CFAR. Like, there’s at some times...
Anna Salamon:
But I’ve been working at CFAR continually.
Geoff Anders:
Yeah. So-
Anna Salamon:
I wasn’t ED, but I was totally working there.
Geoff Anders:
Okay. Yeah. So I-
Anna Salamon:
I’ve been in a leadership role.
Geoff Anders:
Okay. Yeah. So this was a reach out to Eli Tyre et al., I think.
Anna Salamon:
Okay. Yeah, yeah.
This is not very charitable of me at all, but I think I had a perception that you were trying specifically to recruit our junior staff and that you weren’t that interested in talking to people like—by, recruit, sorry, I don’t mean to pull them away from CFAR, but I do mean to sort of ideologically recruit them—but that you weren’t that interested in me because you were less likely to get that much of my mind or soul or something. Confirm or deny?
Geoff Anders:
Yeah, no, it’s a good pointed question.
How do we draw a distinction between “get you ideologically” and “have constructive conversations, wherein both sides update”?
Anna Salamon:
Um. I don’t know. I do think there’s… Well, what do you think there… I think these are quite different. I think they’re both great things, depending on context.
Geoff Anders:
Sure.
Anna Salamon:
Especially the second one.
Geoff Anders:
Yeah, yeah.
anna_salamon: Anja, would love to hear if your perceptions match/differ, what you remember
fiddlemath: Anja: matches up so far
fiddlemath: Anja is carrying the baby, but—she has a pet theory that a lot of the friction / tension traces back to Leverage overstepping boundaries with their recruiting practices.
AgileCaveman: The vibe here is two people talking about a failed relationship, except it’s two large orgs.
habrykas: I can confirm memories of tension that was associated with Leverage’s recruiting practices
habrykas: From the 2014-2016 era
Anna Salamon:
I often learned from conversations with you, though. I remember going over to your house a bunch of times with Nate, it seemed good. You seemed less interested.
Geoff Anders:
What year was that?
Anna Salamon:
That was, I’m going to make this up, my made-up year is 2016 or 2017.
Geoff Anders:
Yeah, I-
Anna Salamon:
Remember?
Geoff Anders:
I was talking - [crosstalk] - I was talking to somebody who jogged my memory on that. And I do think that was less important to me at the time. Like, the...
Anna Salamon:
Okay, so my yeah jerk story as to why we drifted apart—I can tell jerk stories that emphasize me as the bad guy, I’m going to emphasize you as the bad guy -
Geoff Anders:
Well, y’know...
Anna Salamon:
- but you’re welcome to turnabout or whatever.
Geoff Anders:
Yep.
Anna Salamon:
My jerk story is, you were busy trying to build an empire for reasons, that you thought would be able to build toward good things, or whatever. And we didn’t look like good ways for you to do that. Partly because we related differently to power than you do, or power, closure, how much of your stuff to keep inside in what way. And partly because, as you say, we were not doing that much of the kind of psychological research that you guys were trying to do, so we weren’t particularly project-aligned in that way. And I think it’s cool that you guys were trying to do that kind of thing. And I think in CFAR, a lot of us were curious as to what your stuff was and whether it was cool. So I think that was one of the things.
Anyway, I think you were trying to build a thing with power where you could control it, and I think alternate perspectives were not that interesting to you on a visceral level for that reason. And this is part of why we drifted apart.
Geoff Anders:
Okay.
Anna Salamon:
And I think I looked more like a… kind-of-like-resource-kind-of-like-obstacle, whereas some of the junior staff looked more like people you could recruit.
Geoff Anders:
Yeah. But then, you said “recruit”. So, if I could ask some questions about this. So, you said “recruit, but not hire”.
Anna Salamon:
Well, maybe hire, but “hire” isn’t the important part.
Geoff Anders:
Okay. Right. So then I guess my question is something like: I do end up forming views on who I’ll be able to work with, more and less and so forth. And then, I mean… there’s, like, a nefarious-themed-ness to the thing you’re saying. But how does the...
Anna Salamon:
Should I try to actually spell out what the thing is that I think is bad?
Geoff Anders:
Yes! [crosstalk] That would be great. Because I can tell the same story without the nefarious theme. Yeah.
Anna Salamon:
Yeah. Sorry for accidentally having it in the connotation.
Geoff Anders:
Oh, it’s okay.
2. (2:13:22) Narrative addiction and EA leaders being unable to talk to each other
Anna Salamon:
Okay. I’ll take a couple steps back, explain a bit of worldview in which I think it contexts why a thing I think might be bad might have been occurring. And then say the thing, sorry.
So, I guess I’ve been playing around a lot lately, in the last six months or something, with the concept of narrative addiction. Where in my head… I don’t know, I’m gonna talk about stuff I don’t know anything about, nobody should believe me if you think it’s false. If you guys could just figure out which parts are which, that’ll be great.
I tend to think of, for example, anorexia. Which I know nothing about, I’m just going to terribly say bad things about it as a metaphor while violating all rules. I tend to think of anorexia as being something maybe sort of like narrative addiction.
Like, it’s addictive, but it’s not addictive the way heroin is addictive. It’s not physical, but there’s a sense of control or something. And you want to be in control. There’s a narrative, or something, and somehow something will get better in the narrative. In extreme cases, people cling to it even at the expense of life. That’s probably horribly wrong for anorexia, and I apologize to everybody for using it as a metaphor.
But the thing I actually want to talk about is I think, for me, I spent a whole bunch of the last five years or something in what I would call narrative addiction.
Geoff Anders:
Hm!
Anna Salamon:
Yeah, sorry. Accuse me of a thing and then I’ll go accuse you of the thing. And then you can tell me that it’s false about you.
Where according to me, there was a thing that I wanted, which was something like “to make the world a lot better.” And a story that I had in which I was doing a thing like that. And then various things would happen that were kind of uncomfortable, or didn’t feel good, or whatever. And I would sort of retreat to perching on this narrative and to doing things that would locally reinforce the narrative for me.
And I think basically this was pretty costly and not very good for my happiness or my effects in the world, or my ability to have actual conversations where I was listening to the other party.
I think a thing that happens to a lot of people in positions of relative power and influence in a social graph—similar to the one that I was in, and similar also to the one that I believe you were in—is somehow only doing the things that reinforce the narrative, and having much more trouble than other people have hearing perspectives that aren’t their own.
So, like, in terms of this fragmentation of the initial EA community, it seems… Like, I had an interesting conversation with some people several years ago where they were like, “Yeah, I feel like if we stuck...” I can’t remember who they actually named, but something like, “I feel like if you stuck, for example, Anna and Geoff, or Geoff and Eliezer, or Eliezer and Toby Ord, or whatever, in a room, it would actually be like relatively...” I don’t know about Toby. Maybe Toby’s better. ”… It would be relatively difficult for most of these pairs of people to have a real conversation. But if you take random people from the organizations, they do fine. What’s up with this?”
Geoff Anders:
Right.
Anna Salamon:
We were like, “Yeah, that’s kind of interesting.”
spiracularity: I like the way this is going, this feels both interesting and I think it hits something valuable.
larissa24joy: +1
AgileCaveman: oh yeah super valuable,
larissa24joy: something like the narrative addiction concept rhymes with some of my EA experience so this is an interesting and useful concept (thanks Anna)
fiddlemath: Geoff, for later: Some of the narrative addiction effect may come from organizations growing larger and more of their members navigating reality via narratives and social cues
Geoff Anders:
I definitely just affirm—I definitely think there’s a leader’s thing. In different fights and so forth that happened at different times, my observation was, the leaders were more opposed. The people were like, “Can we get along?”
Anna Salamon:
And they could, and the leaders couldn’t; and the leaders also couldn’t talk, which is more the interesting thing. Not just couldn’t get along.
Geoff Anders:
Yeah, yeah.
Anna Salamon:
Anyway, so I believe you… So, look, the negative thing that may or may not have been the case, is that you wanted people who were “small” enough that you could—and I think these people were great and were on great growth trajectories, but nevertheless, to say terrible things—you wanted people who were “small” enough that you could intellectually dominate them and they would follow your thing, rather than being willing to loosen your grip on narrative addiction or something enough to be able to hear alternate perspectives. That would be the-
3. (2:17:14) Early Geoff cooperativeness
Geoff Anders:
Okay. I definitely think that there’s at least… Like, I definitely think I didn’t appreciate really at all the work-life distinction and the value of doing things that weren’t for the mission and so forth. So I think that’s definitely there. But if I could ask a question about the thing, what was the narrative that-
Anna Salamon:
Mine?
Geoff Anders:
No, no, for-
Anna Salamon:
Yours?
Geoff Anders:
Yeah, because from my perspective… I mean, you may recall this and this may be another source of tension. Initially, I wanted to just merge the organizations, right? I tried to be hired as research director at MIRI. I don’t know if you recall this, but-
Anna Salamon:
I remember you tried to get hired at MIRI, at the very beginning, yeah.
Geoff Anders:
Yep. Yep. But that was like, Leverage was super small then. And I’m like, well, we have some differences, but there’s a… There’s the inside-Leverage thing, where there’s like the different projects working in parallel and helping each other. I also thought about that with regard to external organizations. I talked to Will MacAskill in… 2012, I guess. Was it 2012? I think 2012. And I was like, “We should maybe merge Leverage and Giving What We Can.” And he was like, “Yeah, good idea,” or something like that. And then it didn’t happen. And I don’t know if he was… I’m not sure what his attitude was on that.
But, like, even take something like Giving What We Can, which is focused on global poverty, or was focused on global poverty. I always had the perception that it just would be valuable for the other projects to build up a really big Giving What We Can thing. I mean, if you have somebody build up Giving What We Can in a very large, substantial way, that just will in fact pick up more people who are interested in animals and building EA and AI and Leverage’s style of research, and et cetera. So there’s a thing that I’d like you to try to square with-
Anna Salamon:
[crosstalk] Rationality is the common interest of many causes. Yeah, we had this in the beginning, then it became rational-
Geoff Anders:
Right. But, so square that with the idea of needing people to be in my thing.
Whoops, sorry. Hold on, my phone is beeping-
Anna Salamon:
Well, okay. So my story about you and me, and many people, is that we started out not having as much of the narrative addiction and ended up more in the narrative addiction. So I agree that at the beginning you didn’t seem that much like you were doing this, compared to my stories about you later on.
Geoff Anders:
Okay.
Anna Salamon:
I have the same story about myself and several other people.
4. (2:20:00) Possible causes for EAs becoming more narrative-addicted
Geoff Anders:
Do you have any idea what would have caused us to become more narratively addicted? Because I find this to be really an interesting hypothesis. I think there’s some ways that I may not have had as much perspective on things, especially near the end before dissolving Leverage. And so I’m really interested in this hypothesis, but I haven’t been able to… It’s really hard to see how you lose perspective.
Anna Salamon:
Yeah, it is hard to see how you lose perspective. You almost… Yeah, you need to look from inside and from outside at once, or something.
Geoff Anders:
Yeah. But maybe you have leads on… or, what...
Anna Salamon:
Yeah. I mean, I don’t know. One story—this is a very, very generic story, I think it would also be nice to try to track this one with Leverage and CFAR specifically.
Geoff Anders:
Sure. Yeah.
Anna Salamon:
But sort of as a case study in this larger thing, almost. In addition to being a thing in itself.
But, yeah, I have a story that’s something like… Many, many actions run partly on promissory notes. (Borrowing a lot of this, by the way, from Bryce Hidysmith, so if anybody likes these ideas, I think they should give some credit to him.)
Many, many actions run partly on a promissory note. I make myself tea, and this is because I can visualize the tea which is located in the future, and it’s calling to me, and I believe that if I put my cup in the microwave I’m going to successfully get some tea, and it’s going to be great. Right?
But there’s the question of how much of the thing is a promissory note that hasn’t been cashed yet? Like, what the ratio is of “expectation of the future” compared to “realized stuff right now”. And what ratios those have in pulling our actions.
Geoff Anders:
Right.
Anna Salamon:
The EA movement was very, very high in “we’re going to have a whole bunch of glorious stuff in the future, it’s going to be amazing”, compared to how much we had actually done so far in terms of lives saved, or whatever.
Geoff Anders:
Right.
Anna Salamon:
One story I have is that we got together, and initially we all hoped to do the best thing. And the best thing was basically, we were all going to cooperate and it was going to be great. And then various bits of data came in, and the data was not super coherent with the best thing that we were hoping for. And so we retreated farther and farther and farther into smaller and smaller scope of what we were taking in kind of narrative addictions. And less and less willingness and ability to listen and bridge. But I mean, that’s an abstract story and then we can tell a more mundane story.
habrykas: Huh, at the time, it felt like EA was almost only focused on doing things right now
larissa24joy: From my own experience of something that might be like “Narrative addiction,” I feel like I lost some perspective out of fear/responsibility. I felt hugely responsible for EA (more than makes sense on reflection) and I feel like that led me to tunnel vision focus on EA and trying to make that go well.
habrykas: Like, the movement was primarily about global poverty things and high-standards of evidence
5. (2:22:34) Conflict causing group insularity
Geoff Anders:
No, but that’s interesting, because I think the… I’ve been tracking a different aspect that maybe fits with the narrative… I want to think about the narrative addiction part. I definitely think there’s truth to that, so that seems like a good part, and I want to think about it.
The thing that I’ve been tracking is the ways that groups become more insular as a result of conflict. Where, so, really simple examples like: Leverage had a lot of information on its website back in 2011. And then we got a bunch of critique, and we weren’t getting a lot of positive engagement. And so then we eventually, by 2015 I think we’re down to a splash page.
But then there’s something similar. So right now I think one of the potentially really negative, I would imagine unintended, side-effects of the current thing happening right now, is when there’s fighting it sort of scorches the earth between organizations. Where it’s something like… Yeah, go ahead.
Anna Salamon:
I guess… sorry. In my opinion… No, no, I had a thing to say, but maybe I should hear the end of your sentence.
spiracularity: Someone also appears to have petitioned to get the old Leverage website taken down from Internet Archive, too.
spiracularity: …why did you guys feel like you needed to do that, anyway?
habrykas: My memory of the reasoning (when I asked people about this many years ago) was something like: “People keep digging up a bunch of stuff I said and keep taking it out of context. Like, that one time when I said ‘Connection Theory is a complete theory of psychology’, by which I meant ‘Connection Theory is trying to be a bold stick-out-your-neck type theory that aims to be complete, but look, it obviously doesn’t explain everything, but this is the thing it’s aiming for’”
habrykas: “And people kept accusing us of saying wrong things in annoying ways that seemed pretty confused, and it seemed like overall when people looked at the archives, they got more confused than enlightened”
habrykas: But someone should feel free to correct me
larissa24joy: Something like Habryka’s account seems right to me but that’s based on similar conversations with people about their recollection, not from any knowledge at the time
larissa24joy: (so people around at the time should totally fill in gaps instead)
Geoff Anders:
I mean, I think that there are… Basically, I think there’s lots of forces that produce insularity. And I think something like continued unresolved-but-still-real conflict is one of them. I’m not sure that that’s true, but it’s like, if you have like… Let me just add, it’ll just take a second. You’ve got red team and blue team on different sides, right? And each has a negative thing about the other that’s not being expressed, but it’s being held on to. And I’m not saying it’s not—it could both be real. It’s not meant to not be real. But I think if that happens, then it’s really natural for the things to sort of curl in and not… Yeah, basically become more insular. But yeah, go ahead. You were going to say a thing.
6. (2:24:51) Anna on narrative businesses, narrative pyramid schemes, and disagreements
Anna Salamon:
Yeah. I was going to say, I think there’s different kinds of… Sorry. A different word that I want to introduce besides narrative addiction is “narrative pyramid schemes”.
Geoff Anders:
Okay.
Anna Salamon:
Where you get a bunch of people buying into a narrative or something, and this is how you’re able to… Sort of a group-level version of something like narrative addiction. But you get a bunch of people buying into a narrative. Because they bought into the narrative, they’ll do different things depending on what you say. So they’ll work for CFAR, they’ll work for Leverage, they’ll go take a bunch of actions in the world that you want.
Geoff Anders:
Yeah.
Anna Salamon:
But they’ll do it because they’re expecting this narrative to pay off in some fashion. And the thing… yeah, I could just say “narrative business” or something rather than “narrative pyramid scheme”. The thing that makes it a narrative pyramid scheme is if it only works insofar as the narrative is able to get other people to buy into the narrative, is able to get other people to buy into narrative, and in this way-
Geoff Anders:
Sure. Well, yeah. I mean-
Anna Salamon:
- whatever, the movement can grow, even though the movement doesn’t really have an actual business model that produces value in the sort of… Anyway, you could agree or disagree with that, but I want to connect it back to what you were just saying.
Geoff Anders:
Sure.
Anna Salamon:
And then you can reply. It seems to me that there’s lots of different ways that groups A and B could be in conflict. One way we could be in conflict is by, I don’t know… well, maybe there aren’t. Maybe we just have different research agendas and we’re just trying to do our different research agendas. And we just think each other’s research agendas is going to be dumb. And so we’re most [inaudible] to do the research agendas. And in the end it’ll go to the glory of group A, I mean of group B. Okay, that’s a kind of conflict. It’s not a very active kind, because we’re not really disputing what to do with a resource. But anyway.
Geoff Anders:
I do think that can be an actually real thing. Different research projects are based on different assumptions about the world. And I think you can-
Anna Salamon:
You can have pretty important disagreements that way. I’m not sure whether they’re conflicts, exactly.
Geoff Anders:
There’s going to be some reason why they’re not being expressed or being discussed fruitfully, right?
Anna Salamon:
Well, you could have… in particular, I could imagine a set of people who have really interesting intellectual disagreements while disagreeing about which research are going to be fruitful or whatever, and it’s just really fun. I don’t know if that’s a conflict, it’s like conflict...
Geoff Anders:
Oh yeah, no, no. Yeah. I agree, you can have it without the conflicty part. Yeah, basically.
Anna Salamon:
But the interesting thing to me is that if you have two groups that are each pulling on their own people via narrative, and if the narrative is in some important sense false—I don’t mean it’s 100% false, but it’s one of those things that could be destroyed by the truth, in some sense...
Geoff Anders:
Yeah.
Anna Salamon:
Maybe by pointed, carefully selected truths designed to destroy it, or whatever. But it’s not robustly “just go notice this stuff”. It’s, like, a narrative.
Geoff Anders:
Right.
Anna Salamon:
Then conflict between the two groups I think tends to create dead zones of non-conversation. I don’t know, because… sorry, maybe you could just actually flesh this out into a model that I’m not articulating yet.
7. (2:27:51) Geoff on narratives, morale, and the epistemic sweet spot
Geoff Anders:
No, that’s super interesting. I want to add a piece, though, that I think relates to maybe disagreement, but I think is important, and also I think relates to the rationality/Leverage divide.
I think there’s a bunch of dangers associated with narrative. The narrative addiction thing, I think, is real. I want to think about it and understand it better, et cetera, but I’ve noticed some things that are similar—the thing I’ve been thinking of is in terms of “fantasies” basically, and people getting trapped in them. That’s my own language.
But there’s a way in which narratives are really, really valuable. Like, narratives can help people to maintain morale in circumstances that are really bad and really hard. And then I think this is one of the flash points between Leverage and rationality, where I think of it as a very careful navigation. If narratives are misleading and wrong, people become epistemically less good; but I also think that if your morale gets too low, you also become epistemically less good.
And so I think the epistemic sweet spot is—the best thing is “you know how to do things and you can accomplish them, and you can see all the truths and everything works”. But then, as a person’s ability to accomplish things, or the quality of their position goes down, then I think frequently narratives are useful, and sometimes necessary, to maintain morale. And that if you graphed narrative versus epistemics, it’s not “epistemics are the best when narratives are the least”. I think that there’s frequently—like I said, morale. Morale sometimes helps people to look for paths for hope. You can have a person who just doesn’t think about how things could be better. So I think narrative is dangerous, but also useful. And so it has to be used correctly, roughly.
spiracularity: You know what else is often surprisingly adaptive to negative environments, then becomes maladaptive but hard to deconstruct once you’re in other circumstances? A lot of trauma responses!
329zwydswr: what’s morale, other than a belief that a plan will work?
spiracularity: (The tone is not intended to be as bad as it sounds, though. I just think they’re both STICKY.)
WaistcoatDave: @spiracularity exactly. and as everyone is human, they come into these situations with preexisting identities’e
329zwydswr: that seems to conflict with the thing about morale
WaistcoatDave: identities, experiences and often trauma
larissa24joy: Can you elaborate @spiracularity ? (I feel like there might be an important thing here but I don’t quite grokk it yet)
8. (2:30:08) Anna on trying to block out things that would weaken the narrative, and on external criticism of Leverage
Anna Salamon:
Yeah, so I guess the thing I would say is narratives… are great. Like, even something like “I’m going to put the tea in the microwave and then it’ll be warm and I can add a tea bag and then I’ll have tea” sort of a narrative. It’s a carefully selected small portion of a thing that I can use as a storyline. Larger-scale narratives are great too...
Narrative addiction I would like to distinguish from narratives, and to say that it has to do with the specific motion whereby I’m trying to block out things that would otherwise weaken the narrative.
And I am sure there are local benefits that can be obtained by blocking out things that will weaken the narrative. But my current position is that we just shouldn’t do that. Like, it can help locally, but it creates something like technical debt. Let’s not do it.
Geoff Anders:
I roughly agree. I roughly agree. I mean, I think there can be cases, but roughly I agree with that.
Anna Salamon:
Okay, I… don’t believe you that you agree.
Um. Or, sorry, I’m sorry, I believe—I’m sure you tried to tell the truth and so on. But I would like to bring up things that I think you also think, that I think are in tension with what you just said, and see what you say about it. I mean...
Geoff Anders:
Great. Yeah, sure. Yeah, yeah. Great.
Anna Salamon:
So, I think… So, look, I—mm. It’s hard to say all the things in all the orders at once. I’m going to say a different thing and then I’ll [inaudible], sorry.
So, once upon a time I heard from a couple junior staff members at CFAR that you were saying bad things to them about me and CFAR.
Geoff Anders:
[I] believe it.
Anna Salamon:
I forget. They weren’t particularly false things. So that I don’t accidentally [inaudible]-
Geoff Anders:
Okay.
Anna Salamon:
Whatever. Personal view is that it was probably partly silly that I was upset about this. It was not my view at the time; we can talk about it. But anyway, the reason I was upset about it is basically because I did not hold the attitude… I don’t know. Sorry, I could say...
I think it was partly that it didn’t reach me very naturally. I think that was more of a legitimate objection. But, whatever, we can talk about it, I probably am getting the facts of the situation wrong while accidentally maligning you on Twitch TV.
Geoff Anders:
I’m sorry if I… did an incorrect thing there. So I’ll just let you know that-
Anna Salamon:
Well, let me-
Geoff Anders:
Go ahead. Go ahead. Go ahead.
Anna Salamon:
Thanks. I appreciate it. I should skip you for the story. Once upon a time there were various past contexts where I would be upset by people seeming to me to weaken CFAR’s narratives in its staff. I currently think that that was a mistake on my part. And that it violated this heuristic that I currently think we should have, where you don’t use narratives to try to block stuff out.
I think there were also past situations where you, Geoff, were upset at people saying bad things about Leverage in ways that made it harder for your staff members to have, as you would put it, morale. And I’m not sure how that’s consistent with the thing you just said about how you basically agree that one shouldn’t use narratives to try to block things out.
Geoff Anders:
Well, do you have cases in mind? That’d be helpful.
Anna Salamon:
I have only generic cases, but I bet you can fill in the specifics.
Geoff Anders:
Yeah. Try a generic case.
Anna Salamon:
I think that you’re all, like… I don’t know, the straw Geoff in my head is all like, “Man, the rationale of the community was mean to us, because they kept saying bad things about Leverage in ways that made it harder for our staff members to have morale. I wish they would stop it. They kept saying it specifically to staff members and so on, and I don’t like it. They shouldn’t-”
Geoff Anders:
I mean, I do think that that’s true. I do think that Leverage—this is one thing that I think is a really unappreciated aspect of this whole situation. It’s like, a whole bunch of Leverage staff just got tons of negative stuff from people. And it was, like, as soon as you joined.
Anna Salamon:
I know, I know. I would be afraid… just to put it out there, I would be afraid to say positive true things about Leverage because then people would get mad at me.
Geoff Anders:
Right. So I-
Anna Salamon:
And then I would say fewer positive true things about Leverage than I thought, even though… Like, whatever, I’ve thought various true bad things and various true good things. Well, of course I think my things were true. I thought various good and bad things. But like-
Geoff Anders:
But it’s super nice to hear you say that. I mean, wow. Okay. All right. Well, good. I feel like we’re having a good conversation, at least from my side. Hopefully this will turn out good for you also. Okay. I...
AgileCaveman: it was somewhat surprising that some really vocal critics of leverage later applied to join. There was certainly an under-current of envy
habrykas: @AgileCaveman : Huh, do you have an example?
habrykas: None come to mind for me, but I am of course not omniscient
z1zek100: I think there are cases the other way. People wanted to work at Leerage, didn’t for whatever reason and then were very upset.
AgileCaveman: yes, i have a couple people in mind, but i’d rather not name for privacy sake
habrykas: Yeah, seems sensible
AgileCaveman: z1zek100 yes, true as well
spiracularity: To be fair, you’re taking Rationalists who love ’ferretting out weird rare elusive dangerous truths,” and then do stuff like taking down your website. Some of them being envious and then joining doesn’t surprise me. Some of them assuming you had skeletons doesn’t surprise me, but I agree they could be pretty brutal sometimes.
9. (2:34:24) More on early Geoff cooperativeness
Geoff Anders:
Definitely one thing that was a practice, a part of Leverage culture, at least as I understood it—I’m going to say a pro-Leverage thing, but I’m going to say then a negative thing after that. The pro-Leverage thing is: As I understood it and as I experienced it, we had a practice of trying quite hard to understand the value of different projects as part of assessing potential collaboration. And so it was really important to us, something like “how much original research was happening at CFAR”. And the more original research, the more it would be like, “Okay, good,” because that’s something we could conceivably learn from. And also, at least in my… It’s just better to have groups that are… If they’re research groups doing more research, then that’s good.
I think that that’s the positive thing. I mean, there is just a lot of “try to say the positives and negatives”, and then a way that this may have… One of the things that’s been coming to my attention as I’ve been thinking about it, is the way that something like culture is not always transmitted. Like, some of the things about narratives, some of the things that have been said I was really surprised to hear that people were using narratives in that way. But it’s quite possible that people were using the relevant negatives [sic] - something like, trying to paint an overly negative story. But, I mean… Yeah, I don’t want to, again, be overly pro my own thing because there’s obvious biases and so forth. But it’s like, if, I mean… yeah, it’s… anyway.
Anna Salamon:
Maybe best to just display your own thing with obvious biases mixed in, and then anyone can complain about them.
Geoff Anders:
Yeah. I mean, okay. Because invited, and in the name of transparency:
I think we tried hard to cooperate with the other groups. I think we reached out. I think we ran events. I think we did, like, a whole bunch of things. We, like… see, look, now I’m going back into the “We did many things wrong,” which I do think is true, but… I think that it just would be nice to not be excluded or cut out of the histories, or et cetera. So it’s like, you look at tellings of… Look at the EA Global Wikipedia page and see what you think about whether or not we’re all, like, being included and so forth.
And so my big update from 2012 was: I had done a whole bunch of reaching out and trying to cause collaboration, and people just weren’t interested.
And so, I thought about it like: I took a hard lesson. Like, the hard lesson was “the other people don’t want to collaborate.”
And then I had all these theories—like, one of the theories was that it had to do with planning horizons, where it seemed like people were very focused on what would be useful for their organization in the short run. And I know I get critiqued to death for having, like, really long-term plans with all of these parts. One of the things that happens if you have that is you just see ways that other people can fit in.
And so it’s like, huh, if there’s like… I mean, this is part of the EA Summit “all the groups sort of aid each other in various ways” thing. And THINK was meant to be a collaboration. So from my—maybe this is biased, but if score it, I just give us great marks on trying, and I don’t give the other groups great marks on trying.
And so, yeah, people can be mad about that; it does seem… Yeah. Okay. Let me just say one other thing. I think this is important.
I think there’s definitely a way in which I and other people at Leverage have been beaten down by the constant narrative sort-of opposition.
fiddlemath: Anja says: Geoff—I think the people in the Rationality community who say negative things about Leverage, are/were in an arms race with an intention on Leverage’s part to withhold details that could be negatively construed.
spiracularity: @fiddlemath EXACTLY.
fiddlemath: yep
spiracularity: Add an extra layer of “you guys were not optimizing for outside legibility”, so the straight words didn’t really convey things very well.
Geoff Anders:
And part of why I wanted to have a Twitch channel was just to stop doing that. And this is pointing to an area where I think I-
Anna Salamon:
Sorry, stop doing which?
Geoff Anders:
To stop, like… The thing I want to say is something like, “I think we valued collaboration and so forth.” Whereas, the thing that I should somehow be able to say was: We tried to collaborate. We offered help. If other people had wanted to collaborate, it would have happened. That it counterfactually, causally depends on the other people’s not wanting to collaborate. If you, Anna, had come to me and said, “Geoff, let’s figure out how to do a joint psychology research program and have a bunch of workshops,” I would have said “Yes,” or I would have said “Yes, if,” and then worked out some complex thing.
I’m sorry that that’s coming across as, like, directive or determinative, or I don’t know—the way I’m saying it. But I’m, like, pushing through-
Anna Salamon:
I appreciate that you’re putting your narrative out there in a clearer fashion. It seems nice. It seems good for conversation, to me.
Geoff Anders:
Okay. Well, it’s—yeah. I’m just expecting to get critiqued to death on this because-
Anna Salamon:
Well, I mean, I think that is one of the things that happens if you say something clear—is people can respond. And that seems good too.
Geoff Anders:
It is notable, also: I’m usually okay being critiqued to death, or whatever, but, like—Okay. Useful. Useful. There’s my narrative. I want updates. I… anyway. Yeah, you know. Okay, go ahead, say something.
LuliePlays: One thing I’m finding odd about this conversation about narratives is that there were accusations of demon seances—I’m curious what @anna_salamon thinks about that stuff, or if it seems irrelevant to CFAR’s relationship with Leverage
329zwydswr: hm. sounds like geoff is describe a sort of progressive escalation of narrative warfare. like a feedback loop of perceiving a slight narrative tilt → tilt right back, to correct the scales → other people push back → repeat
Turbowandray: +1
habrykas: My current model is we are talking about Leverage from 2013-2017
habrykas: Which did not involve any demon seances
Turbowandray: (to clearer position/etc)
habrykas: By the time demon seances were happening, I think relationships between CFAR and Leverage had mostly come to a stop
spiracularity: Maybe your narrative was protecting you from feeling critiqued to death, but had costs elsewhere? (speculative, though)
DaystarEld: “One thing I’m finding odd about this conversation about narratives is that there were accusations of demon seances—I’m curious what @anna_salamon thinks about that stuff, or if it seems irrelevant to CFAR’s relationship with Leverage” +1
z1zek100: +1
Anna Salamon:
Yeah… Um, sorry, somehow I’m now wanting to read the sentence that has my name in it from the chat.
Geoff Anders:
Yeah, it’s really hard to navigate when people are saying interesting things.
Anna Salamon:
Oh, yeah. They want us to talk about different stuff from what we’re talking about, but I think I should reply to you first. So I’m going to do that, and then we can go there, or not.
Yeah, so… I agree… Like, my perception… I agree that I remember a bunch of concrete things from the early years, like 2012 through 2015 or something − 2014, I don’t know—in which Leverage was doing a bunch of things that I would call trying to collaborate. Especially things in the vicinity of “trying to collaborate on concrete projects that were meant to bring resources to both groups”, or whatever.
Geoff Anders:
Right. Yeah. Yeah.
Anna Salamon:
I recall Leverage as being extraordinarily, by local standards, open to and initiative-y in proposing things of this sort.
Geoff Anders:
Right.
Anna Salamon:
And I believe you, that you would have been receptive to such initiative from other groups [inaudible].
Geoff Anders:
Really?! Wow! Really? Okay.
Anna Salamon:
My brain really thinks I should be adding some sort of “but” afterward, and I keep looking to see what the content of the “but” is, and my brain’s like, “But… I don’t like the rearrangement of egos that happens when I say that sentence!” And I’m like, “Great, brain. Can you give me something with more content?” And my brain’s like, “I think there’s probably something to say. Didn’t Leverage...”
And now I’m going to try it, because there’s sort of “epistemic status: kind-of-like rationalization”.
10. (2:41:45) “Stealing donors”, Leverage’s vibe, keeping weird things at arm’s length, and “writing off” on philosophical grounds
Anna Salamon:
There was that incident at the beginning where everybody was very upset because Leverage was “trying to steal our donors” or something—like, going down some sort of donor list.
Geoff Anders:
Yeah-
Anna Salamon:
What was that? I don’t remember it.
Geoff Anders:
Yeah, yeah. So it’s-
Anna Salamon:
Anja or somebody brought it upthread, is why I remembered it at all.
Geoff Anders:
Yes.
Anna Salamon:
But once she did, I was like, “Yeah, yeah, yeah! That’s how we knew Leverage was bad and out to get us!”
So, I actually don’t think… Sorry. You can reply to that in a second. I’m just going to keep babbling for a second, though.
Geoff Anders:
Yeah.
Anna Salamon:
Actually I think there’s something different, though. I think Leverage came—and by “Leverage” in the beginning, I mean you, Geoff—came in with a different pattern of something that I think a lot of people had an immune response to.
Geoff Anders:
Yeah. I agree.
Anna Salamon:
And a different pattern of something—it was partly the particular way that you weren’t into a particular kind of materialism. The “pole through the head” thing—I can say this more slowly for people to follow it. It was partly that you… I don’t know, you told me a story when we first...
Sorry. I guess I’m inclined to just say all the things, but I-
WaistcoatDave: I think there’s a toxic narrative that critiquing is entirely fact based and therefore no time or focus should be put on HOW the points are raised. It’s clear the consequences of that has put Leverage staff on edge about entering conversations
habrykas: Content I remember: “Leverage reached out to a bunch of people on the top MIRI donor list, and some people were upset about that.”
habrykas: I think the people who were upset about that were kind of wrong. But I thought they were right for a while
sowgone: Cartesians vs Dennettheads
WaistcoatDave: and ultimately open conversations in the face of that fear and where that’s supported by the community, is how things are moved forward healthily.
Geoff Anders:
I have interjections, but you know, it seems good. The thing I, you know-
Anna Salamon:
You told me a story when we first met, I can’t even remember how it went, but it led me to think that you sometimes were, like, sort of more towards manic than most people. It was something about high school. You can fill in the details if you want, or ask me to, or we can skip it.
Geoff Anders:
Definitely energetic.
Anna Salamon:
It wasn’t—It wasn’t particularly [inaudible].
Geoff Anders:
People have accused me of being hypomanic at times.
Anna Salamon:
Yeahh… I was like, “Okay, this guy, I feel like he’s...” Anyway, whatever.
That, plus the materialism… I think you came in with a thing that a bunch of us—not so much me, although I went along with it-
Geoff Anders:
Can I?
Anna Salamon:
But a bunch of us wanted- Mm-hm?
Geoff Anders:
Well, so, interestingly—I mean, we’re getting to a bunch of things. So, my experience was that literally within thirty seconds of our meeting, you had written me off on philosophical grounds because I… I visited SingInst, I got off the plane, I came out, I got picked up by you, and we started talking.
And maybe it wasn’t thirty seconds. Maybe it was sixty seconds. But literally the first thing that happened was that I was subjected to, in my perspective, a philosophical litmus test. I failed. And you were like, “Okay, not that one.”
And so that… You know. What do you think about that?
Anna Salamon:
What do I think about that? I feel-
Geoff Anders:
Do you remember it?
Anna Salamon:
I don’t remember it, but I believe you.
Geoff Anders:
Okay.
Anna Salamon:
Um. Yeah. If you want to fill in more details, it might help me remember it. But probably [inaudible] gotten you from the airport.
Geoff Anders:
No, no. It’s- No, I don’t remember.
Anna Salamon:
Are you sure it was me, for that matter?
Geoff Anders:
Yeah, absolutely. Yes.
Anna Salamon:
Great. So, I don’t feel skepticism about it. So I do have feelings about it, despite not remembering it.
Geoff Anders:
Yeah.
Anna Salamon:
I remember later on, in what I assume was that same visit, in my head, we were sitting.
But what year was this? In my head, we were sitting in the house that I was living in in 2011, 2012 in Berkeley.
Geoff Anders:
I do remember talking in that house a couple times. This was 2011. Yeah, this was in March 2011.
Anna Salamon:
Yeah, but Rob Bensinger was not who got you from the airport, even though he’s saying that he remembers something similar.
Geoff Anders:
No, I don’t remember Rob Bensinger being at SingInst at the time.
Anna Salamon:
Rob wasn’t even around yet.
Geoff Anders:
No, this was this right before Luke Muehlhauser came in as the ED of SingInst, which turned into MIRI.
habrykas: I think I docked Geoff a lot of points on philosophical grounds
DaystarEld: “By the time demon seances were happening, I think relationships between CFAR and Leverage had mostly come to a stop”—Ah, this makes sense, but is the point of the conversation then to try to clear the air about Leverage, or to try to help people understand why it diverged from the rest of the community? (I only joined the call in the last 30 minutes)
habrykas: I still engaged with Leverage a lot though, so definitely not “writing off”
spiracularity: On demons: Look, I think Leverage’s internal phrasing around it is toxic BS that is kinda bad for some people. I personally have never had anyone flip-their-shit when I explain social contagion, so feel free to reach out for phrasing-for-normals help if you want.
habrykas: But I think I also stand by that docking of points
sowgone: I wasn’t involved then, I’m just saying I am a MIRI person who wrote you off later
habrykas: Oh, that’s my docking of points
habrykas: Yeah, that’s what I said too
spiracularity: (I also think it’s bad and panic-inducing in ways that aren’t true; analogies with Roko warranted.)
Anna Salamon:
Yeah. Sorry, I’m just going to randomly respond to parts of the chat without having read [inaudible]-
Geoff Anders:
Fine, fine, all right, well.
Anna Salamon:
Ollie Habryka is all like, “But I think I stand by that docking of points.” I haven’t even read his context, so I’m just responding. Personally, I stand by docking of points or something? Maybe?
What I don’t stand by is a “having written off”. And the reason I don’t stand by the “having written off” is to my mind, it resembles this thing I’m calling narrative addiction.
Geoff Anders:
Yeah.
Anna Salamon:
So, like, to respond to Geoff’s question from a minute ago: Geoff, you were all like, “How do you feel about it now?” How I feel about it now is I try to turn my mental eyes toward this memory, which is somehow located in my chest—maybe this is too much phenomenological detail-
Geoff Anders:
Yep-
Anna Salamon:
And I feel a cringe. I feel a mental cringe, or something, as I look. And I think the reason for it is that to my mind, there was an element of something like cowardice mixed into my response.
Geoff Anders:
Really?
Anna Salamon:
Where it wasn’t just my own authentic response to you seeming to me to be philosophically wrong about a thing. Sorry, and I don’t remember the event that you’re talking about-
Geoff Anders:
No, no, that’s fine. That’s fine.
Anna Salamon:
So I’m thinking of other events that are sort of in the same reference class. But to my mind, it was partly that I didn’t want the group to reject me. And that this was sort of part-
Geoff Anders:
You mean, your group?
Anna Salamon:
My group. I didn’t want my group to reject me.
Geoff Anders:
Okay.
Anna Salamon:
And so I was participating in having more of a “that’s weird and I need to keep it at arm’s length” reaction than I would have had in my own person. As opposed to a “That’s interesting! What’s with a smart person like you seeming to believe something dumb! Let’s have a conversation!”
Geoff Anders:
Right.
Anna Salamon:
Which is an entirely different reaction that doesn’t have the dissociating arm’s-length this-guy’s-a-weirdo component.
Geoff Anders:
Okay. That’s-
Anna Salamon:
Like, when you say “written off”, it’s more like that, I think [inaudible]-
Geoff Anders:
Yeah. No, that’s super interesting because, I mean… Some of the people in the chats are talking about docking points or writing people off on philosophical grounds. I think there’s something like it. I think it can make sense to dock people points. Maybe in some cases writing off? It’s, like, a bit extreme, but I could… But there’s something like… There’s, like, a broader… I’m sort of responding to people in the chat also, but it’s like… There’s just this large corpus of philosophical works and texts from the history… You know, people are going to be writing me off for dualism. Note, I’m not even a dualist. But it’s something like, I think it’s usually not correct to see something like a point-sized view or something like that, and then write something off.
I also think there can be trends, but it’s something like… it would be good to crystallize what exactly is wrong. Like, if you could make a strong argument from… It’s like when people say, “Oh, well, so-and-so believes in God, therefore they’re bad intellectually or something.” And you’re like, “What about Newton?” And they’re like, I don’t know, something.
It’s like, if you can draw a strong link between “believes X” and “has various other features” then, like, sure. But I think that empirically that’s frequently not the case.
And even if there are general trends, you can recognize when there’s something different happening with a person and look into it more. So. Yeah.
But I appreciate the thing you said. I mean, I, yeah. It’s… Why did the group need to write off, like whatever, I mean, we got...
Anna Salamon:
Yeah-
Geoff Anders:
Robby Beninger said “Cartesian”, so, like, Cartesianism is really bad. And like, I know dualism was a really big deal. Like… why?
I mean, it is cool sort of in a way to have communities differ on the basis of really basic philosophical propositions, because, like, at least aesthetically you’re like, “Oh, you know, there’s deep philosophical commitments.” But I feel like we should be able to go meta and notice them and be like, “Okay, deep, different philosophical commitments-”
Anna Salamon:
Well-
Geoff Anders:
Yeah, go ahead.
11. (2:49:42) The value of looking at historical details, and narrative addiction collapse
fiddlemath: Anja notes a significant incident at the EA summit, 2014, where Geoff and Anna got into a … pretty heated? … argument about dualism. Seemed very emotional to both parties, given the degree of abstraction of the object-level conversation.
habrykas: Yeah, that was a big deal
habrykas: It caused me to have nightmares, and was a big component of me distancing myself from Leverage
habrykas: (Nightmares because I was interning at Leverage at the time, and it made me feel very alienated from my environment)
habrykas: (And felt like some kind of common ground was pulled out from under me)
DaystarEld: “(Nightmares because I was interning at Leverage at the time, and it made me feel very alienated from my environment)” - /hug
LuliePlays: “On demons: Look, I think Leverage’s internal phrasing around it is toxic BS that is kinda bad for some people. I personally have never had anyone flip-their-shit when I explain social contagion, so feel free to reach out for phrasing-for-normals help if you want.”—yeah I love Leverage’s psych tech from what I’ve experienced of it (workshops), it just seems like it would be odd if those weird-sounding things were not explicitly addressed + I’m curious how Anna views the surrounding context of that post
z1zek100: A bunch of people were obsessed with that conversation for whatever reason. Will McAskill repeatedly cited it as his most definitive critique of Leverage for example.
z1zek100: I was there and I really never followed the strength of the response.
habrykas: I think that’s kind of fair. I do think the conversation exposed some really big philosophical holes.
z1zek100: (This is Kerry btw)
larissa24joy: I’d find it really helpful to understand the emotional charge that seems to be associated with that (dualism) conversation for people?
habrykas: I think naturalistic reductionism is a really major foundation for my epistemology, and I do think it implicitly supports a lot of other common assumptions, and whenever I did into a bunch of my beliefs and plans, I do think a lot of them are pretty based on naturalistic reductionism
LuliePlays: and I’m finding myself wondering why they’re talking about the past stuff (which I find interesting) when there’s questions about the recent stuff and what CFAR’s relationship to Leverage is now
fiddlemath: Anja says she’d like to try to explain...
habrykas: And I think it’s pretty sensible. I do indeed think that an epistemology without naturalistic reductionism feels a lot less grounded, and I feel a lot more scared of having a group of people who “take ideas seriously” without being grounded in that way
larissa24joy: I’d love to hear from Anja on that if she didn’t mind and we had time?
sowgone: I think Cartesianism is kind of a hard-to-escape trap with lots of implications, like a dangerous meme that affects a lot of other stuff. I don’t think it’s morally bad to be Cartesian (you got trapped!), but I think there are so few Cartesians + it’s so bad, that I tend to write people off. Like a smart very devout Christian
AgileCaveman: imo, people tending to write each other off is a common pattern. I feel like i have been written off because of my theories of social contagion. I also wrote people off because of a rejection of biological substrate
habrykas: Agree that people trying and tending to write each other off is a common and kind of bad pattern
fiddlemath: The way I would have known not to reveal myself socially as a cartesian/dualist has to do with the tone and volume of Eliezer’s writing on the topic
sowgone: (BTW as a *separate* thing, I did the ‘Leverage is low-status so I want to avoid them’, which I don’t endorse and is bad.)
fiddlemath: Plus I remember something in the sequences about epistemic spotchecks being good in particular on philosophical topics
zwydswr: “I think naturalistic reductionism is a really major foundation for my epistemology, and I do think it implicitly supports a lot of other common assumptions, and whenever I did into a bunch of my beliefs and plans, I do think a lot of them are pretty based on naturalistic reductionism”—how is it a foundation for your epistemology?
z1zek100: @habrykas: That makes sense to me. I think I had a reaction that was something like “that seems wrong to me, but I don’t understand why Geoff thinks it and there’s probably an interesting story there at least” which involved not being very threatened about it in some way. I guess I generally expected people to have that reaction, but it could be that I don’t have the same emotional valence on this topic or something.
Anna Salamon:
Yeah. I would also… Sorry, Larissa and the chat, yeah, I’m also pretty interested if anyone on the chat wants to explain about the emotional charge that it had for you personally—or, sorry, not for you personally but for the person in the chat who might explain.
Yeah, people are wondering about the current stuff. Lulie, I think I’m talking about the past stuff partly because Geoff mentioned in pre-call wanting to focus on the past stuff and partly because it’s genuinely easier.
Geoff Anders:
Oh, we also, the thing is...
Anna Salamon:
Right?
Geoff Anders:
We also—we’re at, like—we’re at the 1:30 mark. I mean, so we did say we’d go sort of this long. I’m willing to go a little bit longer, but maybe we should shift gears and be like, “What’s the way forward?” Because—is there a way forward, you know...
Anna Salamon:
I… I’m… yeah. I think we’re all hurtling forward in time. So there’s clearly some ways forward-
Geoff Anders:
Love it.
Anna Salamon:
I don’t know toward where. I mean, personally, I’m really into history and conversation and sharing all the details. And I think once we do that, we’ll find a real way forward to somewhere good. And if we don’t do that, I don’t trust our ability to reason or get anywhere.
I also think sharing all the details is the opposite of narrative addictions. Or, like, it’ll help dismantle them. And I think it’s in fact compatible with morale or with the good parts of morale, in the long run. I may be wrong.
I could be wrong, but I don’t think I’m wrong.
Geoff Anders:
I think I agree with like 80% or 90% of that or something. It’s like, I think there are some things… It’s not like everything needs to be hashed out. I think there are problems other than ones addressed by the history. But I do think history is really undervalued, to put it that way. And I’ve gotten value from looking into some of the things. Yeah, I...
Anna Salamon:
I mean, I think we should-
Geoff Anders:
Go ahead.
Anna Salamon:
Yeah. I guess I’m of the hopeful-to-my-mind opinion that many of the things I’ve been calling “narrative addictions” and “narrative pyramid schemes” and so on are sort of on the point of collapse, or have already begun to collapse. And that maybe we’re heading into a place where we can have real conversations across people, again.
Geoff Anders:
I’d love that. I think that’d be great.
Anna Salamon:
Time based more in a grounded thing and less in a “I’ll support your crazy scheme if you support mine” kind of deal that I don’t think it’s how we build, in the real world for the actual situation.
12. (2:52:23) Geoff wants out of the rationality community; PR and associations; and disruptive narratives
Geoff Anders:
Yeah… Something like that seems generally good. But I do want to say something like, there’s a way that I feel like I’m… You know, I sort of want to say, like, “How do I get out of the rationality community?”
Like, I’ve never considered myself a member. I don’t read LessWrong. People are like, “Did you see the whatever?” and the answer is like, “No, no I didn’t.” And I’m running an organization and we’re going to do all sorts of other stuff. And so it’s not that I don’t see value. There is potential value. But I… it’s like the whole thing is sort of a nightmare in a certain way.
Like, how many people inside the rationality community—like, sum up, do an integral or something, on their attitudes towards me; and it’s like, that doesn’t give them a right to actually interact… Like, the fact that someone’s mad at me doesn’t by itself cause it to make sense for me to engage.
I do think some of the things are my fault. Absolutely. I—theories about various things… Like, sounds like I said some bad stuff about you in the past in a way that you didn’t find toward. Like, I...
Anna Salamon:
Well, but sorry. I was mentioning that partly as an example of ways that I used to be doing a thing that I think was not that great.
Geoff Anders:
Well-
Anna Salamon:
I had mixed feelings about it.
Geoff Anders:
No, but I guess what I mean to say is, like, you have a claim or a stake. And then maybe the rationality community has some stake.
Like, definitely one thing I’ve read a little bit about was, seemed like people were doing a memory consolidation sort of event. Like, I think that’s great. I would love the rationality community to do more of X, whatever that is.
And then I feel like it makes total sense for you to want to go into all the details, because you’re much more closely tied to the rationality community. You’re part of it, a leader, et cetera. I don’t know if you would go for leader, but “leader”, definitely.
How do I set up my relationship properly so that, like… I don’t want people thinking that I’m going to, like… Like, there was one comment somebody directed me to—actually, I think, was it Spiracular? I think Spiracular’s here in the chat.
Anyway, there was some comment that was like, the person’s like, “I’m going to make a stand. There’s all this fear, and somebody needs to stand up. And so it might as well be me.” And I’m like, “Well, that’s good.” You know, I like that. But it’s really different than my reality.
And so I don’t really know how to navigate this sort of circumstance. Do you have any thoughts?
spiracularity: If you critique the rationalists, you are one of them. :P
spiracularity: It’s how we roll.
sowgone: BTW I’d love to have a LW thread/post where we can argue about Cartesian (esp. ‘trust first-person stuff a lot’) stuff
AgileCaveman: you used rationality to critique rationality, it’s inside of you now. A classic blunder
sowgone: (‘write off’ for me doesn’t mean ‘war’ / ‘don’t talk’, it means they’re a friend I don’t expect to do amazing things)
benitopace: “you used rationality to critique rationality, it’s inside of you now. A classic blunder”—hahahaha
sowgone: :o
z1zek100: “you used rationality to critique rationality, it’s inside of you now. A classic blunder”—NotLikeThis
Anna Salamon:
Sorry. So I think you just said, “How do I get the rationalist to be bored by me, or to otherwise leave me alone?”
Geoff Anders:
It’s not “be bored by me”. It’s like—I think Spiracularity just said, “If you critique the rationalists, you’re one of them”! Yeah. That’s a viewpoint. Not one I endorse...
Anna Salamon:
Yeah, to my mind you seem very obviously part of this broader community. Partly because, yes, you critique. Partly because you recruit from. Partly because-
Geoff Anders:
Well, “critique or recruit from”—like, but when? Like, what years? Like, not Leverage more recently, right?
Anna Salamon:
Yeahh.
Geoff Anders:
And like- Yeah.
Anna Salamon:
I don’t have a good solution for you here.
DaystarEld: To me “Why is dualism bad?” can be answered pretty easily with something like “It can reliably lead to people doing things like demonic seances.” I’m not trying to be fighty by saying that, it just seems like a pretty straightforward relationship in my experience/observations of distinguishing philosophies that are more or less likely to go down dangerous paths
DaystarEld: (I get that “demons” can just be a language handle on psychology stuff, but what was described is much more than just a semantic handle)
z1zek100: “To me ‘Why is dualism bad?’ can be answered pretty easily with something like ‘It can reliably lead to people doing things like demonic seances.’ I’m not trying to be fighty by saying that, it just seems like a pretty straightforward relationship in my experience/observations of distinguishing philosophies that are more or less likely to go down dangerous paths”—That kind of implies that once you believe in dualism all bets are off or something and I don’t quite see why that ought to be the case.
habrykas: “I think naturalistic reductionism is a really major foundation for my epistemology, and I do think it implicitly supports a lot of other common assumptions, and whenever I did into a bunch of my beliefs and plans, I do think a lot of them are pretty based on naturalistic reductionism”—My guess is this would take a bit longer to explain. Short answer is something like “It’s what allows me to have a shared foundation for all the different methods of knowledge gathering that I use, from emotions to scientific experiments to expert deference, and is something like the foundation of how I sanity-check different parts of my map against each other”
329zwydswr: how was it different than a semantic handle and how is that related to dualism?
AgileCaveman: i am SUPER interested in learning more about demonic seanses. I am not at all turned of by the label “demons.” I am interested in “did they work?”
habrykas: Also, like, a bunch of your staff was like working at CEA until 2017
larissa24joy: 2 of the 4 staff (me and Kerry) were at CEA until very early 2019.
spiracularity: I was making a pun off of the fact that we use the term for both the philosophy stance, and the community, though. (But also the community kinda loves its best critics.)
habrykas: So at the very least you shouldn’t be surprised to be understood as part of the community if you are literally sharing staff with one of the central community organizations
habrykas: Sorry, meant 2019
spiracularity: Like, the fact that these are different didn’t escape me.
spiracularity: (end explaining the joke)
larissa24joy: So if you’re ever involved in EA or Rationality you can never leave?
Geoff Anders:
Okay. Well, I beseech the powers that be inside of rationality—Anna Salamon, Oliver Habryka, others—Robby Bensinger, et cetera, Ben Pace: I’d like some sort of reasonable exit strategy. I think you guys don’t, like… I just get crazy messages where it’s like, people don’t want me anywhere around and then they don’t want me to leave. That’s not cool. That’s, you know-
Anna Salamon:
The same people, or different people?
Geoff Anders:
I, you know, I’m not even in it enough to know. Okay. And so-
Anna Salamon:
Well, sorry, did you get a message from some person Bob who’s both saying “stay around” and “get out”, or do you get a message from Bob saying “stay around” and Carol saying “get out” or whatever?
Geoff Anders:
I think that… I mean, I’m loathe to point to particular individuals...
Anna Salamon
You don’t have to name them.
Geoff Anders:
… But yeah, I think I’m getting mixed messages from at least one person! And it’s something like… There should just be—I mean, this gets to like a thing that I think is a bigger issue with the rationality community. Like, there’s all this stuff about Leverage managing its PR—like, PR matters in the actual world. Like, we want to do projects with people. They don’t want to—you know, it diminishes the likelihood of their wanting to be associated with you if-
Anna Salamon:
Yeah, I… have an important disagreement with you here that would take-
Geoff Anders:
No, I agree it would take a while, but something thing like: How do we… I think there’s some way that people want, like… I want something a bit more concrete. I want somebody to figure out what I’m supposed to do, and I want to have something I can interact with.
sowgone: I’d love to see a Geoff-reply to Anna’s https://www.lesswrong.com/posts/SWxnP5LZeJzuT3ccd/pr-is-corrosive-reputation-is-not
benitopace: “I’d love to see a Geoff-reply to Anna’s https://www.lesswrong.com/posts/SWxnP5LZeJzuT3ccd/pr-is-corrosive-reputation-is-not″ - +1
habrykas: As long as you have power over people, they will want you to be held accountable somehow, and want you to play by their norms. And you’ve had large amounts of power over people for many years.
z1zek100: Over Rationalists specifically?
habrykas: And EAs
z1zek100: That could be true, but it would be a bit surprising
spiracularity: “As long as you have power over people, they will want you to be held accountable somehow, and want you to play by their norms. And you’ve had large amounts of power over people for many years.” - +1
habrykas: I mean, Leverage shared staff with CEA for like 4 years, and was intimately involved with almost all EA Global conferences.
habrykas: So why would it be surprising?
Geoff Anders:
Look, if it’s a mob, if there’s no way to stop it, right, like, people, I would appreciate a little bit of clarity on that. We could have you and Habryka and other people just get together and issue an official formal statement that says the rationality community is going to continue doing this until they happen to personally feel satisfied. And then I can let my people I work with know that I’ve got this rival/enemy that is planning to constantly interact with me and there’s really not a way out. And then I can at least be transparent to them about that.
Anna Salamon:
Ah. Yeah, sorry, I feel like there’s so many threads and-
Geoff Anders:
Yeah, no, I mean it’s—and I don’t mean to say that I wouldn’t put in time to addressing problems or concerns and so forth. But there’s some people for whom the rationality community, like, is the world, and, like, that’s the thing they care about. And I think that’s fine. I think that’s, like—people have communities. Communities seem fine.
Anna Salamon:
Yeah, sorry, you’re adding more and more to your thing. [crosstalk]
Geoff Anders:
But—Okay, cool. So it’s too hard, alright, alright. Well, it seems like there’s—so, we didn’t manage-
Anna Salamon:
Oh, I’m going to try to reply anyway, I was just [crosstalk]-
Geoff Anders:
Go ahead. Sure, alright.
Anna Salamon:
-how it’s too many threads to reply to you and you were like, “Here’s some more threads!” And I’m like, “Okay, great.”
Geoff Anders:
Alright.
Anna Salamon:
”Here my new preface again.” “Here’s some more threads!” And I’m like “Gaahhh!”
Geoff Anders:
I’m happy to let you say a thing.
Anna Salamon:
I don’t know. It seems to me like there’s a bunch of different things going on. So, personally I feel like Zoe writing that public essay is sort of an invitation for me to take an interest in what happened to that part of Leverage, and an invitation to a bunch of other people.
Geoff Anders:
Yeah.
Anna Salamon:
An invitation from Zoe. Like, otherwise I’d be like, “Oh, well, that’s not exactly my business.” Like, I don’t know whether I would or wouldn’t be that way; but I feel like somebody who was in Leverage if 2018, 2019, sort of invited me and a bunch of other onlookers in, in my opinion.
Geoff Anders:
I agree with that. I accept that.
Anna Salamon:
So that’s part of it. Then there’s a different part that is something like: Everybody has the right to free speech and free thought. Particularly LessWrongers. And I’m not going to be trying to rein them in, and probably neither is anybody. And if they do, I’m probably going to oppose them. And that might get in the way of your PR, as you put it. And I actually feel bad about that, but I don’t think it’s my problem. Or, like, I don’t plan to take any interest in it. (Like, I’m interested. I get to watch!)
Geoff Anders:
No, I think that’s fine. The-
Anna Salamon:
And then there’s a different thing, which is… Okay, here’s my most controversial belief on this topic. Mostly not vis-à-vis you.
So let’s talk about Eliezer for a second. Eliezer has huge communities of people who hang out complaining about him all the time. Not really, that exaggerates slightly—but, like, Sneer Club and so on.
Geoff Anders:
I was thinking about this. Yeah. How does Eliezer get out if he wants to?
Anna Salamon:
He can’t! Sorry, he could, but he won’t be able to. Here’s my story about why it’s happening to Eliezer.
Geoff Anders:
Okay.
Anna Salamon:
As I said, this is my own—I don’t know, I haven’t put this take out there before. I don’t know if it’s right.
Like, it seems to me that Eliezer… Let’s talk about narratives. Lots and lots of people have personal narratives that are something like, “I’m going to be a school teacher and then I’ll help a bunch of kids and that’s good. And that makes me a good person and means I did something useful with my life.” Or whatever. People have all kinds of, like, little local narratives.
Eliezer put these sequences out there, and they’re just, like, a massive vortex that disrupts a huge number of people’s local narratives.
Geoff Anders:
Right.
Anna Salamon:
And then some of those people hang out and get really upset.
Geoff Anders:
Yes.
Anna Salamon:
And this is the sense in which… And there’s a co-dynamic here… I don’t know. Look, I’m probably going to go to various kinds of Said Something Wrong Out Loud On The Internet In Front Of Fifty People Hell for this remark. But, like, it seems to me that Eliezer’s narratives… I, sorry, I love and admire Eliezer more than almost—like, he’s one of my favorite people in the whole world. But also, I imagine when I read the sequences that they’ve got kind of an edge, and that it’s not just a presentation of facts, that there’s sort of a reinforcement of his own narrative in there; I could be wrong about this.
Geoff Anders:
Yeah.
Anna Salamon:
And I think the reinforcement of his own narrative is sort of part of why it erodes other people’s narratives so much.
Geoff Anders:
Yeah.
Anna Salamon:
And it’s part of why a bunch of them then try to get vengeance.
Geoff Anders:
Yeah. Yeah, yeah. Totally agree.
Anna Salamon:
I think there’s something a little like that about you and the rationality community that’s happening on a somewhat smaller scale. And this is part of what’s up with it. Although I think the thing about “Did Leverage actually harm some people, and were we invited into that room to try to help sort it out and make sure that there’s, like-”
Geoff Anders:
Yeah, yeah. Right.
Anna Salamon:
”-restoration of things that those people might want or need,” is a different and pretty legitimate point.
So those are the three things that I have.
Geoff Anders:
Okay. Yeah. Well, so, I agree with the harms and being brought in-
Anna Salamon:
Oh wait, sorry.
Geoff Anders:
Oh, go ahead.
Anna Salamon:
There’s a fourth thing I really want to say. I’m sorry. Is that okay?
Geoff Anders:
Yeah.
Anna Salamon:
The fourth thing is: Rationality was never trying to be an organization in the way that Leverage is trying to be an organization. At least, I don’t think so. Like, I think if I had been like, “Hey, Geoff, your Leverage is messing up my life. How do I get out?” You would have tried to be the responsible person at the front who could have had an answer for me.
Geoff Anders:
Right.
Anna Salamon:
Rationality is really not trying to be the kind of thing that can have a responsible person at the front who has an answer for people.
DaystarEld: “how was it different than a semantic handle and how is that related to dualism?”—From my understanding of Zoe’s post… if people treat semantic handles for psychological phenomenon as real enough to be affected by using crystals, it starts to leave the realm of semantic handles and enter the realm of irrationality
sowgone: motte: ‘Geoff, stop doing abusive actions X, Y, Z.’ bailey: ‘Geoff, stop being weird. And given that you’ve ever been weird, stop trying to improve the world.’ ?
329zwydswr: “It’s what allows me to have a shared foundation for all the different methods of knowledge gathering that I use”—ah ok this makes sense. hard to discuss here, but i want to note that this i think is actually compatible with some kinds of dualism. eg saying “beliefs aren’t material things” (or maybe easier to agree on would be, “algorithms aren’t material things”) is compatible with “everything that happens, happens in accord with physical law”
habrykas: “ah ok this makes sense. hard to discuss here, but i want to note that this i think is actually compatible with some kinds of dualism. eg saying ‘beliefs aren’t material things’ (or maybe easier to agree on would be, ‘algorithms aren’t material things’) is compatible with ‘everything that happens, happens in accord with physical law’”—Yeah, agree, though with many many caveats. I have a tiny steelman in my mind that’s trying to map between Tegmark 4, Platonic Realism and Cartesianism. Though I’ve never gotten it to actually fully typecheck and make work.
329zwydswr: “as real enough to be affected by using crystals.”—well… this is clearly a mistake, but does this really argue against they (the psychological phenomena) being really real?
329zwydswr: like, is talking about “having beliefs” a form of dualism? seems fairly dualistic, in that beliefs aren’t material things. doesn’t have to be irrational / nonsensical / crazy
DaystarEld: “well… this is clearly a mistake, but does this really argue against they (the psychological phenomena) being really real?”—I think there’s a certain type of thinking that *leads* to those mistakes, and we should know that by now
329zwydswr: “Though I’ve never gotten it to actually fully typecheck and make work.”—this problematic seems to be something in the air, there’s maybe an idea here who’s time is coming.
AgileCaveman: One thing that’s somewhat frustrating about the question of “did leverage harm people” is the misunderstanding of “harm base rates”. I have a couple friends severly harmed by rationality or working at normie tech companies. there is a high baseline of people being harmed in the world and the question of whether Leverage is above this baseline is harder than “did it harm people”
habrykas: Agree with this. I think establishing baselines is super important.
329zwydswr: “a certain type of thinking agreed”—i’m curious wat it is, and it doesn’t seem like dualism fits, but maybe i’m missing the point.
DaystarEld: “One thing that’s somewhat frustrating about the question of “did leverage harm people” is the misunderstanding of ‘harm base rates’. I have a couple friends severly harmed by rationality or working at normie tech companies. there is a high baseline of people being harmed in the world and the question of whether Leverage is above this baseline is harder than ’did it harm peopl’e”—Agree with baselines too. But one of the reasons (for example, sorry to use the C word) cults are bad is that they can do targeted harm in excess to baseline organizations to a few people beyond what a randomly selected control group might
DaystarEld: Even as most people have positive or neutral experiences
Geoff Anders:
Okay. I’ve got some thoughts. We still have viewers, so we’re still going.
So, first I should say, I do think that there is a legitimate public interest in the things that Zoe said. I think that those be pursued. I think the people that want to pursue them are doing the right thing, or doing an acceptable thing. And so I’m not trying to push back on that at all.
On the “Eliezer created the narrative and is still reinforcing it, and then the people are stuck inside it and it erodes their narratives thing,” if there’s a thing that’s similar that I’m doing, I want to find it and stop. I hereby offer that I will spend time doing that. And I would love to have one or more interlocutors who could help to try to crystallize what it is, so as to make this process more effective. I don’t know if Eliezer believes that something like that’s happening. I find it plausible that something like this is happening, even though the interactions with me are obviously less thoroughgoing than with Eliezer from the rationality community or from rationalists.
So, but I hereby offer to try to resolve that. And you know, I’m not sure that it’s there, but if it’s… I’ll look, I’ll do this myself. I don’t need other—and I would like help from relevant other people who maybe have a bit more insight into how people are thinking about it, because one of the things that’s harder for me is I don’t in fact track the hive-mind of LessWrong, if it is a hive mind, or the non-hive-mind of many, many individuals who… something, like, I don’t know.
But, yeah, so that sounds good. And then—you said a fourth thing that seemed super important. What was the fourth thing?
Anna Salamon:
Not trying to be that kind of thing, that Leverage I think was trying to be, where there’s somebody at the front who can take calls.
Geoff Anders:
Ah, so here’s-
Anna Salamon:
CFAR maybe is, LessWrong maybe is, but, like, the rationality community, no.
Geoff Anders:
Well, but this is where I think we have a disagreement, but one that’s not like a deep philosophical disagreement about Cartesianism or something like that. It’s a disagreement about social technology.
I think that the correct form for communities involves community leaders and involves walls of particular types. I think that that’s actually from the nature of a community. I think communities—I’m not sure about this, but I think communities come together over shared strengths and shared weaknesses. The community offers an opportunity for the people to, like, collect and grow the shared strengths, and to help protect the people from the shared weaknesses.
And then I think that that’s—like, the reason you need walls is because there is the shared weakness and the walls help to protect the community. And then, in terms of leaders, I think that communities, because there are certain types of shared weaknesses and blind spots, they’re really subject to predators.
And so I think something like “Communities need to have leaders,” and the challenge of the leader is to do their best to try to handle the predators.
But this is a claim about the nature of communities. And as a plug for my Twitch stream, I’m going to be talking about coordination research, which includes looking at different forms of social technologies, including communities. So I’m going to be exploring this.
But I think that there’s actually, like, a positive claim here, not a normative claim, not a philosophical-whatever claim, there’s a claim about a particular social form and its correct thing. And so maybe there’s a fruitful line of discussion to have there.
Anna Salamon:
Yeah. I’m pretty into this conversation.
Geoff Anders:
Great. All right. Well, I feel like we had a really natural breaking point. All right. It took a little bit longer, and I apologize to everybody for the technical difficulties. Those will get better.
Okay. I think this was fine, right? All right. Good. Okay! Awesome. Thanks, everybody.
I will see… Maybe people will be less interested in my philosophical whatever. I don’t know how we’re gonna do better. This was good. Anyway, whatever. [crosstalk] I’ll send out a tweet. All right. I’ll-
Anna Salamon:
I appreciate you chatting.
Geoff Anders:
Great. All right. I’ll see you later-
Anna Salamon:
And all the people in the text.
z1zek100: This seemed good
sowgone: this conversation was amazing, thank you
z1zek100: Love you both
329zwydswr: thanks
habrykas: I appreciate this conversation happening.
larissa24joy: Thank you for doing this Anna and Geoff!
LuliePlays: Thanks Geoff and everyone!
Turbowandray: bye
habrykas: Sorry for not being able to fix the audio things
linacalabria: +1
Turbowandray: ty!
WaistcoatDave: see you a
z1zek100: <3
DaystarEld: Thanks everyone
WaistcoatDave: see you all
Geoff Anders:
All right-
Anna Salamon:
Bye.
Geoff Anders:
Bye everybody.