AnnaSalamon

Karma: 12,235

AnnaSalamon 17 Oct 2021 1:12 UTC
38 points
in reply to: AnnaSalamon’s comment on: My experience at and around MIRI and CFAR (inspired by Zoe Curzi’s writeup of experiences at Leverage)
To be clear, a lot of what I find so relaxing about Jessica’s post is that my experience reading it is of seeing someone who is successfully noticing a bunch of details in a way that, relative to what I’m trying to track, leaves room for lots of different things to get sorted out separately.

I just got an email that led me to sort of triggeredly worry that folks will take my publicly agreeing with the OP to mean that I e.g. think MIRI is bad in general. I don’t think that; I really like MIRI and have huge respect and appreciation for a lot of the people there; I also like many things about the CFAR experiment and love basically all of the people who worked there; I think there’s a lot to value across this whole space.

I like the detailed specific points that are made in the OP (with some specific disagreements; though also with corroborating detail I can add in various places); I think this whole “how do we make sense of what happens when people get together into groups? and what happened exactly in the different groups?” question is an unusually good time to lean on detail-tracking and reading comprehension.

AnnaSalamon 16 Oct 2021 23:00 UTC
18 points
in reply to: AnnaSalamon’s comment on: My experience at and around MIRI and CFAR (inspired by Zoe Curzi’s writeup of experiences at Leverage)

I, in fact, asked a CFAR instructor in 2016-17 whether the idea was to psychologically improve yourself until you became Elon Musk, and he said “yes”. This part of the plan was the same.

Re: “this part of the plan was the same”: IMO, some at CFAR were interested in helping some subset of people become Elon Musk, but this is different from the idea that everyone is supposed to become Musk and that that is the plan. IME there was usually mostly (though not invariably, which I expect led to problems; and for all I know “usually” may also have been the case in various parts and years of Leverage) acceptance for folks who did not wish to try to change themselves much.
What links here?
- My experience at and around MIRI and CFAR (inspired by Zoe Curzi’s writeup of experiences at Leverage) by jessicata (16 Oct 2021 21:28 UTC; 102 points)

AnnaSalamon 16 Oct 2021 22:53 UTC
42 points
on: My experience at and around MIRI and CFAR (inspired by Zoe Curzi’s writeup of experiences at Leverage)
Here is a thread for detail disagreements, including nitpicks and including larger things, that aren’t necessarily meant to connect up with any particular claim about what overall narratives are accurate. (Or maybe the whole comment section is that, because this is LessWrong? Not sure.)

I’m starting this because local validity semantics are important, and because it’s easier to get details right if I (and probably others) can consider those details without having to pre-compute whether those details will support correct or incorrect larger claims.

For me personally, part of the issue is that though I disagree with a couple of the OPs details, I also have some other details that support the larger narrative which are not included in the OP, probably because I have many experiences in the MIRI/CFAR/adjacent communities space that Jessicata doesn’t know and couldn’t include. And I keep expecting that if I post details without these kinds of conceptualizing statements, people will use this to make false inferences about my guesses about higher-order-bits of what happened.

AnnaSalamon 16 Oct 2021 22:23 UTC
54 points
on: My experience at and around MIRI and CFAR (inspired by Zoe Curzi’s writeup of experiences at Leverage)
FWIW, the above matches my own experiences/observations/hearsay at and near MIRI and CFAR, and seems to me personally like a sensible and correct way to put it together into a parsable narrative. The OP speaks for me.

(I of course still want other conflicting details and narratives that folks may have; my personal ‘oh wow this puts a lot of pieces together in a parsable form that yields basically correct predictions’ level is high here, but insofar as I’m encouraging anything because I’m in a position where my words are loud invitations, I want to encourage folks to share all the details/stories/reactions pointing in all the directions.) I also have a few factual nitpicks that I may get around to commenting, but they don’t subtract from my overall agreement.

I appreciate the extent to which you (Jessicata) manage to make the whole thing parsable and sensible to me and some of my imagined readers. I tried a couple times to write up some bits of experience/thoughts, but had trouble managing to say many different things A without seeming to also negate other true things A’, A’’, etc., maybe partly because I’m triggered about a lot of this / haven’t figured out how to mesh different parts of what I’m seeing with some overall common sense, and also because I kept anticipating the same in many readers.

AnnaSalamon 14 Oct 2021 20:08 UTC
40 points
on: Zoe Curzi’s Experience with Leverage Research
I imagine a lot of people want to say a lot of things about Leverage and the dynamics around it, except it’s difficult or costly/risky or hard-to-imagine-being-heard-about or similar.

If anyone is up for saying a bit about how that is for you personally (about what has you reluctant to try to share stuff to do with Leverage, or with EA/Leverage dynamics or whatever, that in some other sense you wish you could share — whether you had much contact with Leverage or not), I think that would be great and would help open up space.

I’d say err on the side of including the obvious.

AnnaSalamon 14 Oct 2021 19:59 UTC
8 points
in reply to: Spiracular’s comment on: Common knowledge facts about Leverage Research 1.0
That’s right; I am daydreaming of something very difficult being brought together somehow, in person or in writing (probably slightly less easily-visible-across-the-whole-internet writing, if in writing). I’d be interested in helping but don’t have the know-how on my own to pull it off. I agree with you there’re lots of ways to try this and make things worse; I expect it’s key to have very limited ambitions and to be clear about how very much one is not attempting/promising.

AnnaSalamon 14 Oct 2021 15:26 UTC
8 points
in reply to: Kaj_Sotala’s comment on: Common knowledge facts about Leverage Research 1.0
Yes.

AnnaSalamon 14 Oct 2021 9:00 UTC
10 points
in reply to: rohinmshah’s comment on: Zoe Curzi’s Experience with Leverage Research

I’m also not a fan of requests that presume that the listener …

From my POV, requests, and statements of what I hope for, aren’t advice. I think they don’t presume that the listener will want to do them or will be advantaged by them, or anything much else about the listener except that it’s okay to communicate my request/hope to them. My requests/hopes just share what I want. The listener can choose for themselves, based on what they want. I’m assuming listeners will only do things if they don’t mind doing them, i.e. that my words won’t coerce people, and I guess I’m also assuming that my words won’t be assumed to be a trustworthy voice of authorities that know where the person’s own interests are, or something. That I can be just some person, hoping and talking and expecting to be evaluated by equals.

Is it that you think these assumptions of mine are importantly false, such that I should try to follow some other communication norm, where I more try to only advocate for things that will turn out to be in the other party’s interests, or to carefully disclaim if I’m not sure what’ll be in their interests? That sounds tricky; I’m not peoples’ parents and they shouldn’t trust me to do that, and I’m afraid that if I try to talk that way I’ll make it even more confusing for anyone who starts out confused like that.

I think I’m missing part of where you’re coming from in terms of what good norms are around requests, or else I disagree about those norms.

you have way more private info than me, so perhaps...

I don’t have that much relevant-info-that-hasn’t-been-shared, and am mostly not trying to rely on it in whatever arguments I’m making here. Trying to converse more transparently, rather.

AnnaSalamon 14 Oct 2021 3:34 UTC
21 points
in reply to: farp’s comment on: Zoe Curzi’s Experience with Leverage Research

I would like it if we showed the world how accountability is done

So would I. But to do accountability (as distinguished from scapegoating, less-epistemic blame), we need to know what happened, and we need to accurately trust each other (or at least most of each other) to be able to figure out what happened, and to care what actually happened.

The “figure out what happened” and “get in a position where we can have a non-fucked conversation” steps come first, IMO.

I also sort of don’t expect that much goal divergence on the accountability steps that very-optimistically come after those steps, either, basically because integrity and visible trustworthiness serve most good goals in the long run, and vengeance or temporarily-overextended-trust serves little.

Though, accountability is admittedly a weak point of mine, so I might be missing/omitting something. Maybe spell it out if so?

AnnaSalamon 14 Oct 2021 3:16 UTC
35 points
in reply to: rohinmshah’s comment on: Zoe Curzi’s Experience with Leverage Research
Thanks for the clarifying question, and the push-back. To elaborate my own take: I (like you) predict that some (maybe many) will take shared facts in a politicized way, will use them as an excuse for false or uncareful judgments, etc. I am not guaranteeing, nor predicting, that this won’t occur.

I am intending to myself do inference and conversation in a way that tries to avoids these “politicized speech” patterns, even if it turns out politically costly or socially awkward for me to do so. I am intending to make some attempt (not an infinite amount of effort, but some real effort, at some real cost if needed) to try to make it easier for others to do this too, and/or to ask it of others who I think may be amenable to being asked this, and/or to help coalesce a conversation in what I take to be a better pattern if I can figure out how to do so. I also predict, independently of my own efforts, that a nontrivial number of others will be trying this.

If “reputation management” is a person’s main goal, then the small- to medium-sized efforts I can hope to contribute toward a better conversation, plus the efforts I’m confident in predicting independently of mine, would be insufficient to mean that a person’s goals would be well-served in the short run by following my request to avoid “refraining from sharing true relevant facts, out of fear that others will take them in a politicized way, or will use them as an excuse for false judgments.”

However, I’m pretty sure most people in this ecosystem, maybe everyone, would deep down like to figure out how to actually see what kinds of fucked up individual and group and inter-group dynamics we’ve variously gotten into, and why and how, so that we can have a realer shot at things going forward. And I’m pretty sure we want this (in the deeper, long-run sense) much more than we want short-run reputation management. Separately, I suspect most peoples’ reputation-management will be kinda fucked in the long run if we don’t figure out how to get enough right to do actual progress on the world (vs creating local illusions of the same), although this last sentence is more controversial and less obvious. So, yeah, I’m asking people to try to engage in real conversation with me and others even though it’ll probably mess up parts of their/our reputation in the short run, and even though probably many won’t manage to joint this in the short run. And I suspect this effort will be good for many peoples’ deeper goals despite the political dynamics you mention.

Here’s to trying.

AnnaSalamon 13 Oct 2021 17:30 UTC
40 points
in reply to: Rob Bensinger’s comment on: Common knowledge facts about Leverage Research 1.0

???? I’m so confused about what happened here. The aliens part (as stated) isn’t a red flag for me, but the Kant thing seem transparently crazy to me. I have to imagine there’s something being lost in translation here, and missing context for why people didn’t immediately see that this person was having a mental breakdown?

FWIW, my own experience is that people often miss fairly blatant psychotic episodes; so I’m not sure how Leverage-specific the explanation needs to be for this one. For example, once I came to believe that an acquaintance was having a psychotic episode and suggested he see a psychiatrist; the psychiatrist agreed. A friend who’d observed most of the same data I had asked me how I’d known. I said it was several things, but that the bit where our acquaintance said God was talking to him through his cereal box was one of the tip-offs from my POV. My friend’s response was “oh, I thought that was a metaphor.” I know several different stories like this one, including a later instance where I was among those who missed what in hindsight was fairly blatant evidence that someone was psychotic, none of which involved weird group-level beliefs or practices.

AnnaSalamon 13 Oct 2021 4:32 UTC
36 points
on: How to think about and deal with OpenAI
I think we should not be hesitant to talk about this in public. I used to be of the opposite opinion, believing-as-if there was a benevolent conspiracy that figured out which conversations could/couldn’t nudge AI politics in useful ways, whose upsides were more important than the upsides of LWers/etc. knowing what’s up. I now both believe less in such a conspiracy, and believe more that we need public fora in which to reason because we do not have functional private fora with memory (in the way that a LW comment thread has memory) that span across organizations.

It’s possible I’m still missing something, but if so it would be nice to have it spelled out publicly what exactly I am missing.

I agree with Lincoln Quirk’s comment that things could turn into a kind of culture war, and that this would be harmful. It seems to me it’s worth responding to this by trying unusually hard (on this or other easily politicizable topics) to avoid treating arguments like soldiers. But it doesn’t seem worthwhile to me to refrain from honest attempts to think in public.

AnnaSalamon 13 Oct 2021 3:40 UTC
15 points
in reply to: lincolnquirk’s comment on: How to think about and deal with OpenAI
I disagree with Lincoln’s comment, but I’m confused that when I read it just now it was at −2; it seems like a substantive comment/opinion that deserves to be heard and part of the conversation.

If comments expressing some folks’ actual point of view are downvoted below the visibility threshold, it’ll be hard to have good substantive conversation.

AnnaSalamon 13 Oct 2021 3:25 UTC
58 points
on: Zoe Curzi’s Experience with Leverage Research
More thoughts:

I really care about the conversation that’s likely to ensue here, like probably a lot of people do.

I want to speak a bit to what I hope happens, and to what I hope doesn’t happen, in that conversation. Because I think it’s gonna be a tricky one.

What I hope happens:
- Curiosity
- Caring,
- Compassion,
- Interest in understanding both the specifics of what happened at Leverage, and any general principles it might hint at about human dynamics, or human dynamics in particular kinds of groups.
What I hope doesn’t happen:
- Distancing from uncomfortable data.
- Using blame and politics to distance from uncomfortable data.
- Refraining from sharing true relevant facts, out of fear that others will take them in a politicized way, or will use them as an excuse for false judgments.
This is LessWrong; let’s show the world how curiosity/compassion/inquiry is done!

AnnaSalamon 13 Oct 2021 3:24 UTC
18 points
in reply to: Aella’s comment on: Common knowledge facts about Leverage Research 1.0
CFAR recently hosted a “Speaking for the Dead” event, where a bunch of current and former staff got together to try to name as much as we could of what had happened at CFAR, especially anything that there seemed to have been (conscious or unconscious) optimization to keep invisible.

CFAR is not dead, but we took the name anyhow from Orson Scott Card’s novel by the same name, which has quotes like:

“...and when their loved ones died, a believer would arise beside the grave to be the Speaker for the Dead, and say what the dead one would have said, but with full candor, hiding no faults and pretending no virtues.”

“A strange thing happened then. The Speaker agreed with her that she had made a mistake that night, and she knew when he said the words that it was true, that his judgment was correct. And yet she felt strangely healed, as if simply saying her mistake were enough to purge some of the pain of it. For the first time, then, she caught a glimpse of what the power of speaking might be. It wasn’t a matter of confession, penance, and absolution, like the priests offered. It was something else entirely. Telling the story of who she was, and then realizing that she was no longer the same person. That she had made a mistake, and the mistake had changed her, and now she would not make the mistake again because she had become someone else, someone less afraid, someone more compassionate.”

“… there were many who decided that their life was worthwhile enough, despite their errors, that when they died a Speaker should tell the truth for them.”

CFAR’s “speaking for the dead” event seemed really good to me. Healing, opening up space for creativity. I hope the former members of Leverage are able to do something similar. I really like and appreciate Zoe sharing all these details, and I hope folks can meet her details with other details, all the details, whatever they turn out to have been.

I don’t know what context permits that kind of conversation, but I hope all of us on the outside help create whatever kind of context it is that allows truth to be shared and heard.

AnnaSalamon 17 Aug 2021 17:53 UTC
47 points
in reply to: mirona’s comment on: Rule Thinkers In, Not Out
Um, good point. I am not sure which details you’re asking about, but I am probably happy to elaborate if you ask something more specific.

I hereby apologize for the role I played in Michael Vassar’s ostracism from the community, which AFAICT was both unjust and harmful to both the community and Michael. There’s more to say here, and I don’t yet know how to say it well. But the shortest version is that in the years leading up to my original comment Michael was criticizing me and many in the rationality and EA communities intensely, and, despite our alleged desire to aspire to rationality, I and I think many others did not like having our political foundations criticized/eroded, nor did I and I think various others like having the story I told myself to keep stably “doing my work” criticized/eroded. This, despite the fact that attempting to share reasoning and disagreements is in fact a furthering of our alleged goals and our alleged culture. The specific voiced accusations about Michael were not “but he keeps criticizing us and hurting our feelings and/or our political support” — and nevertheless I’m sure this was part of what led to me making the comment I made above (though it was not my conscious reason), and I’m sure it led to some of the rest of the ostracism he experienced as well. This isn’t the whole of the story, but it ought to have been disclosed clearly in the same way that conflicts of interest ought to be disclosed clearly. And, separately but relatedly, it is my current view that it would be all things considered much better to have Michael around talking to people in these communities, though this will bring friction.

There’s broader context I don’t know how to discuss well, which I’ll at least discuss poorly:
- Should the aspiring rationality community, or any community, attempt to protect its adult members from misleading reasoning, allegedly manipulative conversational tactics, etc., via cautioning them not to talk to some people? My view at the time of my original (Feb 2019) comment was “yes”. My current view is more or less “heck no!”; protecting people from allegedly manipulative tactics, or allegedly misleading arguments, is good — but it should be done via sharing additional info, not via discouraging people from encountering info/conversations. The reason is that more info tends to be broadly helpful (and this is a relatively fool-resistant heuristic even if implemented by people who are deluded in various ways), and trusting who can figure out who ought to restrict their info-intake how seems like a doomed endeavor (and does not degrade gracefully with deludedness/corruption in the leadership). (Watching the CDC on covid helped drive this home for me. Belatedly noticing how much something-like-doublethink I had in my original beliefs about Michael and related matters also helped drive this home for me.)
- Should some organizations/people within the rationality and EA communities create simplified narratives that allow many people to pull in the same direction, to feel good about each others’ donations to the same organizations, etc.? My view at the time of my original (Feb 2019) comment was “yes”; my current view is “no — and especially not via implicit or explicit pressures to restrict information-flow.” Reasons for updates same as above.
It is nevertheless the case that Michael has had a tendency to e.g. yell rather more than I would like. For an aspiring rationality community’s general “who is worth ever talking to?” list, this ought to matter much less than the above. Insofar as a given person is trying to create contexts where people reliably don’t yell or something, they’ll want to do whatever they want to do; but insofar as we’re creating a community-wide include/exclude list (as in e.g. this comment on whether to let Michael speak at SSC meetups), it is my opinion that Michael ought to be on the “include” list.

Thoughts/comments welcome, and probably helpful for getting to shared accurate pictures about any of what’s above.
What links here?
- AnnaSalamon's comment on Rule Thinkers In, Not Out by Scott Alexander (28 Feb 2019 0:02 UTC; 100 points)

AnnaSalamon 7 Aug 2021 1:41 UTC
22 points
on: What are some good pieces on civilizational decay / civilizational collapse / weakening of societal fabric?
I find it very difficult to reason rigorously about this topic, and none of the below is rigorous. But in terms of sources of hypotheses or proto-models of what if anything is up here, some of my favorite books:
- Atlas Shrugged (a fictional portrayal of societal decay that has been increasingly reminding me of current events, particularly since 2016 or 2020)
- Zen and the Art of Motorcycle Maintenance (an argument that the West was, since ~Aristotle, built on an almost-coherent philosophy whose incoherence has been becoming more visible across time, and a theory of what the gap is. Pays some rent IMO).
- Moral Mazes (similar content to Atlas Shrugged, and not as deep I think, but maybe easier to read because based in current observations instead of fiction)
The above are my favorites. Others that seem less good but still helpful to me:
- Possibly Pink Floyd’s movie “The Wall”, which shows societal programming since WW1 as a set of increasingly out of sync software programs being echoed and richocheted by a bunch of overwhelmed/traumatized people and processes
- Maybe Venkatesh Rao
- The Tao Te Ching — it’s pretty abstract but it has a fair bit to say about what happens when Goodhart’s Law hits mass human behavior
- Chapter 1 of Vol. 1 of Hayek’s book “Law, Legislation, and Liberty” (the introduction was too hard, and I didn’t make it much past the first chapter, but I got a lot from that chapter)
- Possibly “Century of the Self” (a 4-hour propaganda film about the impact of marketing/psychology/self-help on the contemporary world; possibly a must for anyone as involved in trying to develop self-help or “rationality” techniques as you are), and/or the more recent movie sequence that I am partway through by the same person, “Can’t get you out of my head”. Both by Adam Curtis.
- Possibly Vervaeke’s 50-hour history of Western Civilization called “Awakening from the meaning crisis”, on themes similar to Pirsig’s “Zen and the Art of Motorcycle Maintenance”
- Possibly Eric Weinstein’s model of “embedded growth obligations”, as a theory of how society in general accidentally overpromised and then headed toward lies/unsustainability rather than rethink itself, kinda like a Ponzi scheme.
I find most relevant Michael Vassar, Ben Hoffman, Jessica Taylor, and Zvi on the same topics; I’d have listed them toward the top except that you (Eli) already know them, and I’m afraid of setting off peoples’ “politicized topic” detectors. I’d be pretty interested to try exploring why/how their writing sets this off in people, and what it indicates about them/us, if anyone wants to.

AnnaSalamon 21 Jul 2021 20:42 UTC
13 points
in reply to: Bjartur Tómas’s comment on: The shoot-the-moon strategy
Trying to advance AI in hopes that your team will get all the way to AGI first, and in hopes that you’ll also somehow solve alignment at the same time. Backfires if your AI-advancing ideas leak, especially if you don’t actually manage to be first. Backfires worst in worlds where you came closest to succeeding by actually making tons of AI progress. Reminds me of “shooting the moon” in the card game Hearts that way.

Sometimes I wonder whether the whole current AI safety movement is net harmful via prompting many safety-concerned people and groups to attempt AI capabilities research with unlikely “shoot-the-moon”-style payoffs in mind.

AnnaSalamon 2 Jun 2021 22:05 UTC
21 points
on: Curated conversations with brilliant rationalists
Also one Spencer recorded with me: “Lines of Retreat and Incomplete Maps”. Not sure why it isn’t above; maybe was from earlier than the ones listed.

AnnaSalamon 26 May 2021 23:17 UTC
2 points
in reply to: Craig Falls’s comment on: Peekskill Lyme Incidence

monitoring for symptoms and taking a large dose of antibiotics within ~48 hours of symptoms is extremely effective.

I’m confused about this. Can you say more about what your threshold is for “extremely effective,” or why you think so? Wikipedia states: “People who receive recommended antibiotic treatment within several days of appearance of an initial EM rash have the best prospects.[106] Recovery may not be total or immediate. The percentage of people achieving full recovery in the United States increases from about 64–71% at end of treatment for EM rash to about 84–90% after 30 months; higher percentages are reported in Europe.[171][172] Treatment failure, i.e. persistence of original or appearance of new signs of the disease, occurs only in a few people.[171] Remaining people are considered cured but continue to experience subjective symptoms, e.g. joint or muscle pains or fatigue.[173] These symptoms usually are mild and nondisabling.[173]”

This leaves me thinking that even with rapid antibiotics, the debilitation per (infection that causes a rash) is significant.