AnnaSalamon

Karma: 19,955

AnnaSalamon 25 Dec 2025 19:02 UTC
4 points
0
in reply to: Vaniver’s comment on: Why did everything take so long?
Yes, but: a ~200 person tribe still limits the total amount of know-how that can a given tribe can remember and pass down, and then the next tribe over has to remember most of it redundantly, rather than specializing in something else.

AnnaSalamon 24 Dec 2025 22:34 UTC
14 points
2
in reply to: AnthonyC’s comment on: Why does Eliezer make abrasive public comments?
Yes; a different one of the “ten commandments” Szilard tried to live by (they’re really short and are worth reading IMO, will take you 3 min) was “never lie without need.” (This ofc suggests there are times when one does need to; and Szilard helped many Jewish families get out of Nazi Germany at the last minute, in addition to convincing Fermi to not publish; so I would guess he navigated many actual such needs).
In terms of what’s a “need” to lie: IMO the differentiator between “worth lying” and “worth telling the truth” isn’t the stakes (AI existential risk is of course extremely high stakes); it’s more like, how much one needs to avoid “isolation” / having “the meaning of life slip through your fingers” vs how much one needs to get people to do something very specific and local that one already has a sufficient map of, e.g. to walk away from the attic containing Anne Frank. This claim is similar to the claim that AI risk is not well-served by some “emergencies”-suited heuristics, despite being hugely urgent and important.

AnnaSalamon 24 Dec 2025 3:56 UTC
4 points
0
in reply to: k64’s comment on: Why does Eliezer make abrasive public comments?
I appreciate the comment, and agree the case for venting feelings, allowing one’s own status-beliefs to be visible, etc., is worth considering separately from the case for sharing facts accurately.
I do think the quote from Szilard, above, is discussing more than facts / [things with a truth value]. And I think there’s real “virtue of having more actual contact with the world, and with other people” in sharing more of one’s thoughts/feelings/attitudes/etc. Not, as you say, “all the false and irrational things that pass through one’s head,” because all kinds of unimportant nonsense passes through my head sometimes. But I do see some real “virtue of non-consequentialist communication” value to e.g. sharing those feelings, attitudes, viewpoints, ambitions, etc that are the persistent causes of my other thoughts and actions, and to sometimes trying to convey these via direct/poetical images (“smarter than a potted plant”) rather than clinical self-description (“I seem to be annoyed”).
Main upside to doing this:
- They say things I’m not expecting; I say things they’re not expecting; and we can more see through each others’ eyes. (And then e.g. they help me notice ways I’ve been being unfair, or help me figure out what it is that’s weirding me out about a particular thing, or are changed by some aspect of how I’m seeing things in a way that seems reality-contact-increasing, or etc.)
(I agree not all triggered sentences are a good idea to say, because sometimes everybody goes haywire in a useless+damaging way, and sometimes other people don’t want to have to deal with my nonsense and shouldn’t need to, and there’s a whole art to this, but I don’t think it’s an art based in asking whether communication will have good effects.)

AnnaSalamon 23 Dec 2025 16:11 UTC
58 points
32
on: Why does Eliezer make abrasive public comments?
I’m sticking this in comments (not answers) section, because this doesn’t directly bear on the OP’s (1) and (2), nor on Eliezer in particular. But: a different important aspect of public, and private, communication, is that they have direct effects on what the speaker learns, and on whether others can see how the speaker is seeing the world. I mean: communication is sometimes about communicating, rather than about having consequentialist effects on those one is talking to.

Leo Szilard is in the running for all time best rationalists IMO, and one of the “ten commandments” he tried to live by was
Speak to all men as you do to yourself, with no concern for the effect you make, so that you do not shut them out from your world; lest in isolation the meaning of life slips out of sight and you lose the belief in the perfection of creation.
I think there’s something to that.

AnnaSalamon 19 Dec 2025 9:49 UTC
4 points
−4
in reply to: Raemon’s comment on: Help keep AI under human control: Palisade Research 2026 fundraiser
I do think Palisade is operating in the realm of “trying to persuade people of stuff, and that is pretty fraught”
I haven’t had that much contact with Palisade, but I interpreted them as more like “trying to interview people, see how they think, and provide them info they’ll find useful, and let their curiosities/updates/etc be the judge of what they’ll find useful”, which is … not fraught.

Or rather, as somewhere in between this and “trying to persuade people of stuff”, but close enough to the former that I’m in favor, which I’m usually not for “persuasion” orgs.
Am I wrong?

AnnaSalamon 18 Dec 2025 16:09 UTC
6 points
0
on: Announcing RoastMyPost: LLMs Eval Blog Posts and More
Thanks for building; I’m looking forward to trying it. A main thing I keep wanting from LLM writing assistance (I’m not sure how hard this is; I’ve tried prompting LLMs myself, and failed to get the quality I wanted, but I didn’t try with much patience or skill) is help applying Strunk and White’s “The Elements of Style” to my writing. That is, I want help flagging phrases/words/sentence constructions that fail to be short and to the point.

AnnaSalamon 18 Dec 2025 10:17 UTC
4 points
0
in reply to: Raemon’s comment on: The impossible problem of due process
I mean, I might be being dumb on all these points. But I personally disagree about:
- There being a viable “good system for community resolution of conflicts” in larger-than-Dunbar groups (to be fair, the post author does too… except then not at the end?)
- Phrasing the cause-of-enforcement as “you decide a person should ‘face consequences for their actions’” (IMO, kicking people out of a community should usually be more about “they impose risks/costs we can’t live with” and less about “making them face consequences”)
- A sort of missing mood in the third bullet point (“Holistic judgment: a person should be kicked out if they seem, on the whole, to be bad for the community. They don’t need to be found guilty beyond reasonable doubt of a specific egregious crime.”) I agree with the denotation of what’s written. But there are two memorable-to-me cases where I was part of kicking someone out of the bay area rationalist community (not Brent, nothing most overseas readers would’ve heard about; quieter affairs); and where their lives and sanity rapidly got a lot worse, to the point where I’d put like 30% that the decision to exile them “ruined their lives”. I wouldn’t advise past-me against either decision, because they were people we really didn’t know how to live with, with major repeated situations. But … if someone seemed, on the whole, to be a mild force for boredom and awkwardness in the community, say, I certainly woudln’t kick them out? (And again, I assume neither would the post’s author, mingyuan; but I wish the bullet point e.g. said “if they seem, on the whole, to be someone we can’t live with in a healthy fashion”, or else differentiated between kicking someone out of a random meetup, vs doing things that’ll trigger exile from a place that includes almost all a person’s social ties, built up over years).
I think my problem with the last section is only that it is not up to the very high standard that the rest of the post seems to me to hit, in which things are made unusually clear to even a young/inexperienced reader who is happy to believe relayed events but who wants to see the why of things for themself. (And I’m not providing these ‘disagreements’ because I think the article would be better with my opinions instead of the authors; I don’t think I”m especially correct about these matters; I’m providing them as evidence that this part of the article is less visibly-true-to-all-readers, e.g. to me)

AnnaSalamon 18 Dec 2025 8:59 UTC
4 points
0
on: The impossible problem of due process
I appreciate this post for spelling out an unsolved problem that IMO is a major reason it’s hard to build good community gatherings among large groups of people, and for including enough detail/evidence that I expect many, after reading it, can see how the trouble works in their own inside views. I slightly wish the author had omitted the final section (“What would be the elements of a good system?”), as it seems less evidence-backed than the rest (and I personally agree with its claims less), and its inclusion makes it a bit harder for me to recommend the article to those needing a problem-description.

AnnaSalamon 18 Dec 2025 7:56 UTC
2 points
0
on: Partial value takeover without world takeover
I love this post and suspect it’s content is true and underappreciated. (Though I admittedly haven’t found any new ways to test it / etc since it came out.)

AnnaSalamon 18 Dec 2025 7:41 UTC
4 points
0
in reply to: Screwtape’s comment on: On Not Pulling The Ladder Up Behind You
I like it, but I wish its main point would stick better in my mind somehow. (This was true when I read it last year, and again when I re-skimmed it now.) I, too like the ladder metaphor; I agree that it helps get people thinking about on-ramps, and that that this is valuable; I like the examples and techniques about remembering how you got there, imagining a new early-you who showed up today, etc. But: I still feel there’s a “whole” you’re gesturing at that’s not quite sticking in my head, and I wonder if a slight rewrite could get it to?

AnnaSalamon 18 Dec 2025 7:28 UTC
4 points
0
on: Neutrality
I read this once when Sarah wrote it, just over a year ago, and I still think about it ~every two weeks or so. It convinced me that it’s possible and desirable to be neutral along some purpose-relevant axes, and that I should keep my eye on where and how this is accomplished, and what it does. (I stayed convinced.) Hoping it makes it in.

AnnaSalamon 16 Dec 2025 3:17 UTC
8 points
2
on: Parental Writing Selection Bias
I appreciate the explicit, fairly clear discussion of a likely gap in what I’m reading about parenting and kids. I was aware of a gap near here, but the post added a bit of detail to my model, and I like having it in common knowledge; I also hope it may encourage other such posts. (Plus, it’s short and easy to read.)

AnnaSalamon 16 Dec 2025 2:47 UTC
2 points
0
on: Deep and obvious points in the gap between your thoughts and your pictures of thought
Nominating this for 2024 review. It seems like an accurate (in many cases, at least) model of a phenomenon I care about (and encounter fairly frequently, in myself and in people I end up trying to help with things) that I didn’t previously have an accurate model of.

AnnaSalamon 11 Dec 2025 0:22 UTC
12 points
2
in reply to: TsviBT’s comment on: Eliezer’s Unteachable Methods of Sanity
A further wrinkle / another example is that a question like “what should I think about (in particular, what to gather information about / update about)”, during the design process, wants these predictions.
Yes; this (or something similar) is why I suspect that “‘believing in’ atoms” may involve the same cognitive structure as “‘believing in’ this bakery I am helping to create” or “‘believing in’ honesty” (and a different cognitive structure, at least for ideal minds, from predictions about outside events). The question of whether to “believe in” atoms can be a question of whether to invest in building out and maintaining/tuning an ontology that includes atoms.

AnnaSalamon 10 Dec 2025 15:33 UTC
19 points
7
in reply to: Eliezer Yudkowsky’s comment on: Eliezer’s Unteachable Methods of Sanity
Prediction and planning remain incredibly distinct as structures of cognitive work,
I disagree. (Partially.) For a unitary agent who is working with a small number of possible hypotheses (e.g., 3), and a small number of possible actions, I agree with your quoted sentence.
But let’s say you’re dealing with a space of possible actions that’s much too large to let you consider each exhaustively, e.g. what blog post to write (considered concretely, as a long string of characters).
It’d be nice to have some way to consider recombinable pieces, e.g. “my blog post could include idea X”, “my blog post could open with joke J”, “my blog post could be aimed at a reader similar to Alice”.
Now consider the situation as seen by the line of thinking that is determining: “should my blog post be aimed mostly at readers similar to Alice, or at readers similar to Bob?”. For this line of thinking to do a good estimate of ExpectedUtility(post is aimed at Alice), it needs predictions about whether the post will contain idea X. However, for the line of thinking that is determining whether to include idea X (or the unified agent, at those moments when it is actively considering this), it’’ll of course need good plans (not predictions) about whether to include X, and how exactly to include X.
I don’t fully know what a good structure is for navigating this sort of recombinable plan space, but it might involve a lot of toggling between “this is a planning question, from the inside: shall I include X?” and “this is a prediction question, from the outside: is it likely that I’m going to end up including X, such that I should plan other things around that assumption?”.
My own cognition seems to me to toggle many combinatorial pieces back and forth between planning-from-the-inside and predicting-from-the-outside, like this. I agree with your point that human brains and bodies have all kinds of silly entanglements. But this part seems to me like a plausible way for other intelligences to evolve/grow too, not a purely one-off humans idiosyncrasy like having childbirth through the hips.

AnnaSalamon 14 Oct 2025 2:13 UTC
11 points
17
in reply to: cata’s comment on: What is Lesswrong good for?
Also we understand basic arithmetic around here, which goes a long way sometimes.

AnnaSalamon 13 Oct 2025 4:39 UTC
4 points
0
in reply to: ChristianKl’s comment on: Ethical Design Patterns
It’s a good point, re: some of the gap being that it’s hard to concretely visualize the world in which AGI isn’t built. And also about the “we” being part of the lack of concreteness.
I suspect there’re lots of kinds of ethical heuristics that’re supposed to interweave, and that some are supposed to be more like “checksums” (indicators everyone can use in an embodied way to see whether there’s a problem, even though they don’t say how to address it if there is a problem), and others are supposed to be more concrete.
For some more traditional examples:
- There’re heuristics for how to tell whether a person or organization is of bad character (even though these heuristics don’t tell how how to respond if a person is of bad character). Eg JK Rowling’s character Sirius’s claim that you can see the measure of a person by how they treat their house-elves (which has classical Christian antecedents, I’m just mentioning a contemporary phrasing).
- There’re heuristics for how countries should be, e.g. “should have freedom of speech and press” or (longer ago) “should have a monarch who inherited legitimately.”
It would be too hard to try to equip humans and human groups for changing circumstances via only a “here’s what you do in situation X”. It’s somewhat easier to do it (and traditional ethical heuristics did do it) by a combination of “you can probably do well by [various what-to-do heuristics]” and “you can tell if you’re doing well by [various other checksum-type heuristics]. Ethics is help to let us design our way to better plans, not to only always give us those plans.

AnnaSalamon 13 Oct 2025 4:19 UTC
2 points
0
in reply to: AnnaSalamon’s comment on: Adele Lopez’s Shortform
(Nevermind, after thinking about it a bit more I think I get it.)

AnnaSalamon 13 Oct 2025 3:53 UTC
6 points
0
in reply to: Adele Lopez’s comment on: Adele Lopez’s Shortform
Another place where I’ll think and act somewhat differently as a result of this conversation:
- It’s now higher on my priority list to try to make sure CFAR doesn’t act as a “gateway” to all kinds of weird “mental techniques” (or quasi-cults who use “mental techniques”). Both for CFAR’s new alumni, and for social contacts of CFAR’s new alumni. (This was already on some lists I’d made, but seeing Adele derive it independently bumped it higher for me.)
What links here?
- AnnaSalamon's comment on Adele Lopez’s Shortform by Adele Lopez (13 Oct 2025 4:19 UTC; 2 points)

AnnaSalamon 13 Oct 2025 3:36 UTC
2 points
0
in reply to: Wei Dai’s comment on: Ethical Design Patterns
Okay, but: it’s also find individuals who are willing to speak for heuristic C, in a way I suspect differs from what it was like for leaded gasoline and from what I remember as a kid in the late 80′s about the ozone layer.
It’s a fair point that I shouldn’t expect “consensus”, and should’ve written and conceptualized that part differently, but I think heuristic C is also colliding with competing ethical heuristics in ways the ozone situation didn’t.