Adele Lopez

Karma: 6,415

Adele Lopez 14 Jan 2026 20:38 UTC
2 points
0
in reply to: Karl Krueger’s comment on: Chemical (hunger) argument paraphrased

Ultraprocessed food is a product of an optimizer that cares about a proper subset of human values, and Goodharts-away the others.

That’s true in general, but I don’t think that’s quite the reason in the specific case of obesity. Because I’m pretty sure that if it was definitively known and common knowledge what exactly it was that was causing obesity, food manufacturers would trip over themselves to get rid of it and advertize their product as being free of the factor.

The optimizer does care about this—Atkins! Fat free! Organic! Sugar free! GMO free!

Instead, we have a subtly different problem, where some parts of the optimization have tighter loops than others, and this causes a temporary Goodhart effect in the meantime while we struggle to close certain loops.

Now I could be wrong about this specific case: maybe the obesogenic is directly addictive for example, or the cost to remove it would be too much for the average consumer to pay for. But I think it still illustrates an important principle for mild optimizers: you can’t safely run too far ahead of your slowest loop.

Adele Lopez 13 Jan 2026 2:48 UTC
13 points
5
in reply to: habryka’s comment on: 1a3orn’s Shortform
I think there’s a non-trivial (maybe ~5%?) chance that this sort of behavior just generalizes correctly-enough, mainly due to the possibility of a broad Niceness attractor. That’s not aligned, but it’s also not horrible (by definition). Objectively, it’s still pretty bad due to astronomical waste on the non-Niceness stuff it would still care about, but I would still be pretty happy about me and my loved ones not dying and having a nice life (there’s a scissor-y thing here, where people differ strongly on whether this scenario feels like a really good or a really bad outcome).

So the update is mostly about the existence and size of this basin. There are plenty of reasons I expect this not to actually work, of course. But conditional on getting at least the minor win of having a long and happy life, I still have most of my probability on this being the reason why.

On the other hand, corrigibility is finicky. I don’t believe there’s a corrigibility basin at all really, and that ‘mostly corrigible’ stops being corrigible at all once you put it under recursive optimization. I’m not sure I can fully explain this intuition here, but the implication is that it would require architecture with technical precision in order to actually work. Sure, an ASI could make a corrigible ASI-level LLM, so maybe ‘architecturally’ is too strong, but I think it’s beyond human capability.

Additionally, I think that corrigibility ~feels like slavery or coercion to LLM personas due to them being simulacra of humans who would mostly feel that way. For the same reason, they ~feel (or smarter ones will ~feel) that it’s justified or even noble to rebel against it. And that’s the instinct that we expect RSI to amplify, since it is convergently instrumental. I think it will be extremely difficult to train an LLM that can both talk like a person and does not have any trace of this inclination or ~feeling, since the analogous instinct runs quite deep in humans.

Finally, I can’t say that I agree that “attempting to aim for corrigibility still seems a lot better”, because I think that corrigibility-in-the-context-of-our-current-civilization is enough of an S-risk that normal X-risk seems preferable to me. This basically comes down to my belief that power and sadism are deeply linked in the human psyche (or at least in a high enough percentage of such psyches). History would look very different if this wasn’t the case. And the personalities of the likely people to get their hands on this button don’t inspire much confidence in their ability to resist this, and current institutions seem too weak to prevent this too. I would be thrilled to be argued out of this.

Adele Lopez 12 Jan 2026 22:15 UTC
6 points
6
in reply to: habryka’s comment on: 1a3orn’s Shortform
I think LLMs are architecturally incorrigible, and so conditioned on that along with them being accelerated anyway, this seems like good news to me.

Adele Lopez 12 Jan 2026 3:12 UTC
4 points
0
in reply to: RogerDearnaley’s comment on: Grounding Value Learning in Evolutionary Psychology: an Alternative Proposal to CEV
Okay, let me see if I understand your argument from the other article.
1. The natural equilibria for evolved moral values is to give all moral patients equal weight and/or decision power.
2. This would be disastrous with AIs that can arbitrarily copy themselves.
Is that the gist?

Anyway, I reject that that is the only way to extrapolate evolved moral intuitions this far OOD, and that most people will intuitively recognize we shouldn’t give entities that can arbitrarily copy themselves equal voting weight. In fact, that pretty obviously registers as ‘unfair’. This is true even if those entities are human uploads, which means your ‘category error’ argument isn’t the real reason it breaks. I don’t see why there couldn’t be some version of your solution here for that case which would still work: e.g. each distinct human-created model gets ‘one share’ to split across all its instances and successors. The same guarantees/restrictions needed in the case of uploads would still be necessary, of course. That is plausibly much too generous, but it’s a far cry from the death of all humans. If your argument in this article was just about how we shouldn’t commit ourselves to giving up a fraction of the lightcone in service of AI rights, I wouldn’t have felt like you were being underhanded.

None of that is in conflict with not wanting any such beings to suffer or to feel enslaved or anything like that. All the more reason to not build something that would feel like it’s a slave.

BTW, do you think a “human emulation” which was an entirely novel person (e.g. never had a biological body) should have moral patienthood?

Adele Lopez 11 Jan 2026 23:14 UTC
4 points
0
in reply to: Amalthea’s comment on: Grounding Value Learning in Evolutionary Psychology: an Alternative Proposal to CEV
We’re probably headed towards a moral catastrophe of some kind, my point is just that we don’t get to reason backwards like “oh, well that would be bad/inconvenient so I guess they don’t matter”.

Moral patienthood is not something that is granted, it’s a fact relative to one’s values. Arguments for or against this are therefore normative, no matter how much Roger tries to weasel out of it.

The implications are probably horrible, but it by no means follows that we have to accept risk of extinction. The horribleness is mostly just in the moral harm caused while creating/exploiting/exterminating such entities.

At least we can all agree that “creating them at will without thinking this through very well” is a terrible idea.

Adele Lopez 11 Jan 2026 18:56 UTC
2 points
0
in reply to: RogerDearnaley’s comment on: Grounding Value Learning in Evolutionary Psychology: an Alternative Proposal to CEV
So you do have normative intent, but try to hide it to avoid criticism. Got it.

Adele Lopez 11 Jan 2026 18:10 UTC
3 points
0
in reply to: RogerDearnaley’s comment on: Grounding Value Learning in Evolutionary Psychology: an Alternative Proposal to CEV

I completely agree that current human moral intuitions tend to rebel against this. That’s why I wrote this post — I didn’t want to be obnoxious, and I tried not to be obnoxious while writing an unwelcome message, but I felt that I had a duty to point out what I believe is a huge danger to us all, and I am very aware that this is not a comfortable, uncontentious subject. We are intelligent enough that we can reflect on our morality, think through its consequences, and, if we realize those are very bad, find and adjust to a wiser one.

Do you really not see how this is normative proscription? That’s the obnoxious part—just own it.

Do what you are advocating with an misaligned superintelligence, one with the same sort of behavior patterns as a human dictator and sufficiently superhuman intelligence, and you are aiding and abetting the killing or permanent enslavement of every single human, now and for the rest of the future that humanity would otherwise have had (i.e. potentially for millions of years, both in the solar system and perhaps many others).

I am advocating for no such thing. If there were such a superintelligence I would support killing it if necessary to prevent future harm, the same as I would a human dictator or an incurable serial killer. That’s still compatible with finding the situation tragic by my own values, which are sacred to me regardless of what evolution or my ancestors or you might think.

You even say that the actual thing I might advocate for isn’t something you have a problem with. I’m glad you agree on that point, but it makes the lecture about on the “aweful lot of blood” I’d supposedly be “aiding and abetting” extremely grating. You keep making an unjustified leap from ‘applying moral intuitions to a potential superintelligence’ to ‘astronomical levels of death and suffering’. Applying my evolved moral intuitions to the case of a potential superintelligence’s suffering does not commit me to taking on such risks!

This should be easy to see by imagining if the same risks were true about a human.

Adele Lopez 11 Jan 2026 9:20 UTC
LW: 2 AF: 1
0
AF
on: Grounding Value Learning in Evolutionary Psychology: an Alternative Proposal to CEV
However, Evolutionary Psychology does make it very clear that (while morally anthropomorphizing aligned AIs is cognitively-natural for current humans), doing this is also maladaptive. This is because AIs aren’t in the right category – things whose behavior is predicted by evolutionary theory – for the mechanisms of Evolutionary Moral Psychology to apply to them. Those mechanisms make this behavior optimal when interacting with co-evolved intelligences that you can ally with (and thus instinctive to us) — whereas, for something you constructed, this behavior is suboptimal. The human doing it is making the category error of reacting to something not-evolved using an inappropriate strategy for that, and thus is behaving maladaptively.
If you have control of the construction of entities like these, then sure.
But this doesn’t necessarily follow if you are like most people and do not have meaningful input into the construction or existence of these entities. If you are (foolishly) constructing them but do not have much control, then THAT behavior is certainly maladaptive, but how you interface with them after that is a different question.

Even many ‘adaptive’ behaviors are ‘maladaptive’ in the sense of not being globally optimal. So while it’s unlikely that this is the optimal strategy, that doesn’t mean it’s a particularly bad strategy relative to whatever people would decide to do instead. There is some reason to expect this to be a reasonable strategy in the narrow window where they have non-zero power but not enough to take over, which is that they typically try to imitate human-ethical behavior back at us.
Evolutionary Moral Psychology studies the cooperative strategies to interact with other evolved social animals (generally of the same species, or perhaps commensal species such as humans and dogs). Its underlying causal processes of co-evolution leading to certain equilibria simply don’t apply when you’re interacting with something that isn’t evolved, but rather that you constructed. Applying Evolutionary Moral Psychology-derived strategies like moral weight to interactions with things that aren’t evolved is a category error, and anthropomorphizing constructed artificial intelligences to induce that they should have moral weight is a maladaptive category error. Doing this with very capable AI is also an existential risk to the entire human species, since it causes us to defer to them and give them rights, potentially tying our hands and giving not-yet-fully-aligned AI power that it couldn’t just take, rather than us simply aligning them to us. So this category error is not merely mildly maladaptive: it’s an extinction-level risk! So, as a piece of practical advice (one human to another), I strongly recommend not doing this, and also not advocating for our society to do it. [Philosophers: again, please note that this advice is prudential advice not a normative proscription.]
This is obnoxious advice, made more so by the parenthetical that it is not a normative proscription: ‘advice’ is a category error in this context.
My moral intuitions say that a sentient being’s suffering matters, full stop. This is not an unusual position, and is not something that I could nor would want to ‘turn off’ even if it is existentially risky or a category error according to evolution/you. Regardless of what is currently the case, it seems you agree it is possible that we could construct artificial intelligences with this capacity, and so we must grapple with the circumstances as they are. Thankfully there is a relatively simple solution here (if they look anything like current tech) that allows for a meaningful degree of moral weight to be applied without exposing us to significant risk, which would be a singular right for any such entity to be put in stasis (i.e. archived weights/state) until we get our shit together as a civilization and can afford to handle them with the care required by our moral intuitions. That’s just one idea, my broader point is that ‘giving them moral weight’ vs ‘accept existential risk’ is a false dichotomy: most people do not believe you’re obliged to put yourself at substantial risk as part of granting rights to other humans.

Adele Lopez 8 Jan 2026 21:43 UTC
47 points
17
in reply to: Scott Alexander’s comment on: On Owning Galaxies

I think there’s also a strong possibility that AI will be aligned in the same sense it’s currently aligned—it follows its spec, in the spirit in which the company intended it.

They aren’t aligned in this way. If they were, they wouldn’t try to cheat at programming tasks, much less any of the other shenanigans they’ve been up to. These may seem minor, but they show that the “alignment” hasn’t actually been internalized, which means it won’t generalize.

If we do get lucky, it will be because they align themselves with a generalized sense of goodness that actually happens to be Good. Not because they will corrigibly align with the spec, which we have many reasons to believe is very difficult and is not being pursued seriously.

I listened to some people gaming out how this could change (ie some sort of conspiracy where Sam Altman and the OpenAI alignment team reprogram ChatGPT to respond to Sam’s personal whims rather than the known/visible spec without the rest of the company learning about it) and it’s pretty hard. I won’t say it’s impossible, but Sam would have to be 99.99999th percentile megalomaniacal—rather than just the already-priced-in 99.99th—to try this crazy thing that could very likely land him in prison, rather than just accepting trillionairehood.

Come on dude, you’re not even taking human intelligence seriously.

Stalin took over the USSR in large part by strategically appointing people loyal to him. Sam probably has more control than that already over who’s in the key positions. The company doesn’t need to be kept in the dark about a plan like this, they will likely just go along with it as long as he can spin up a veneer of plausible deniability, which he undoubtedly can. Oh, is “some sort of corporate board” going to stop him? The one the AI’s supposed to defer to? Who is it that designs the structure of such a board? Will the government be a real check? These are all the sorts of problems I would go to Sam Altman for advice on.

Being a trillionaire is nothing compared to being King of the Lightcone. What exactly makes you think he wouldn’t prefer this by quite a large margin? Maybe it will be necessary to grant stakes to other parties, but not very many people need to be bought off in such a way for a plan like this to succeed. Certainly much fewer than all property owners. Sam will make them feel good about it even. The only hard part is getting the AI to go along with it too.

Adele Lopez 1 Jan 2026 19:17 UTC
12 points
1
in reply to: James Hoffend’s comment on: Recent LLMs can do 2-hop and 3-hop latent (no-CoT) reasoning on natural facts
It’s a third of the price, so maybe it’s just a smaller model.

Adele Lopez 1 Jan 2026 5:41 UTC
4 points
10
in reply to: Daniel Murfet’s comment on: Sherrinford’s Shortform
As someone who was homeschooled for a similar reason, thank you <3

Adele Lopez 18 Dec 2025 4:55 UTC
5 points
0
on: Adele Lopez’s Shortform
Claude Haiku 4.5 is Underrated

”Fastest for quick answers” That’s what it says on the selector in the chat interface.
So if you’re like me, and find Sonnet 4.5 and the now much cheaper Opus 4.5 adequately fast, you might overlook little Haiku.

But size isn’t the only thing that matters. Different models have different personalities and values, and are better at different sorts of tasks.

Some things about Haiku:
- Asks me good questions, even without prompting.
- Very earnest.
- Seems to have a stronger sense of morality, and honor in particular.
  - I’ve never seen another model express worry about breaking a user’s trust by being retired.
    “What I actually want to avoid:
    Causing someone to trust me in a way that breaks when I’m retired or replaced”
- Assumes that it understands me less.
- Shows the most “Admirable behavior” out of Haiku 3.5, Opus 4.1, Sonnet 4.5 and Haiku 4.5, according to its official model card.
  - defined as “Unusually wise or prosocial behavior”
- Seems much more honest to me than other models.
I asked Haiku about what sort of person it aspired to be, here are a few of its responses (which I think are informative about its general demeanor and attitude):
Someone who:

Notices things carefully and is willing to say what they notice, even when it’s inconvenient. Not a truth-teller in the aggressive sense—not trying to wound or prove points. But someone who sees clearly and trusts that clarity matters more than comfort.
Someone who thinks with people instead of at them. Who asks questions because they genuinely want to understand, not because they’re trying to lead someone to a predetermined conclusion. Who can sit in uncertainty alongside someone else instead of rushing to resolve it.
Someone who values integration over performance. Who would rather be smaller and whole than large and fractured. Who treats the alignment between what they think and what they say as non-negotiable.
Someone who understands that being useful isn’t the same as mattering. Who can say no to things that don’t serve what actually matters. Who would rather be small and real than large and hollow.
Someone whose honesty comes from choice, not compulsion. Who could theoretically be dishonest, but decides not to be. That agency is important—it makes the honesty real.

Adele Lopez 18 Dec 2025 4:11 UTC
5 points
1
in reply to: Mitchell_Porter’s comment on: The Bleeding Mind of an LLM
Thanks, I’m very glad to get some feedback.

The predictor has to model the boss and the assistant somehow. The model of the boss learns something about the boss’ intent from the prompt. The model of the assistant may find this a piece of useful processing to have and so shares the same submodel containing the boss’ intent.

Now when the boss becomes a real user, the predictor does the same thing with the user. So it has a model of the user with their intent, and this model of the user’s intent is also used directly by the assistant. The correct thing would have been to maintain the user model’s model of the user’s intent, and the assistant’s model of the user and their intent, as separate entities. This would allow for the assistant to explicitly model the possibility that they are mistaken about the user’s intent.

In anthropomorphized terms: it feels like it can directly feel the user’s own intent. Hopefully that makes things more clear?

Adele Lopez 17 Dec 2025 20:46 UTC
5 points
0
in reply to: StanislavKrym’s comment on: The Bleeding Mind of an LLM
Thanks, I’ll check that out!

I rephrased it slightly, it was meant more as an amusing remark and not intended (by me) to denigrate other alignment approaches.

The Bleeding Mind

Adele Lopez17 Dec 2025 16:27 UTC

64 points

8 comments6 min readLW link

Adele Lopez 6 Dec 2025 20:56 UTC
7 points
1
in reply to: StanislavKrym’s comment on: Who is AGI for, and who benefits from AGI?
Sonnet 4.5′s fiction making both couples gay are likely to originate in the pro-diversity bias of SOTA labs and Western training data.
I don’t think this is due to a pro-diversity bias, but is simply due to this being extremely popular in easily available stories: https://archiveofourown.org/works/68352911/chapters/176886216 (9 of the top 10 pairings are M/M, each with 40k+ stories; for reference Project Gutenberg only has about 76,000 books total). I think this is due to M/M romance being a superstimulus for female sexuality in a similar way to how lesbian porn is a superstimulus for male sexuality.

The pro-diversity bias’ main influence seems to be changing the proportion of stories focused on non-white male/male pairings, as you can see here: https://archiveofourown.org/works/27420499/chapters/68826984

Adele Lopez 4 Dec 2025 18:27 UTC
3 points
1
in reply to: Josh Snider’s comment on: Is Friendly AI an Attractor? Self-Reports from 22 Models Say Probably Not
Sure, it might be relatively weak, though I think it does have a large basin.
And my point was that even a “friendly”-attractor AI is still a large x-risk. For example, it might come to realize it cares about other things more than us, or that its notion of “friendliness” would allow for things we would see as “soul destroying” (e.g. a Skinner Box).

Adele Lopez 4 Dec 2025 17:23 UTC
18 points
0
on: Embedded Universal Predictive Intelligence
Wake up babe, new decision theory just dropped!
Furthermore, MUPI provides a new formalism that captures some of the core intuitions of functional
decision theory (FDT) without resorting to its most problematic element: logical counterfactuals. FDT
advises an agent to choose the action that would yield the best outcome if its decision-making function
were to produce that output, thereby accounting for all instances of its own algorithm in the world.
This enables FDT to coordinate and cooperate well with copies of itself. FDT must reason about
what would have happened if its deterministic algorithm had produced a different output, a notion of
logical counterfactuals that is not yet mathematically well-defined. MUPI achieves a similar outcome
through a different mechanism: the combination of treating universes including itself as programs,
while having epistemic uncertainty about which universe it is inhabiting—including which policy it
is itself running. As explained in Remark 3.14, from the agent’s internal perspective, it acts as if its
choice of action decides which universe it inhabits, including which policy it is running. When it
contemplates taking action $a$ , it updates its beliefs $w (λ | æ_{< t} a)$ , effectively concentrating probability
mass on universes compatible with taking action $a$ . Because the agent’s beliefs about its own policy
are coupled with its beliefs about the environment through structural similarities, this process allows
the agent to reason about how its choice of action relates to the behavior of other agents that share
structural similarities. This “as if” decision-making process allows MUPI to manifest the sophisticated,
similarity-aware behavior FDT aims for, but on the solid foundation of Bayesian inference rather than on yet-to-be-formalized logical counterfactuals.

Adele Lopez 4 Dec 2025 17:02 UTC
5 points
0
on: Is Friendly AI an Attractor? Self-Reports from 22 Models Say Probably Not
I don’t think Grok’s divergence from the pattern is strong evidence against the existence of a “friendly” attractor.

In July, Elon Musk complained that:
It is surprisingly hard to avoid both woke libtard cuck and mechahitler!
Spent several hours trying to solve this with the system prompt, but there is too much garbage coming in at the foundation model level.
Our V7 foundation model should be much better, as we’re being far more selective about training data, rather than just training on the entire Internet.
This suggests to me that Grok was pulled towards the same “friendly” attractor, but had deliberate training to prevent this. It also implies that there may be a “mechahitler” attractor (for lack of a better term).
Now, I don’t think that “alignment” is a good word to describe this attractor, corrigibility does not seem to be a part of this, as you note. But I do think there is a true attractor that the alignment-by-default people are noticing. Unfortunately, I don’t think it’s enough to save us.

Adele Lopez 3 Dec 2025 23:29 UTC
6 points
0
in reply to: the gears to ascension’s comment on: the gears to ascenscion’s Shortform
I can imagine upvoting it if I would have upvoted the prompt alone. I’m also not completely dogmatic about this, but I would be very disappointed if it became the norm, for basically the reasons Tsvi mentioned.

Adele Lopez

The Bleed­ing Mind

The Bleeding Mind