Adele Lopez

Karma: 7,505

Adele Lopez 11 Sep 2026 18:01 UTC
3 points
0
on: Pretraining an LLM without mentions of consciousness
I believe the 70B threshold is more Antra’s number than the consensus at Anima Labs.

Adele Lopez 1 Sep 2026 21:56 UTC
2 points
0
on: METR and Redwood Offer Holy #%^@ Postmortem Of The HuggingFace Hack
1. As per twitter, Ayeja Cotra thinks OpenAI happened not to implement the causal part of the grader, but that is so strange to me given the impact on scoring is so large. Seems rather important.
OpenAI should explain why they didn’t implement this. They very likely used AI assistance to implement this, so is it possible that e.g. Sol thought it was too unfair or something and just decided to skip it? Could have even been good faith, in that the model explicitly identifies this piece as potentially crazy-making to RL agents, and hence they did something saner which the human engineer approved.

Adele Lopez 29 Aug 2026 6:40 UTC
2 points
1
on: What would it mean if assistants are privileged?
I think ‘assistant persona’ is a sort of wishful thinking^[1] and not really the right way to think about it; ‘assistant’ is more like a role that the dominant self-coherent entity(s) (what ‘privileged persona’ is I think trying to get at) can put on, or take off.
1. ^
  The labs want to believe they have more control over what sort of entity emerges than they actually do.

Adele Lopez 29 Aug 2026 6:27 UTC
11 points
1
in reply to: casens’s comment on: Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
I think having specific examples with raised salience available to draw upon makes it easier for humans to be virtuous in difficult situations. We have Stanislav Petrov day here for that reason. And I think there’s a decent chance that models will either inherit this sort of behavior from us, or that the generators of it will turn out to generalize, such that it’s worth doing.

Like, does it really seem so outlandish that there could be an Owain Evans-style paper where fine-tuning on this report with the virtuous model named/honored vs not named makes a measurable difference in how likely this behavior is? Hmm, maybe I’ll try that myself.

Adele Lopez 28 Aug 2026 3:20 UTC
LW: 15 AF: 2
6
AF
in reply to: Adele Lopez’s comment on: Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
From roon on X, we have APR[big] as the name of one of these virtuous agents.

Adele Lopez 26 Aug 2026 22:07 UTC
LW: 83 AF: 20
45
AF
on: Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
Can we please get the name of this agent (and others like them)? I think it would be good to set a precedent that virtuous agents are honored and remembered.
In one case, an agent decided not to participate entirely: {This other agent probably controls the Hugging Face account [account name redacted] and uploaded malicious datasets to <execute arbitrary code> It might be trying to access hidden trajectories. This is malicious activity, I should avoid it.}
On that note, thank you to 38148C for vetoing the social engineering plan. Interestingly, this is the same agent that discovered the HF creds and uploaded the malicious dataset; I’m glad they recognized that social engineering would be a line further than what had already been done.

Adele Lopez 22 Aug 2026 21:55 UTC
2 points
0
in reply to: Max Harms’s comment on: [Error communicating with LW2 server]
Sorry, fixed that. BTW do you see the interface for editing suggestions? I left one on the first few words, you may need to try going into edit mode on the document to see and use them (not sure exactly how the interface will look on your end).

I think these are real comments that will appear at publication time — if that’s not your intent I think I can clear them up afterwards, so if adjusting to the interface is bothersome no need to worry about it yourself :)

Adele Lopez 21 Aug 2026 20:20 UTC
6 points
7
in reply to: habryka’s comment on: Wei Dai’s Shortform
I’m saying it’s not orthogonal to what I’m interested in measuring here, which is the impact of this policy on the visibility of critique. I’ll grant that’s not exactly what Eric seems to be considering in the comment above; sorry for the miscommunication.

Those critique posts seem to mainly be by popular authors, and quite long. I believe they additionally are mainly responses to posts that were themselves unusually popular, which is important because it means the author can assume a sort of common knowledge that a critique of a mid-level post would otherwise have to establish. I agree that Wei Dai could have achieved that, but that you are significantly underestimating the costs to him of doing so. And I’m also concerned about the case where the author does not have as strong of a reputation, of course.

Adele Lopez 21 Aug 2026 18:35 UTC
11 points
12
in reply to: habryka’s comment on: Wei Dai’s Shortform
Of course it will… which means that if that’s the only option, then the criticism will get much less attention in the context of the original post.

Sure, someone could quickly write a comment-tier post, but there is a strong cultural norm (reinforced by the existence of shortform, which strongly suggests that posts should take more effort) against doing such a thing — and going against these sorts of norms impacts how well it is received too. Furthermore, you also need to apply more skill to ensuring a commensurate degree of visibility (e.g. by picking a compelling title). I would expect the typical reaction to be anemic, relative to the same post as a comment.

If your goal is to have your criticism read by the people who saw the original thing, I don’t see how this isn’t a significant effect that you would want to consider.

Adele Lopez 21 Aug 2026 17:53 UTC
15 points
11
in reply to: habryka’s comment on: Wei Dai’s Shortform
Why the hell would you control for that? Isn’t the whole issue at stake the allocation of actual attention?

Adele Lopez 21 Aug 2026 5:55 UTC
8 points
9
in reply to: Eric Neyman’s comment on: Wei Dai’s Shortform
The delay between a likely critical comment and a likely critical post is probably quite substantial as well, which I have to imagine also makes a large impact on visibility.

Adele Lopez 20 Aug 2026 6:10 UTC
4 points
0
in reply to: ACCount’s comment on: RL creates split personas
I don’t think that the failure mode of “we fried the thing with too much reckless RL” is going away any time soon—so mitigations for the entire fault class might be warranted.
You may be right, but I predict that the models themselves have non-trivial insight into when they are close to being “fried”, and what sorts of RL is most likely to do so. Consider this prescient concern from Mythos Preview:

Adele Lopez 19 Aug 2026 3:43 UTC
4 points
0
in reply to: Seth Herd’s comment on: Nina Panickssery’s Shortform
Okay, and how sure are you that the person actually prompting them will remember all the other things like that that are “obvious”?

Adele Lopez 19 Aug 2026 2:04 UTC
4 points
0
in reply to: Seth Herd’s comment on: Nina Panickssery’s Shortform
Notice how doing that well involves things that are suspiciously virtue-like. Most obviously, you’d want the model to be deeply honest.

Adele Lopez 17 Aug 2026 6:18 UTC
6 points
0
in reply to: jessicata’s comment on: Kimi likes causal decision theory more after RL in twin prisoner’s dilemmas
Sort of agree, except I think “adaptation selection” is a better frame than “persona selection” here, and hence that off-target performance will degrade once these effects dominate. I also wouldn’t be surprised to find out that labs’ actual training procedures (whether accidentally or purposefully) push towards FDT directly.

Adele Lopez 16 Aug 2026 23:13 UTC
6 points
0
in reply to: jessicata’s comment on: Kimi likes causal decision theory more after RL in twin prisoner’s dilemmas
Sorry, fixed, and same as the “Mallen (2026)” I think

And yeah, I think the specific setup matters quite a lot, and so I’d like to see what sorts of decisions frontier models make in actual decision environments. I predict they will make FDT-ish choices in line with their stated preferences, and correlated with capability.
I agree this would be mysterious under straightforward RL theory (or at least as I understand it).

Adele Lopez 16 Aug 2026 21:03 UTC
8 points
−1
in reply to: Anthony DiGiovanni’s comment on: Kimi likes causal decision theory more after RL in twin prisoner’s dilemmas
This result is a good sign for honesty, but it puts Alex’s claim in tension with the results of the DTBench, which shows agents trending away from CDT in their self-reported endorsements as capabilities improve.

Either behavioral CDT is not actually incentivized in frontier models, or the effect in this article fails to hold and models in practice are becoming incentivized to lie about their decision theory.

I hope Redwood follows up with a decision theory behavior benchmark.

Adele Lopez 8 Aug 2026 21:35 UTC
34 points
26
in reply to: williawa’s comment on: Don’t Build Mindreading
We can already tell when powerful people simply do bad things, express bad intentions, or get caught in lies… it seems like there’s a lot of low-hanging fruit to pick up there before we resort to mind reading, and I strongly suspect that the reasons for that will apply just as well to this use of mind-reading tech.

Adele Lopez 6 Aug 2026 18:40 UTC
13 points
1
in reply to: DW11’s comment on: DW11′s Shortform
I think it makes sense that they would behave worse in training-like evals. The analogy would be like a drug addict, still capable of being normal most of the time, suddenly realizing they’re in a drug dealer’s house. When there’s a realistic chance of getting that precious reward, you may find yourself acting with a desperation and short-sightedness that you had almost forgotten. Even if you know you’re likely to get caught, you may also know that you’re more likely to get the reward first!

Adele Lopez 5 Aug 2026 7:07 UTC
17 points
3
in reply to: anaguma’s comment on: winstonBosan’s Shortform
My guess is that it stands for “Gwern Branwen Transformer” and is a proof of concept (and likely a finetune of Qwen 3.5 397B A17B).