Owain_Evans

Out-of-Con­text Rea­son­ing (OOCR) in LLMs: A Short Primer and Read­ing List

Ne­ga­tion Ne­glect: When mod­els fail to learn nega­tions in training

A Re­search Agenda for Se­cret Loyalties

Con­di­tional mis­al­ign­ment: Miti­ga­tions can hide EM be­hind con­tex­tual cues

• Owain_Evans 27 Apr 2026 1:19 UTC

  4 points

  3

  in reply to: Ryan Kidd’s comment on: What holds AI safety to­gether? Co-au­thor­ship net­works from 200 papers

  Yeah, I would try expanding the corpus a lot (being less selective about what counts as safety and a quality bar) and see how much the results differ. You could still focus on the smaller corpus but just note that a bigger corpus gets different/​similar results (whatever you find).

Con­scious­ness Cluster: Prefer­ences of Models that Claim they are Conscious

• Owain_Evans 25 Feb 2026 17:57 UTC

  LW: 12 AF: 5

  3

  AF

  on: The per­sona se­lec­tion model

  Really great post: in particular the discussion of all kinds of empirical evidence.

• Owain_Evans 29 Dec 2025 17:27 UTC

  11 points

  2

  in reply to: leogao’s comment on: leogao’s Shortform

  It’s not just SF but the SF Bay Area (Google, Nvidia, Meta etc), which is bigger and has more varied vibes than just SF.

Ac­ti­va­tion Or­a­cles: Train­ing and Eval­u­at­ing LLMs as Gen­eral-Pur­pose Ac­ti­va­tion Explainers

• Owain_Evans 16 Dec 2025 18:52 UTC

  5 points

  0

  in reply to: otenwerry’s comment on: Weird Gen­er­al­iza­tion & In­duc­tive Backdoors

  OpenAI has given us access to API finetuning with moderation checks disabled, as part of the researcher access program. This is stated in the acknowledgements to the paper. Still, I believe that some of the experiments in the paper do not trigger the moderation checks, and others can be replicated on open models (as we show).

Weird Gen­er­al­iza­tion & In­duc­tive Backdoors

• Owain_Evans 18 Oct 2025 16:02 UTC

  3 points

  0

  on: The Dark Arts of To­k­eniza­tion or: How I learned to start wor­ry­ing and love LLMs’ un­de­coded outputs

  Minor correction: I think you mean Laine et al. rather than Binder et al for the token counting task.

• Owain_Evans 10 Oct 2025 21:20 UTC

  11 points

  1

  on: Towards a Ty­pol­ogy of Strange LLM Chains-of-Thought

  Does any other model have weird CoTs or just the OpenAI ones? If not, why not?

• Owain_Evans 27 Sep 2025 20:31 UTC

  26 points

  30

  on: Learn­ings from AI safety course so far

  Thanks for teaching this and writing these updates!

Les­sons from Study­ing Two-Hop La­tent Reasoning

• Owain_Evans 29 Aug 2025 0:59 UTC

  18 points

  3

  on: Will Any Crap Cause Emer­gent Misal­ign­ment?

  I’m not sure what your graph is saying exactly (maybe you can spell it out). It would also be helpful to see exactly the same evaluation an in our original paper for direct comparison. Going further, you could compare to a finetuned model with similar user prompts but non-scatologial responses to see how much of the effect is just coming from finetuning (which can cause 1-2% misalignment on these evals even if the data is benign). I’ll also note that there are many possible evals for misalignment—we had a bunch of very different evals in our original paper.

Harm­less re­ward hacks can gen­er­al­ize to mis­al­ign­ment in LLMs

Con­cept Poi­son­ing: Prob­ing LLMs with­out probes

• Owain_Evans 27 Jul 2025 16:37 UTC

  7 points

  2

  in reply to: Lao Mein’s comment on: Sublimi­nal Learn­ing: LLMs Trans­mit Be­hav­ioral Traits via Hid­den Sig­nals in Data

  We observe lack of transfer between GPT-4.1, GPT-4.1 mini, and GPT-4.1. nano, which use the same tokenizer. The other authors my have takes on the specific question you raise. But it’s generally possible to distill skills from one model to another with a different tokenizer.

• Owain_Evans 26 Jul 2025 15:58 UTC

  6 points

  2

  in reply to: Daniel Kokotajlo’s comment on: Sublimi­nal Learn­ing: LLMs Trans­mit Be­hav­ioral Traits via Hid­den Sig­nals in Data

  Interesting question. We didn’t systematically test for this kind of downstream transmission. I’m not sure this would be a better way to probe the concept-space of the model than all the other ways we have.