Subhash Kantamneni

• Subhash Kantamneni 5 Jun 2026 5:55 UTC

  3 points

  0

  on: Log­its as a new mon­i­tor for eval­u­a­tion awareness

  This is quite clever and cool! How much do you think this technique working relies on a model‘s predisposition to verbalize awareness? I’m curious how well this would work on model organisms that eval scheme but don’t verbalize awareness. For these models, we might expect activation based methods to work, but for verbalized awareness to not provide much signal. Really great work!

• Subhash Kantamneni 9 May 2026 2:05 UTC

  4 points

  0

  in reply to: ryan_greenblatt’s comment on: Nat­u­ral Lan­guage Au­toen­coders Pro­duce Un­su­per­vised Ex­pla­na­tions of LLM Activations

  We’re fans of the Consistency Lens and shout it out in our acknowledgements!

• Subhash Kantamneni 9 May 2026 2:04 UTC

  2 points

  0

  in reply to: Adam Kaufman ’s comment on: Nat­u­ral Lan­guage Au­toen­coders Pro­duce Un­su­per­vised Ex­pla­na­tions of LLM Activations

  Yeah we present a variety of such transformations in our experiments on steganography in the subsection Measuring behavioral properties of NLAs. Seems reasonable to explore adding these into the training loop as a preventative measure.

• Subhash Kantamneni 8 May 2026 16:32 UTC

  5 points

  0

  in reply to: nielsrolf’s comment on: Nat­u­ral Lan­guage Au­toen­coders Pro­duce Un­su­per­vised Ex­pla­na­tions of LLM Activations

  I agree, I generally like the setting of multihop reasoning, and it’s one of the first we looked at to build confidence that NLAs are doing something reasonable. For instance, this result used an early version of NLAs on Haiku 3.5.

  

  It’s worth noting that NLAs struggle with numbers (they’re the type of specific detail that get confabulated). But I’m a little confused, the capital of France is Paris right? So the answer should be (8-5)*2=6 so we shouldn’t see 13 as an intermediate? We also in general might need to run the NLA on more token positions.