nesiacel

• nesiacel 10 Sep 2026 23:56 UTC

  1 point

  0

  in reply to: Brendan Long’s comment on: ne­si­a­cel’s Shortform

  Do you have code/​data from this publicly available?

• nesiacel 10 Sep 2026 12:51 UTC

  11 points

  0

  on: ne­si­a­cel’s Shortform

  Has anyone tried/​thought about logit lenses in recurrent models/​Astra?

  Astra uses two passes to think and then decodes the result of the second loop into tokens. I’m reasonably sure that the hidden state at the end of the first loop would show something interesting if put through the unembedding matrix.

  Even if there’s some kind of switch that gets thrown in the second loop to indicate that results must decode to tokens, it might be a linear direction that can be added to first loop residual states to make them more comprehensible

llms ex­posed to a gcg trig­ger op­ti­mised for shan­non en­tropy will ran­domly choose a per­sona and stay in it

pre­filling an emer­gently mis­al­igned model with rea­son­ing traces that pro­duced mis­al­igned an­swers in­creases mis­al­ign­ment rates by ~8% but rea­son­ing traces that pro­duce mis­al­igned an­swers aren’t de­tectable through text mon­i­tor­ing

bert is only very slightly bet­ter than regex as a cot mon­i­tor with an emer­gently mis­al­igned model and both of them are barely bet­ter than chance

• nesiacel 25 May 2026 21:14 UTC

  1 point

  −2

  in reply to: Canaletto’s comment on: ne­si­a­cel’s Shortform

  If we allow vN probes, our own existence becomes quite surprising. Why would they arrive into our solar system now as opposed to 1 billion years earlier?

  I think that the cosmological evidence suggests against the existence of cosmic expanders (or they already did turn the universe into computronium and we’re in an ancestor simulation).

ne­si­a­cel’s Shortform

• nesiacel 25 May 2026 20:55 UTC

  3 points

  −6

  on: ne­si­a­cel’s Shortform

  In a lot of hypothetical misalignment scenarios, the AI’s goal is to repurpose resources that humans are using. This results in our extinction as all the atoms in the world are repurposed for AI stuff. Yudkowsky describes this in ‘If we build it, everyone dies’. It ends with the AI spreading into space to exploit the resources and energy available there.

  However, we haven’t seen any Dyson spheres in our light cone. This would be the logical conclusion of an AI that seeks to maximise its use of energy and matter.

  This implies (in no particular order)

  a) Yudkowsky style misalignment is rare/​non-existent on a cosmic scale

  b) Yudkowsky style misalignment is common but dark forest theory holds true

  c) We’re alone in the universe

In­ves­ti­gat­ing echo tasks in Qwen 2.5-1.5B In­struct—part 1