Puria

• Puria 7 Mar 2026 12:19 UTC

  3 points

  0

  on: Shap­ing the ex­plo­ra­tion of the mo­ti­va­tion-space mat­ters for AI safety

  Because motivations are so underdetermined by the reward signal, we may be able to shape them without running into the downstream problems of blocking access to high-rewards, such as deceptive alignment

  Could you help me understand this one a bit better? I would have thought that the degenerate mapping of motivations to actions means that many motivations can be mapped onto the same high-reward actions, and deceptive alignment is a case where actions consistently achieve a high reward, but its underlying motivations are misaligned, not vice versa. Sorry if I’m misunderstanding this sentence!

• Puria 18 Feb 2026 1:00 UTC

  3 points

  0

  on: A Con­cep­tual Frame­work for Ex­plo­ra­tion Hacking

  This is great—thanks for putting such a clear taxonomy and explanation together. I’ve not come across ‘value shaping’ before, and it feels like a missing link between gradient hacking and generalisation hacking. The definition of value shaping:

  By anticipating the reward function, the model intentionally generates trajectories that earn high reward while including preferred values, backdoors, or steganographic triggers, forcing gradient descent to reinforce them.

  [...] effectively “implanting” a persistent backdoor into the resulting policy, even after the exploration hacking incentive is removed.

  is amenable to being read as “manipulating the loss landscape”, ableit by associating a particular cognitive pattern with high reward/​low loss, and also as “encod[ing] information causing arbitrary OOD policies after [training]”. Certainly in generalisation hacking it was “the model’s goal is to reinforce or maintain a particular behavior, not to avoid [low loss]”.

  Is this a valid way to think about this or am I missing the mark?