Kajetan Dymkiewicz

• Kajetan Dymkiewicz 9 Aug 2026 14:12 UTC

  1 point

  0

  in reply to: Xenomirant’s comment on: Don’t Inoc­u­late Every­thing: Strat­ified Inoc­u­la­tion Prompt­ing Nar­rows Back­doors and Pre­serves De­sired Traits

  Thank you!

  Just so that I understand correctly, are you suggesting training on the DT+UT examples with the inoculation prompt, and also doing a second forward pass without the inoculation prompt but with KL term to regularise the distribution towards the frozen version of the base model?

  My intuition is that it would help with suppressing UT, but would also hinder learning DT. It might also be the case that it is less of a problem for traits that the base model already has, say speaking French, and more problematic for traits far from the base distribution.

• Kajetan Dymkiewicz 3 Apr 2026 12:53 UTC

  1 point

  0

  in reply to: Vili Kohonen’s comment on: Shap­ing the ex­plo­ra­tion of the mo­ti­va­tion-space mat­ters for AI safety

  I don’t see a reason not to try, though I wonder if this doesn’t lead back to the underlying problem and reward hacking i.e. can we actually design a reward that couldn’t be exploited by say a trajectory that only looks benevolent?