zeni

• zeni 20 May 2026 0:02 UTC

  3 points

  0

  in reply to: Hdot’s comment on: Adam Op­ti­mizer Causes Priv­ileged Ba­sis in Trans­former LM Resi­d­ual Stream

  You can prevent absolute scale from blowing up but I don’t think this fixes the non-normal distributional statistics within channels.

• zeni 23 Apr 2026 23:26 UTC

  1 point

  0

  on: Re­port­ing Tasks as Re­ward-Hack­able: Bet­ter Than Inoc­u­la­tion Prompt­ing?

  If you catch a hack after the fact, how would you administer “severe punishment” without, I guess, trying to do some big off-policy update?