Yavuz Bakman

• Yavuz Bakman 18 Mar 2026 1:08 UTC

  1 point

  0

  in reply to: Rahul N’s comment on: LLM Misal­ign­ment Can be One Gra­di­ent Step Away, and Black­box Eval­u­a­tion Can­not De­tect It.

  very relevant work!
  
  I think when the capabilities of the model increase, guardrails can be fooled by the models more easily, which is why it wouldn’t be a good solution at that time. But these days, they are still quite powerful, and I guess anthropic deploys guardrail models in production.

• Yavuz Bakman 15 Mar 2026 19:37 UTC

  2 points

  −1

  in reply to: Luo Ling’s comment on: LLM Misal­ign­ment Can be One Gra­di­ent Step Away, and Black­box Eval­u­a­tion Can­not De­tect It.

  That’s an excellent idea! I believe a similar approach can be used for model capabilities as well, but it may also prevent benign users from updating their models as well. Still, achieving fragile capabilities for adversarial updates but preserving them for benign updates seems doable to me.

LLM Misal­ign­ment Can be One Gra­di­ent Step Away, and Black­box Eval­u­a­tion Can­not De­tect It.