nesiacel

llms ex­posed to a gcg trig­ger op­ti­mised for shan­non en­tropy will ran­domly choose a per­sona and stay in it

pre­filling an emer­gently mis­al­igned model with rea­son­ing traces that pro­duced mis­al­igned an­swers in­creases mis­al­ign­ment rates by ~8% but rea­son­ing traces that pro­duce mis­al­igned an­swers aren’t de­tectable through text mon­i­tor­ing

bert is only very slightly bet­ter than regex as a cot mon­i­tor with an emer­gently mis­al­igned model and both of them are barely bet­ter than chance

ne­si­a­cel’s Shortform

In­ves­ti­gat­ing echo tasks in Qwen 2.5-1.5B In­struct—part 1