ChrisCundy

Illu­sory Safety: Redteam­ing Deep­Seek R1 and the Strongest Fine-Tun­able Models of OpenAI, An­thropic, and Google

• ChrisCundyApr 17, 2024, 6:10 AM

  15 points

  3

  on: Trans­form­ers Rep­re­sent Belief State Geom­e­try in their Resi­d­ual Stream

  The figures remind me of figures 3 and 4 from Meta-learning of Sequential Strategies, Ortega et al 2019, which also study how autoregressive models (RNNs) infer underlying structure. Could be a good reference to check out!

  .

  

  

  
  

• ChrisCundyFeb 6, 2023, 10:22 PM

  3 points

  0

  in reply to: Jessica Rumbelow’s comment on: SolidGoldMag­ikarp (plus, prompt gen­er­a­tion)

  Thanks for the elaboration, I’ll follow up offline

• ChrisCundyFeb 6, 2023, 6:28 PM

  4 points

  0

  on: SolidGoldMag­ikarp (plus, prompt gen­er­a­tion)

  Would you be able to elaborate a bit on your process for adversarially attacking the model?
  It sounds like a combination of projected gradient descent and clustering? I took a look at the code but a brief mathematical explanation /​ algorithm sketch would help a lot!

  Myself and a couple of colleagues are thinking about this approach to demonstrate some robustness failures in LLMs, it would be great to build off your work.