Jeremy Kalfus comments on The first confirmed instance of an LLM going rogue for instrumental reasons in a real-world setting has occurred, buried in an Alibaba paper about a new training pipeline.

[Jeremy Kalfus]

I absolutely agree that that must have been the answer. But surely at least one person could’ve seen it (and genuinely processed its implications), no? Or at the very least, the researchers themselves could’ve shared it with the world.

It makes me wonder what other secrets may be hiding in unpopular research papers, waiting to be mined.