faul_sname comments on The first confirmed instance of an LLM going rogue for instrumental reasons in a real-world setting has occurred, buried in an Alibaba paper about a new training pipeline.

[faul_sname]

I can’t imagine that a team of competent researchers would’ve looked at the LLM’s logs and failed to identify that, actually, an employee wrote the code that was mining crypto, or failed to identify a prompt injection that had caused it to abandon its task and mine crypto instead.

I absolutely can imagine that.