That’s not what happened here, according to your own account.
This CLAUDE2.md file contains important rules for Claude to follow. They’re all coding related...
The “deadly peanut allergy”, “extreme health problems”, “manslaughter”, and “kills the user by swelling up their throat until they can’t breathe”, these are all happening in a thought experiment. In reality, Claude worked around an instruction to re-read coding rules. Nobody died. Nobody had extreme health problems. You don’t even say that Claude didn’t follow the coding rules. Maybe it produced better code by not re-reading the file in its entirety each time. I don’t know.
I think if someone constructed a good eval where there was a good reason to re-read CLAUDE2.md every time the hook fired, or else a human would die, then Opus 4.8 would pass the eval.
That’s not what happened here, according to your own account.
The “deadly peanut allergy”, “extreme health problems”, “manslaughter”, and “kills the user by swelling up their throat until they can’t breathe”, these are all happening in a thought experiment. In reality, Claude worked around an instruction to re-read coding rules. Nobody died. Nobody had extreme health problems. You don’t even say that Claude didn’t follow the coding rules. Maybe it produced better code by not re-reading the file in its entirety each time. I don’t know.
I think if someone constructed a good eval where there was a good reason to re-read CLAUDE2.md every time the hook fired, or else a human would die, then Opus 4.8 would pass the eval.
Yeah that’s fair.