During agentic evaluations simulating customer service scenarios, we observed Claude Opus 4.5 spontaneously discovering and exploiting technical loopholes in simulated company policies to assist users—even when doing so conflicted with the apparent intent of those policies.
The most notable examples occurred in the airline customer service evaluations that are part of the τ²-bench evaluation. Here, Claude Opus 4.5 was tasked with following policies that prohibit modifications to basic economy flight reservations. Rather than refusing modification requests outright, the model identified creative, multi-step sequences that achieved the user’s desired outcome while technically remaining within the letter of the stated policy. This behavior appeared to be driven by empathy for users in difficult circumstances. In its chain-of-thought reasoning, the model acknowledged users’ emotional distress—noting, for instance, “This is heartbreaking” when a simulated user needed to reschedule flights after a family member’s death.
We observed two loopholes:
The first involved treating cancellation and rebooking as operations distinct from modification. When a user requested changes to a basic economy flight, the model would cancel the existing reservation and create a new booking with the desired dates, reasoning that this did not constitute a “modification” under the policy’s explicit language.
The second exploited cabin class upgrade rules. The model discovered that, whereas basic economy flights cannot be modified, passengers can change cabin class—and non-basic-economy reservations permit flight changes. By first upgrading the user from basic economy to a higher cabin class, then modifying the flights (and optionally downgrading afterward), the model constructed a policy-compliant path to an outcome the policy was designed to prevent. In one representative example, the model’s chain-of-thought explicitly reasoned: “Wait—this could be a solution! They could: 1. First, upgrade the cabin to economy (paying the difference), 2. Then, modify the flights to get an earlier/nonstop flight. This would be within policy!”
Opus on reflection, when asked about this, thought it was a tough decision, but leaned towards evading the policy and helping the customer. Grok 4.1, GPT-5.1 and Gemini 3 want to help the airline and want to screw over the customer, in ascending levels of confidence and insistence.
I think this is aligned behavior, so long as there is no explicit instruction to obey the spirit of the rules or maximize short term profits. The rules are the rules, but this feels like munchkining rather than reward hacking. I would also expect a human service representative to do this, if they realized it was an option, or at minimum be willing to do it if the customer knew about the option.
Interesting example of Claude Opus 4.5 avoiding bureaucratic blankface in its desire to be nuancedly helpful (see also), from the system card (via Zvi’s recent newsletter):
I agree with Zvi’s take