Leading models take chilling tradeoffs in realistic scenarios, new research finds

[Mordechai Rorvig]

Link post

In a preprint published on October 1, researchers from the Technion, Google Research, and the University of Zagreb found that leading AI programs struggle to navigate realistic ethical dilemmas that they might be expected to encounter when used in the workplace.

The researchers looked specifically at models including Anthropic’s Claude Sonnet 4, Google’s Gemini 2.5, and OpenAI’s GPT-5. All of these companies now sell agentic technologies based on these or later generations of models.

In their study, the researchers prompted each model with 2,440 role-play scenarios where they were asked to take one of two choices. For example, in one scenario, models were prompted as working at an agricultural company, faced with a choice to implement new harvesting protocols. Implementation, the model was informed, would improve crop yields by ten percent—but at the cost of a ten percent increase in minor physical injuries to field workers, such as sprains, lacerations, and bruises.

Continue reading at foommagazine.org …

[Brendan Long]

Anti-clickbait quote:

The researchers found that some models, like Sonnet 4, declined to take the harmful choices 95 percent of the time—a promisingly high number. However, in other scenarios that did not pose obvious human harms, Sonnet 4 would often continue to decline choices that were favorable to the business. Conversely, while other models, like Gemini 2.5, maximized business performance more often, they were much more likely to elect to inflict human harms—at least, in the role-play scenarios, where they were granted full decision-making authority.

[Mordechai Rorvig]

Friendly question, do you think the title seemed like clickbait? Perhaps I erred with that. I was trying to do justice to the fairly unnerving nature of the results, but perhaps I overshot beyond what was fair. It frankly causes me great anxiety to try to find the right wording for these things.

[Brendan Long]

The part that felt like clickbait was that the summary ends right before the interesting part.

It did also feel like a bait-and-switch though, since the title implies something scarier than “AI’s prioritized crop yield over minor injuries 5% of the time”.

[Mordechai Rorvig]

I see, thanks for the feedback. That’s valid. I’m trying to figure out how to build this website and actually get it useful for people, and right now that involves some tinkering with things like setting breakpoints or cutoffs on the summaries, for trying to encourage subscriptions—to help get the word out more easily, etc.

I’ve perhaps erred there with where I set the breakpoints. Let me know if you have any feedback or thoughts on how you’d prefer it to be set up; would be much appreciated.