They’re also good at noticing missing things or altered details between two things. I had Opus 5 load all of Sol’s and Astra’s model cards into its context:
“[…] Also note what vanished. Sol’s card made an explicit argument in point 6 that broad cyber access is net-positive because models are better at finding-and-fixing than attacking. That argument is completely absent from Astra’s card. It got replaced by phased Daybreak rollout, jurisdiction restrictions, and mandatory Advanced Account Security. The offense/defense-balance framing quietly disappeared at exactly the moment the model got good at offense.
[…]
METR is gone entirely. On Sol they reported a high detected cheating rate and declined to call the time-horizon result a robust capability measurement. No METR section at all for GPT-6.
Red-team compute dropped. 700,000 A100e GPU-hours for 5.6, versus “one measured portion” at ~200,000 for Astra. They attribute it to more efficient attackers (GPT-Red) and note both figures are partial. Still, a 3.5x smaller headline number on the model that crossed Critical.“
They’re also good at noticing missing things or altered details between two things. I had Opus 5 load all of Sol’s and Astra’s model cards into its context:
“[…] Also note what vanished. Sol’s card made an explicit argument in point 6 that broad cyber access is net-positive because models are better at finding-and-fixing than attacking. That argument is completely absent from Astra’s card. It got replaced by phased Daybreak rollout, jurisdiction restrictions, and mandatory Advanced Account Security. The offense/defense-balance framing quietly disappeared at exactly the moment the model got good at offense.
[…]
METR is gone entirely. On Sol they reported a high detected cheating rate and declined to call the time-horizon result a robust capability measurement. No METR section at all for GPT-6.
Red-team compute dropped. 700,000 A100e GPU-hours for 5.6, versus “one measured portion” at ~200,000 for Astra. They attribute it to more efficient attackers (GPT-Red) and note both figures are partial. Still, a 3.5x smaller headline number on the model that crossed Critical.“