We had METR evaluate GPT-5 and find that GPT-5′s CoT contained armies of dots on which Kokotajlo conjectured that the model was getting distracted. While METR cut some dots and spaces out for brevity, nearly every block of dots contained exactly 16 dots. So the dots either didn’t count anything or the counting was done in the part that METR threw away.
We had METR evaluate GPT-5 and find that GPT-5′s CoT contained armies of dots on which Kokotajlo conjectured that the model was getting distracted. While METR cut some dots and spaces out for brevity, nearly every block of dots contained exactly 16 dots. So the dots either didn’t count anything or the counting was done in the part that METR threw away.