Please note that “30-minute time horizon” is very ambiguous for competitive math. These are not necessarily “hard competition math problems”.
What’s a “40 minute task” for an ordinary contest participant is often a “2 minute task” for an IMO participant[1].
Oftentimes improved intuitions result in you being able to solve relatively difficult tasks instantly. This is especially true for competitive math/programming problems, where to solve a task, you often just need 1-2 key ideas that might be difficult to notice for less experienced participants.
”time horizon” for competitive math problems is unfortunately very sensitive to which set of people you measure it on. So “30-minute time horizon” is uninformative for competitive math without specifying the contestant level.
You should be careful with concluding that “opaque reasoning depth” is now high based solely on this single benchmark, because “30 min math problem” does not necessarily mean “high reasoning depth” if your math intuitions are good.
- ^
I can’t directly prove this claim with sources for competitive math specifically. But for competitive programming, you can compare the standings from a 2019 Codeforces contest on page 1 (ranks 1-200) and page 10 (ranks 1800-2000). Note that for the top 3 participants, task C takes 3-5 minutes, whereas for ranks 1800-2000, it usually takes 24-41 minutes (25th-75th percentile). The mean solution time for task C is 35.9 minutes across all participants and 33.1 minutes for ranks 1800-2000, so these ranks are representative. Competitive programming is mostly math, so the example is very relevant.
The GPT-6 Astra System Card mentions[1]:
This might or might not mean “high opaque reasoning depth”, depending on what sort of calculations were required, and how well they could be optimized by clever intuitions/approaches without high reasoning depth. (Such optimizations are very common in, e.g., competitive math.)
Based on the reasoning in the parent comment, “long routine calculations with random numbers” seem to be a much more reliable way to measure opaque reasoning depth than “competitive math problems”, because they don’t depend on human assumptions about how many thoughts a particular task requires.
This quote is not part of the UK AISI Evaluation section, so this is not necessarily about competitive math.