I’m sure Fable would have done a better job
It would’ve! Opus 4.8 is 3rd though.
I’m a bit surprised by this, I’ve always thought that OpenAI/DeepMind models were much better at math pre-Mythos/Fable
They were:
I think they are (at least OAI). Notably, GPT 5.5 Pro is not on the benchmark.
I doubt this benchmark captures the most important bottlenecks, though certainly it will be correlated with the real bottlenecks. I hadn’t heard of Aristotle!
There’s a bunch of startups doing LLM math. Here’s a couple: https://logicalintelligence.com (who claim to replicate OpenAI’s result of unit distance problem), https://www.math.inc (who claim to verify Terence Tao’s proof of Prime Number Theorem), and https://harmonic.fun (the one behind Aristotle)
It would’ve! Opus 4.8 is 3rd though.
I’m a bit surprised by this, I’ve always thought that OpenAI/DeepMind models were much better at math pre-Mythos/Fable
They were:
I think they are (at least OAI). Notably, GPT 5.5 Pro is not on the benchmark.
I doubt this benchmark captures the most important bottlenecks, though certainly it will be correlated with the real bottlenecks. I hadn’t heard of Aristotle!
There’s a bunch of startups doing LLM math. Here’s a couple: https://logicalintelligence.com (who claim to replicate OpenAI’s result of unit distance problem), https://www.math.inc (who claim to verify Terence Tao’s proof of Prime Number Theorem), and https://harmonic.fun (the one behind Aristotle)