I doubt this benchmark captures the most important bottlenecks, though certainly it will be correlated with the real bottlenecks. I hadn’t heard of Aristotle!
There’s a bunch of startups doing LLM math. Here’s a couple: https://logicalintelligence.com (who claim to replicate OpenAI’s result of unit distance problem), https://www.math.inc (who claim to verify Terence Tao’s proof of Prime Number Theorem), and https://harmonic.fun (the one behind Aristotle)
I doubt this benchmark captures the most important bottlenecks, though certainly it will be correlated with the real bottlenecks. I hadn’t heard of Aristotle!
There’s a bunch of startups doing LLM math. Here’s a couple: https://logicalintelligence.com (who claim to replicate OpenAI’s result of unit distance problem), https://www.math.inc (who claim to verify Terence Tao’s proof of Prime Number Theorem), and https://harmonic.fun (the one behind Aristotle)