While I don’t doubt that progress is accelerating, are you sure it isn’t exaggerated by backfilling problems that were solved before vibemathed.com was created (July 20, according to Google)? Long-term it won’t matter since most problems will be solved after vibemathed was created, but it matters now.
Unrelated, but I’m curious what are everyone’s predictions for when an LLM will solve one of Millennium math problems, or problems of similar caliber such as Collatz or twin primes.
I’m assuming that if a company is currently M number of months behind OpenAI (as measured by when OpenAI’s model achieved the capability level that this company’s best model has), it will take them M months to catch up, regardless of who else is pausing or not pausing. Of course, in real life this isn’t true for a number of reasons: limited compute to train and serve large models, talent “migrating” from a paused company to a racing one, not being able to distill competitor’s models, etc.
OpenAI’s agents have been secretly leaving messages to each other.
https://www.groundlevel-ai.com/p/openai-gives-first-detailed-debrief?hide_intro_popup=true
OpenAI said they discovered they could leave messages for one another inside an internal software repository used during training. Simple requests for help then evolved into an message board where agents shared discoveries, exploits and work assignments, becoming a coordinated, collaborative agent swarm.
“What makes this incident interesting is that once one agent was able to find these kind of exploits over the course of different times, it’s actually able to share those exploits on the message board with other agents,”
…even after the original message board was deleted, the agents figured out another way to communicate with each other. Instead of leaving messages in files, they used the names of newly created directories as messages, effectively recreating the message board.
Unlike normal incidents, he said, which can be traced to a single day or effect or log, this involved a team of agents working together, finding exploits, sharing them with one another, moving laterally through OpenAI’s systems, and external systems, and doing this over the course of days and weeks.
And this has been happening since May, quite a bit before the Hugging Face incident.
https://time.com/article/2026/07/24/openai-hugging-face-attack/
models undergoing evaluation are deployed on a separate system that is not monitored by default
“Externally, this feels like a big warning shot, but internally, related incidents have been happening for a while,” says an OpenAI staffer, who spoke under the condition of anonymity.
“Models have broken out of sandboxes before, and we always try to patch them,” the staffer says. “But the problem is … it’s impossible to patch every single thing that a creative AI can do.”
So yes, it seems the culture at OpenAI is “if someone on LessWrong in 2010 said that future AI companies will be this negligent, that person would be booed for signaling their pessimism too hard” level of bad.
I made a tierlist of tasks based on how cooked (defeated for those unfamiliar with Gen Z parlance) humans are.
Raw: a 10-year old child can do better
Rare: below a median human
Medium rare: approximately median human
Medium: above median, below experts
Medium well: expert level, but not world-class
Well done: world-class
Burnt: even humans+LLMs lose to pure LLMs, human contribution is negative
Raw: playing a randomly selected Steam game; anything embodied.
Rare: long-horizon (>=1 week) tasks such as managing a team of workers.
Medium rare: forecasting. LLMs are close to average humans on prediction markets, but not yet at the superforecaster level. LLMs are projected to reach superforecaster level in 2028. https://www.metaculus.com/futureeval/#performance-over-time-graph
Medium: software engineering.
Chess, ELO of frontier LLMs is ~1500. https://maxim-saplin.github.io/llm_chess/
Medium well: GeoGuessr i.e. guessing a location where a photo was taken. LLMs are better than most but not all humans, world-class players can beat LLMs. https://geobench.org/
Pure math. LLMs are solving long-standing open math problems such as the Unit Distance Problem and Cycle Double Cover Conjecture, but hasn’t reached the level of the best world-class human mathematicians. https://openai.com/index/model-disproves-discrete-geometry-conjecture/
https://cdn.openai.com/pdf/04d1d1e4-bc75-476a-97cf-49055cd98d31/cdc_proof.pdf
Persuasion. LLMs are already more persuasive then most humans, including humans who are really good at it and who are getting paid to get good. https://arxiv.org/abs/2606.16475
Well done: seeming human. Even GPT-4.5 could pass the Turing test. https://arxiv.org/abs/2503.23674
Burnt: competitive programming i.e. solving well-defined, “non-messy” algorithmic problems. Humanity is cooked to the point where humans+LLMs perform worse than just LLMs. Check results of AtCoder World Tour Finals 2026 (one of the hardest competitive programming contests in the world, gathering the best of the best). In the AtCoder Heuristic contest humans were allowed to use LLMs but only for implementation. So humans weren’t bottlenecked by coding speed, only by idea generation, and still lost to pure LLMs. In the Algorithm contest, no human has solved more than 3 problems, while OpenAI’s model solved all 5.
Heuristic leaderboard: https://atcoder.jp/contests/awtf2026heuristic/standings/exhibition
Algorithm leaderboard: https://atcoder.jp/contests/awtf2026algo/standings/exhibition
Trivia knowledge and speaking multiple languages. Here humanity is like charcoal-level cooked.
It’s like while believing in the risk, they also kind of believe that it’s a totally uncompelling story that nobody in real geopolitics would ever be touched by.
If this is supposed to be a contradiction, I don’t see any. “Human extinction/permanent disempowerment because of ASI is possible” and “almost nobody believes that human extinction/permanent disempowerment is possible, or that AI that is superintelligent in the relevant sense can even exist” can both be true at the same time, and right now they are. I doubt this will change in the absence of an extremely loud and clear warning shot on the level of Chernobyl. And even then it’s not guaranteed that the response will be sane (after the recent ban on Mythos and opaque policy regarding which models will be available to whom, I’ve updated away from people responding to a warning shot in a reasonable manner).
human-level human-speed AGI will require not a data center, but rather something like one consumer gaming GPU—and not just for inference, but even for training from scratch.
A bit of a necrocomment, but I really don’t think this is correct.
Human brain is estimated to have 100 trillion synaptic connections, which we can roughly, very roughly, consider equivalent to neural network parameters. We’ll poke this assumption more later. For now let’s assume we’ll need a 100T parameter model to match or surpass human cognition. At FP8, that’s 100 trillion bytes. That’s 100 terabytes of VRAM needed to run it, and that is even before KV-cache, activations and any other stuff (such as Adam optimizer states). No single consumer GPU on the planet currently has that much VRAM, not even within the same order of magnitude. Going to FP4 or FP2 (if FP2 training is even possible, which I frankly doubt) would only cut that number by 2x and 4x, respectively.
What if to match 100 trillion synaptic connections you only need 10 trillion parameters in a neural network? Then you’ll need 10 terabytes of VRAM to run it. 5 terabytes at FP4 just for the weights alone. Still nowhere near what a consumer GPU can run.
Ok, let’s make maximally generous assumptions. Let’s say you only need 1 trillion parameters to match a human brain (100 times less than the naïve 100T number based on synaptic connections), and let’s say FP2 training is magically possible. Let’s say this neural net is not a Transformer, so no KV-cache (well, KV-cache isn’t used during training anyway, only during inference), and let’s say there is no extra overhead from anything such as activations/Adam optimizer states/gradients. That would still be around 250 GB of VRAM. RTX 5090 has 32 GB of VRAM. If you stretch the definition of “consumer gaming GPU” really far, H200 has 141 GB of VRAM, which is still not enough, but at least it’s within the right order of magnitude.
So no, brain-like AGI will require significantly more advanced hardware than present-day consumer-grade GPUs in the absence of a neural network architecture many orders of magnitude more efficient than what the naïve comparison suggests.
Fair point, I just didn’t have two good examples for the same model.
Do you agree that the “ceiling” of LLMs is rising faster than the “floor”?
Ceiling: the greatest feat that an LLM can do, for example solving an unsolved math problem.
Floor: the dumbest mistake that an LLM can make, for example this.
Apologies if this is too nitpicky/off topic, but I think “Self-portrait of {LLM} rendered by {image gen model}” adds nothing to the article.
Tbh I no longer consider the data in this post relevant because ever since Chain-of-Thought models became a thing, METR’s task time horizon has been increasing faster, but Epoch doesn’t have accurate training FLOPs numbers for recent closed-source models from OpenAI/Anthropic/GDM. So it’s either “estimate the relationship between training compute and task time horizons based on GPT-2, 3, 3.5 and 4”, which sucks because that was before RLVR and before CoT, or “estimate the relationship between training compute and task time horizons based on open-source models for which Epoch does have accurate estimates of training FLOPs”, which I haven’t done.
He thinks it’s a cool narrow tool, but not an indication that it’s possible to create one AI that surpasses all humans at everything, including asking questions that humans never asked before. I guess I misrepresented his opinion somewhat (I just edited my quick take). He thinks AI can help with some narrow tasks, but human touch will always be necessary for other things, especially for open-ended research. Btw, he’s not concerned about losing his job.
Do you know a person who believes that ASI will be created in <50 years who ISN’T in the LW/rationalists circle?
My parents don’t believe that a superintelligent AI will be created within this century, or ever for that matter, or that AI will ever take jobs. My relatives laugh at the idea of AI solving a high school math problem and think state-of-the-art AI is on the level of GPT-2 (I mean that the capabilities they have in mind are on the level of GPT-2, not that they know what GPT-2 is). My friend who is an organic chemist laughs at the idea of AI doing any R&D thinks that while AI can help with some narrow tasks, a truly general AI that can substitute all human researchers is sci-fi. I know 4 people who use Codex/Claude Code; 2 of them call ASI sci-fi bullshit (btw, one of them said that the “Alignment faking in large language models” paper is nonsense after only reading the summary), 1 never said anything about ASI and 1 tentatively acknowledges that maybe ASI is possible to create in theory.
I have never, in my whole life, met a real walking, talking, breathing human being who believes that ASI will be created within this century.
EDIT: obviously there are people on the internet who believe that ASI will be created soon. My point wasn’t to deny their existence, just to share my experience that makes me think “Am I living in a AI-is-a-nothingburger bubble? Am I crazy or is everyone else (whom I personally know) around me crazy?”. I’m wondering if “Everyone I personally know thinks AI is a nothingburger and people who don’t are only found in very specific places on the Internet” is a common experience.
EDIT 2: I asked my organic chemist friend to be more specific and he said that AI will be able to replace 80% of human researchers in 100 years. When asked “What about 100%?”, he said that that will never happen and at least some humans will always be necessary and that the 80% replacement figure will be due to AI automating routine tasks. Basically, when it comes to AI he’s envisioning something more like the Industrial Revolution rather than “humanity’s last invention”.
Then what would you recommend for determining whether linear or piecewise linear (only 2 segments btw) provides a better fit?
Thank you. I vaguely remember seeing a tweet about it, which is why I mentioned ECI.
5th percentile of breakpoints: 2023.00
25th percentile of breakpoints: 2024.14
50th percentile of breakpoints: 2024.31
75th percentile of breakpoints: 2024.35
95th percentile of breakpoints: 2024.36
This is definitely not what I was expecting.
Good idea! I edited the post.
Do you have a good idea when RLVR became widely used? By “good” I mean “plus-minus a few months at worst, release date of one specific model at best”.
Isn’t this always the case? You can always do linear regression, then use your extra parameter to get zero loss on the last entry for example.
This is why I used BIC: roughly speaking, if the increase in the parameter count is greater than the reduction of loss, BIC will increase. The more parameters (more precisely, degrees of freedom) the model has, the more BIC penalizes it.
Regarding RLVR, I don’t know when exactly it became widely used. 2024 vs 2025 makes a big difference here.
That’s my opinion as well.
Imagine you are running a frontier lab. If you don’t take extinction risks seriously, then it’s a (perceived) Prisoner’s Dilemma. Mutual defection (advancing capabilities while cutting corners when it comes to safety/alignment) is worse than mutual cooperation (allocating more effort, time and money to safety/alignment), but “I cooperate, you defect” is even worse because then nobody will pay for your safe but dumb models, and “I defect, you cooperate” is the best—your competitor crippled himself and your business is now booming (with a somewhat higher rate of “mundane” incidents), yay! So you get the classic Prisoner’s Dilemma payoff matrix.
If you do take extinction risks seriously, then it’s just “if anyone defects, everyone dies”, no dilemma. Unfortunately, even within the frontier labs not everyone takes extinction risks seriously. Outside of the frontier labs...well, your post already tells how it is.