I’ve tried a few hand-wavy methods to estimate this, and they seem to point to 8 minutes
Can you elaborate?
simulus
Karma: 228
Thought inspired by SSI seemingly failing to achieve anything with their money and years of secret research
Nvidia’s recent huge investment in SSI seems like strong counter-evidence to this claim.
In fact, if you believe in their conviction, then receiving massive investments while the public hears nothing is probably what you would expect if they were succeeding in their goals.
All these specialized chips have their own advantages which could be helpful in fast takeoff
This is a good point. More options could increase p(there exists a chip that does what the AI wants).
Also, a sufficiently advanced AI could design its own ASICs for various tasks in parallel if it needed to, perhaps within a few weeks/months
Depending on who you talk to, weeks/months might not be a “fast” takeoff.
But this brings up another point: the diversification of chip manufacturing chains could make it faster/easier for an advanced AI to produce custom chips of its own design, which would increase takeoff speed.
For various reasons, I think it’s likely that methods that involve continual learning (i.e. modifying the weights during deployment) will come online soon
What are the particular reasons or evidence that make you think this?
But to play the devil’s advocate, you’re looking at natural selection with a massive amount of hindsight. Perhaps the best fit to humanity’s pre-Darwin data (which might have been more limited than you think) isn’t natural selection. There are many ways to fit a curve [1], and sometimes the best (or equally good) fit for limited data is the wrong one.
While you may be right that Darwin could have come up with natural selection without collecting data, you would have a much stronger argument if you chose someone who actually did it that way. Again I think Einstein and Newton (at least regarding gravity) are better examples.
1: https://www.lesswrong.com/posts/wgzd7y6icyMmNvFKi/why-aren-t-there-more-alphafolds
PS
as far as I know, he didn’t spend time gathering geological evidence
He actually started out with a big (or even primary?) focus on studying and collecting geological data.
I don’t strongly agree or disagree with the core thesis, but Darwin is a bad example.
He spent a lot of time collecting new data while traveling on the HMS Beagle. Much of his evidence for natural selection was not previously known to science, including the variation of Galapagos finches and the distribution of other endemic species.
Maybe he could have come to the same conclusions without embodiment, but he didn’t. If you’re right then surely there is a better example out there. Einstein might be a good one.
Together, these factors mean that it’s often feasible to have an AI produce a lot of data, curate the very best samples, and then train the next version on those.
It should be emphasized that this is basically what reinforcement learning is.
In fact there are perfectly valid RL constructions (naive REINFORCE with no baseline, no KL penalty, +1 rewards for good outputs and 0 otherwise) that are exactly this: sample a bunch of outputs, and train on the good ones using next-token prediction.
Given that the bottleneck now seems to be verification rather than generation for RL training, I’m surprised that we haven’t seen any labs going all-in on building verification models.
Being worse at explaining things will make those around you less able to understand your points, which may make them seem less intelligent.
Therefore, being worse at explaining may make you feel more intelligent relative to those around you.
Not quite the same, but TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning uses something similar to successfully reduce hallucinations.
Instead of binary correct/incorrect rewards, they give ternary rewards: +1 for correct, 0 for abstaining, and −1 for incorrect.
ProRL is a pretty hard counter to the “RLVR only elicits existing capabilities” argument:
Furthermore, Nemotron-Research-Reasoning-Qwen-1.5B offers surprising new insights —RL can
indeed discover genuinely new solution pathways entirely absent in base models, when given sufficient
training time and applied to novel reasoning tasks. Through comprehensive analysis, we show that
our model generates novel insights and performs exceptionally well on tasks with increasingly
difficult and out-of-domain tasks, suggesting a genuine expansion of reasoning capabilities beyond
its initial training. Most strikingly, we identify many tasks where the base model fails to produce any
correct solutions regardless of the amount of sampling, while our RL-trained model achieves 100%
pass rates (Figure 4).Conflicting results from other papers probably come from:
Not enough scale (RL is super inefficient, so it takes a long time to really work)
Poor training methods (continued improvement requires entropy regulation to prevent diversity collapse and stalled exploration)
Another counter-argument is classic RL experiments like AlphaZero and Atari gameplay. These obviously elicit new capabilities, since the models reach superhuman performance after starting with no knowledge at all. In theory, there’s no reason that RLVR couldn’t teach untrained LLMs to do math from scratch (but in practice sparse rewards mean that the sun would burn out before the model figures it out).
I don’t think (A) is inconsistent. If one believes that “freeing other people from work” is a very good thing, then the value of doing so is worth the personal sacrifice/effort/labor (even if there is no personal financial motive).
This is a great write-up and I found myself reading a few of your previous posts too. However, I’m left by all of this with a disappointingly mundane vision of the future where technology doesn’t change much.
I would be curious to hear about some tech and ideas that you’re optimistic about or that you think *will* deliver.
The arguments here match a lot of my own intuitions. I want to add a few things:
1. Even benchmarks that supposedly measure sample efficiency on abstract problems fall victim to human priors: ARC Is a Vision Problem! - https://arxiv.org/abs/2511.14761
2. Human learning largely performed on-policy, while pretraining is primarily off-policy. This means that humans can seek out the information that they specifically lack, and receive feedback to address their specific mistakes. I predict that the shift towards RL (and more recently on-policy distillation) is the first phase of a broader transition towards primarily on-policy training pipelines that will bring gains in both sample and parameter efficiency.
I like this idea.
Models with access to a python interpreter might be able to solve it trivially by calling a random function.
I wonder if there are examples of RL training on (more useful) tasks like this where reward is predicated on the distribution of the model’s outputs over multiple samples.
I think part of this is due to the choice of tested models: Sonnet Sonnet, Opus, GPT-5, Sonnet. If the tested models were consistently flagship offerings (Opus, GPT-whatever), then the trend would be more clear.
Furthermore, the most recent model that they tested was Sonnet 4.5. Most people agree that there was a noticable jump in quality around Opus 4.5/4.6, and that is missing from the graph. Opus 4.5/4.6 also seemed to kick off a trend of labs focusing heavily on agentic coding, which would change the slope of the trend.
In section 1 of your findings, the “Expected” column of the table is misleading. It assumes that probabilities within each bucket are uniformly distributed (or more generally, symmetrically distributed around the center of the bucket’s range).
A more faithful Expected value would be the average over each market’s probability within a given bucket. This is the true rate that perfectly calibrated markets would resolve at.
I suspect that this is the cause of the significant discrepancy between the Expected and Actual resolution rates near the 0% and 100% extremes.
Can you speak more on what the experimental results would have looked like if theories like VC dimension had been correct?
What do you see as the difference between the continual learning you’ve described here and fixed-state-size architectures like KV-binding TTT or even the DeltaNet architectures already used in Kimi and Qwen? Is it just a matter of longer context length?