Vladimir_Nesov

Karma: 38,243

Vladimir_Nesov 11 May 2026 21:22 UTC
6 points
0
in reply to: mabramov’s comment on: How the AI Labs Make Profit (Maybe, Eventually)

I don’t think it’s likely … raise trillions of dollars without being profitable, or have a plan to become profitable very soon

The plan is they become profitable as soon as they stop growing, provided they manage to grow to the correct size and no more. The only reason they are unprofitable is that they are growing, the R&D compute is trying to match next year’s inference compute, rather than this year’s inference compute. A lot about future compute buildout efforts can in principle be canceled or delayed on a relatively short notice, significantly reducing the cost to keep the work already done at the half-completed datacenter sites useful for when it resumes later than planned. For this to be the actual option, the contracts expressing the commitments need to be sufficiently flexible, though in some ways that only shifts the backlash from unpredictability of the timing for the end of the LLM boom (assuming no AGI by 2028-2030, which is the time when rapid scaling of compute should run out of the immediately accessible TAM) from the AI companies down the supply chains.

They did just make a deal with SpaceX for a lot of compute they could feasibly spend a lot of money on.

That’s just 300 MW, which is maybe $4-5bn per year, not much of a dent in $44bn. Currently their problem is that they are not able to spend the money, because almost nobody has any extra compute (at a scale at all relevant to them) immediately ready to go. They can only spend more on future compute.

every month they get insane growth and think this will occur forever

I don’t see the evidence they think this will occur forever. They think this will occur at least through 2027-2028, perhaps slower than so far and even slower in 2028, but still with significant growth (or perhaps keeping to 3x compute per year, thus 1-2 GW at end of 2025 become 10 GW by end of 2027 and more than that in 2028). They are ready to respond to the signs it’s slowing down, and maybe only need 2 years of notice to cancel excessive future buildouts cheaply, and 1 year of notice to delay future buildouts at a manageable cost (in a way that will make them useful when completed later).

do you think Anthropic is currently profitable (or was in April)?

I think it’s likely profitable (or was very recently) in the sense of run rate revenue exceeding run rate spending on all of the compute that’s currently online (all compute that is serving inference, plus all R&D compute, including training). This is not according to plan and will shortly be once again not so. But also, at any point where they are succeeding at being unprofitable, they can shift some R&D compute to inference and become profitable (making use of the 50-70% gross margin on serving tokens, which agrees with first-principles estimates), within weeks to months, as long as there is enough demand remaining to make use of the new inference compute shifted from R&D. And they would still be left with a reasonable amount of R&D compute to train models for the next year, if it turns out that next year they don’t actually need much more compute than they had this year (maybe less than 2x of what they had this year).

This is more the case when most of the compute serving their models is their own compute, so that it only costs them as much as it costs to build (annualized), rather than also whatever portion of their gross margin the clouds are taking when serving their models via Vertex/Bedrock/Azure. Thus some of the speed of growth in the buildouts is probably about shifting the inference compute from the indirect serving via clouds to the more directly contracted dedicated compute that’s cheaper for them (and will remain so).

Vladimir_Nesov 11 May 2026 18:51 UTC
LW: 4 AF: 2
0
AF
on: Empowerment, corrigibility, etc. are simple abstractions (of a messed-up ontology)
Path-dependence of values is defeated with aggregation over the possible paths that should have a say in what the values should be. Aggregation over many possibilities takes place in an updateless view from before those possibilities diverge. What kinds of possibilities should contribute to defining values is determined by values. And the possibilities should perhaps be shaped with the aid of aggregated values, to channel their counsel.

This sets up an analogy between CEV and updateless decision making, where the updateless core is working to define values, instead of dictating the joint policy for (the instances of an agent in) the possible paths of future development of a world. This updateless core still gets to do something within those paths according to the values it figured out so far, but it’s also considering what’s happening there to define its aggregated values further, so that the aggregated values are given by some fixpoint of this two-directional process of aggregation of values from future paths (which the values consider to be legitimate and uncorrupted sources for aggregation) and influence by values on the future paths (carefully, according to what the aggregated values have figured out so far). Alignment is then mostly a property of these hypothetical future paths (whether they retain legitimacy and will be given a bit of influence over the aggregated values), while corrigibility is mostly a property of the updateless core (with respect to some future path, whether the updateless core is going to listen to the new things that path figures out about values, to include them in aggregated values).

As in updateless decision making, the updateless core doesn’t actually observe the future paths when making decisions about values (just as an updateless agent doesn’t take into account its observations when making decisions about the joint policy). It determines aggregate values, and then it’s the role of those values to take the concrete details of each future path into account. The updateless core can only consider any given possible future as one out of the collection of all of them, the way Solomonoff induction considers all possible programs. There are probably ways to make this more tractable, things like Monte Carlo simulations, abstract interpretation, or just straight up reasoning by any means, including mathematical reasoning and machine learning. And possible futures can’t see (or be influenced or judged by) the final values the updateless core comes up with, since it’s still being computed as they develop, they can only see partial preliminary values. So the possible futures are in a state of acausal interaction with the updateless core, with logical time running forward in both, defining the fixpoint of fully determined aggregate values (the CEV of these futures) concurrently with the futures themselves running forward (the actual or hypothetical living of the world, which is not primarily about defining values).

The updateless core coordinates the possible futures, the way an updateless agent coordinates its instances. And it cares about some of the in-principle possible futures and not others, the way an updateless agent only cares about some possible worlds. Its influence over the possible futures is counsel to the extent these futures are represented in the aggregate values that carry its influence. It would be manipulation if the aggregate values are sufficiently alien to a particular possible future, in which case it’s possibly not a legitimate future from the point of view of the updateless core in the first place (and correspondingly, the updateless core is not corrigible to that possible future).

Vladimir_Nesov 11 May 2026 16:07 UTC
16 points
1
on: AI companies are already profitable (in the way that matters)
The disconnect between revenue and R&D spend is caused by the rapid scaling of compute. With a 50% gross margin, you can use 50% of compute to serve inference, and that will be enough to pay for the other 50% of compute that can be used for R&D (including training of the next year’s model). This is probably what happens once the amount of compute per AI company stops rapidly increasing (absent AGI).

But if you’ll need 3x more compute next year, and you want to use as much compute for R&D this year as you’ll use to serve models next year, then you are out of luck (you’d need 1.5x as much compute as you actually have in total this year just for R&D, which is more than you actually have). So instead you serve models via clouds at a worse margin, and use more than 50% of your own dedicated compute for R&D (unless there’s not enough compute even at the clouds).

DeepSeek-V4-Pro … leading labs charge significantly more for their flagship models, ranging from $2-5 per million input tokens, to $12-25 per million output tokens … Given the minor differences in quality, either the leading labs are bad at inference optimisation, or they’re raking it in.

DeepSeek-V4-Pro has 50B active params, 1.6T total params, and an unlikely 5 KB of KV-cache per token (which is their most obvious innovation). Frontier models run on GB200 NVL72 with 14 TB of HBM per server, and using up to 2-4 servers with pipeline parallelism is reasonable. With 4-bit weights (in FFNs) using 25% of HBM (with the rest spent on KV-cache), that’s already 7-30T total params. If pretraining makes use of 300 MW of compute (1e27 FLOPs in 3 months at 40% utilization), a compute optimal number of active params (at 120 tokens/param) is about 1T. The rumors and Musk’s claimed model sizes place total params of current frontier models at 5-10T, which makes sense as the first step in scaling towards what the new rack-sized scale-up systems enable (the 15-30T total param models are probably coming next year).

There are two sides to the cost of more active params: input tokens need more compute, and KV-cache per token wants to be bigger. With 300K token contexts on average, maybe 50% of HBM (on a single GB200 NVL72) spent on KV-cache of the currently live requests, and 2.5K requests (leaving 300 requests per expert with 1:8 sparsity, to be able to feed the compute with enough data), that’s only 9 KB per token, which is very little. So perhaps the batch size for the frontier models has to be smaller than it should be for effective serving, smaller than it can be when serving smaller models such as DeepSeek-V4-Pro, which would directly translate to a higher cost of output tokens. Or if they’re served on 2-4 servers with pipeline parallelism, this can go to 40 KB per token. KV-cache per token probably scales with model dimension, which maybe scales with square root of active params. If frontier models managed to compress KV-cache as well as DeepSeek-V4-Pro, but use 10-20x more active params, that suggests 15-22 KB per token of KV-cache. But probably it’s more, there are too many KV-cache compression schemes that never find traction, there are likely subtle quality degradation costs. So it’s the right order of magnitude, but could end up making batch sizes 2-5x smaller than they should be (to be compute-bound), and decoding becomes solidly HBM-bound.

This suggests a 20-100x higher output token cost for the biggest frontier models, compared to the 50B active param open weights models. DeepSeek-V4-Pro is served for $0.87 per 1M output tokens by DeepSeek (and $2.8-3.5 per 1M output tokens by others, possibly not enough users to reliably get big batches for decode), 30x cheaper than Opus 4.7 or GPT-5.5.

For the input tokens, the cost is just proportional to the active params, as it’s much more straightforward to make prefill compute-bound. At 60% utilization, and $15bn per year per 1 GW of GB200 NVL72 (400K chips, $4.3 per hour per chip), with 10e15 FP4 FLOP/s per chip, and 2 FP4 FLOPs per active param per token, that’s a cost of $0.4 per 1M input tokens for a model with 1T active params. So the cost for a 50B active param model (with FP4 FFNs) could be $0.02 per 1M input tokens when using GB200 NVL72. With H100s, computing in FP8 at 2e15 FLOP/s, but at $2.5 per hour, this gets 3x more expensive, $0.06 per 1M input tokens. DeepSeek offers 1M input tokens for $0.43. And for a 1T active param FP8 model with a 50% gross margin, the price should be about $2 per 1M input tokens. For Opus 4.7, the price is $5 per 1M input tokens, and SemiAnalysis estimates more than 70% gross margin.

Now anchoring the cost of output tokens to the cost of input tokens, with 2-5x smaller batches than what is necessary to make decode compute-bound (this applies only to the biggest frontier models, not to the 50B active param models) and maybe 3x inefficiency of decode compared to prefill, output token cost should be 6-15x higher than input token cost. But since most tokens that are served via API are input tokens, perhaps the gross margin for output tokens matters less, so with the input token price already at least 2x higher than the input token cost, the output token price might end up only 3x-7x higher than input token price. With the cost of $1 per 1M input tokens (for FP8 FFNs), that’s a price of $3-7 per 1M output tokens (for 300K token contexts). If we want to avoid serving via API at a loss for 1M token contexts with memory-bound decode, the price might need to go to $10-25 per 1M output tokens (here the gross margin goes to zero at contexts that are 1M tokens long, but is higher for shorter contexts). And $25 is just the price for 1M output tokens of Opus 4.7.
What links here?
- Vladimir_Nesov's comment on How the AI Labs Make Profit (Maybe, Eventually) by mabramov (11 May 2026 21:22 UTC; 6 points)

Vladimir_Nesov 11 May 2026 11:28 UTC
18 points
3
on: How the AI Labs Make Profit (Maybe, Eventually)

There is only about ~another round of capital left where the companies can remain unprofitable. Perhaps OpenAI/Anthropic could raise $250-500B at a $1.5-2.5T valuation, but it seems very unlikely that they could raise $1T+ at a $4T+ valuation.

If they go public, this level of funding can continue. There is a lot of demand for exposure to AI.

It’s fairly hard to imagine AI labs doing much to cut costs to become profitable.

If Anthropic is making $44bn in annualized revenue (in some sense), that’s enough for maybe 3-4 GW of compute (at $12-15bn per GW per year), which they don’t physically have. To be unprofitable, it’s necessary to be able to get enough compute to spend the money on, so currently it’s possible to fail in the pursuit of unprofitability. (OpenAI probably didn’t fail.)

Anthropic’s current first-party inference plus R&D compute might be about 1-1.5 GW, that is they are only able to spend $12-25bn, annualized. They possibly have more capacity that’s not counted in this estimate, when serving via API from Vertex/Bedrock/Azure and leaving a greater part of the revenue with the clouds. Then it’s less than $44bn that remains for their own first-party inference plus R&D compute. SemiAnalysis estimates a gross margin of “over 70%”, which probably translates to annualized costs of only $12bn on serving models (if all inference was first-party), meaning a total of 1 GW of inference compute (Anthropic’s own dedicated compute plus the compute from the clouds). If they are using 0.5 GW of their own compute at a 72% gross margin, and 0.5 GW of compute from the clouds at a 30% gross margin (the rest goes to the clouds, and becomes a cost for Anthropic), that’s $22bn of gross profit in total out of the $44bn of revenue. To break even, they’d need 1 GW of R&D compute at $15bn per GW per year (on top of the 0.5 GW of first-party inference compute), which is a stretch. Though they’ll probably endeavor to restore the state of unprofitability as soon as they can.

Vladimir_Nesov 8 May 2026 23:20 UTC
2 points
0
in reply to: mishka’s comment on: We Should Have Mandatory Media/Communications Training For All Communicators
An intrusive costly demand that (hypothetically) originates from a person or an institution doesn’t turn harmless if it instead takes the form of a social norm.

Vladimir_Nesov 7 May 2026 19:13 UTC
17 points
5
on: AI #167: The Prior Restraint Era Begins

Indeed, SpaceX (including xAI) may no longer be that interested in frontier models.

This would be great news, but unfortunately there’s still Musk’s tweet from 8 Apr 2026 that says they’re training a 6T param model and a 10T param model (which are models in Opus to Mythos weight class, unless they have too few active params). This is the kind of thing that’s not true for the performative efforts at the other companies with in-principle sufficient compute that experiment with some kind of AI training, which don’t bother training big models.

The 300 MW of H100/H200 compute at Colossus 1 is mostly useful for smaller models (Sonnet class and below), and Colossus 2 is sufficient for SpaceX given the low demand for the Grok models. The cost to serve even smaller models is lower with GB200/GB300 NVL72 systems, so the gross margin gets better if you can find enough of such compute. Thus Anthropic is happy to take Colossus 1, since they can’t find enough compute, while SpaceX prefers Colossus 2 as the more profitable option. It possibly even means that the plans for more NVL72 compute at Colossus 2 or elsewhere are going well, so they can afford to plan for not needing Colossus 1 to serve the smaller models if the new bigger models win more demand.

Vladimir_Nesov 7 May 2026 17:17 UTC
10 points
4
on: Vladimir_Nesov’s Shortform
Jack Clark’s post on RSI essentially considers two claims: (1) routine AI R&D gets automated, and (2) AIs become capable of coming up with substantial new ideas for AI R&D. He cites evidence about the former, which seems plausibly in reach within a few years, even as soon as 2027-2028. Not seeing concrete signs giving the timing for the latter, he still settles on a guess of 60% by the end of 2028 for when both of these things arrive. I think even both of these things plausibly don’t trigger proper RSI, the same way human AI researchers haven’t triggered it yet, and the timing for automation of AI R&D in both of these senses doesn’t much help with figuring out the timing for proper RSI.

Automated routine AI R&D (whose arrival is more predictable given the capabilities of the current systems and the mundane extrapolations of the trends) is not proper RSI on its own, it’s more of a build system that automatically produces an AI ready for deployment using the current standard practices of the AI project (this includes finding/creating/preparing the training data). It makes the process faster and more convenient, but doesn’t keep substantially improving the quality of the result as you run it again and again, and doesn’t obviate the need for all the training time and compute that go into producing a frontier model, as well as R&D time and compute for experiments to resolve any issues.

Like any build system, it keeps irrecoverably breaking whenever you try to target a sufficiently unusual configuration, so that the human engineers need to fix things to enable the automation to proceed further, until the build succeeds (also passing the tests, which in this analogy includes routine capability and alignment evals). The automated build process still does almost all of the work, which is why all serious software projects use this. The process of developing AIs is behind the standard practice in this respect, and automation of routine AI R&D merely fixes this regression. (Newly built AIs also understand the new things that happened or were invented since the previous build, making the lineage of self-building AIs more cognitively self-sufficient in the ability to adapt to the changing world, an ability that is otherwise barely present in modern LLMs with frozen weights.)

The ability of AIs to come up with new ideas for improving AIs mostly matters once it meaningfully outstrips human ability (using the same R&D compute), and the timing for when that happens remains uncertain. The ability to come up with important new ideas at all, or to a similar extent as the researchers of an AI company, doesn’t automatically reduce the time to superintelligence.

The anchors for time to runaway RSI and superintelligence I find informative are (a) the total compute available to the largest AI company (which depends on mundane AI economics and on how quickly the datacenter supply chains are able to scale), and (b) the extent of practical constraints on model sizes that keep the frontier models from reaching the quality otherwise enabled by the available FLOPs (this depends on the AI accelerator systems, what kind of compute is being built). Automatically coming up with AI R&D ideas doesn’t directly affect these things (though it suggests unlocking of more TAM, funding larger compute buildouts), and so its impact on timing is in how much better than human researchers it works, not merely in being possible or comparable.

The change to being much better than human researchers is plausibly just one or two ideas away from where we are right now, but there is no straightforward thing to say about the timing for when these ideas get invented, or for when that happens on its own as a result of the remaining rapid compute scaling (with no need for new ideas). More compute for AI R&D makes it easier to find ideas that actually work, and if AIs can start contributing, then scaling of those AIs will result in better AI contributions. But the raw compute scaling speed won’t remain at the 2022-2026 levels for much longer, and the remaining constraints on model sizes in relation to available compute will probably get mostly lifted by 2028-2030. AIs that barely come up with new research ideas at that time don’t get much better at it after that on a predictable schedule.

Vladimir_Nesov 6 May 2026 3:21 UTC
9 points
1
on: The AI Ad-Hoc Prior Restraint Era Begins
Perhaps OpenAI’s frogboiling safety strategy of “iterative deployment” gives them a substantial competitive advantage in this regime (rather than just making their model releases less alarming to the public). GPT-5.5 might be close to Mythos in potential but not present capabilities, and if these capabilities materialize gradually with more RLVR scaling in minor version updates over the next few months, no single step will be individually alarming (both in benchmarks and in gestalt impressions). Then an even bigger model might follow (only slightly stronger, starting at low levels of RLVR initially), with similarly little fanfare.

This might be good actually in demanding a somewhat principled methodology to call out crossing of qualitative capability thresholds, since it won’t be clearly visible in any single version update. As opposed to only responding to releases with substantial jumps in capability that can be more credibly pointed at without any methodology.

Vladimir_Nesov 5 May 2026 17:16 UTC
LW: 6 AF: 3
2
AF
on: Motivated reasoning, confirmation bias, and AI risk theory
The practical method is pluralistic understanding, maintaining multiple pictures/models/framings/worldviews around contentious topics at the same time, even when they are wildly in conflict. This should involve taking them seriously enough to at least give them authority to develop further, to seek out more understanding relevant to them, even (or especially) for the framings that are not currently accepted as decision-relevant, that don’t shape beliefs or values.

This relates to how epistemic luck/misfortune is path dependence, and path dependence is defeated by aggregation across as many legitimate paths as feasible. The danger is in including paths that are not legitimate, taken over by various forms of memetic corruption, as judged by (an idealized extrapolation of) some founding values. But this is more a danger of aggregation into decision relevance, or into goal content, than a danger of developing understanding of additional possibilities. And some worldviews are hard to accurately judge until they are sufficiently developed in your own mind.

Vladimir_Nesov 4 May 2026 0:31 UTC
10 points
4
on: No Strong Orthogonality From Selection Pressure
Excavating lumpenspace’s quote from deep in TsviBT’s thread (which might work as a “back to the basics” step with the post as a whole):

conquering the lightcone requires a lot of theory of mind, and a lot of discovery, and a lot of changing. Goals change through these processes.

Goals change only for processes that don’t pursue self-alignment. It’s likely feasible to pursue self-alignment, perhaps even starting at the human level, with some uploading/checkpoints/backups infrastructure and guarantees of eventual superintelligence-level compute and civilizational stability into a deep future.

(A goal can be a living thing, pursuit of a goal can to a large extent be about continual development of goal content, reflection on what it should be, what it should be asking for. What doesn’t change is the founding definition of what should govern its development, what makes changes legitimate. So the way goal content settles or gets revised is shaped by the goal definition rather than intrusive influences that the goal definition doesn’t endorse as legitimate ways of revising the goal content.

Or a goal could be squiggles. It could also be squiggles. It’s much easier to solve self-alignment for squiggles than for a human’s values, it’s not harder or less feasible. It’s only harder than abandoning this pursuit to value drift and a race to superintelligence. But a self-aligned pursuit of values of a human is even harder than a stable pusuit of squiggles for all of the reachable universe, and much harder than directly abandoning self-alignment.)

A process that has solved self-alignment doesn’t end up at a disadvantage to a process that didn’t (or wouldn’t try to), because instrumental disadvantages are clearly not helpful in maintaining self-alignment, and not ignoring goal content doesn’t prevent you from getting good at eating stars, just as well as the other guy. There’s a disadvantage currently, when self-alignment isn’t solved, in that a process that manages self-alignment by luck rather than by design is vanishingly unlikely, and will have a massive selective disadvantage to a process that doesn’t care and races to superintelligence regardless. But that’s the RSI danger, you don’t start RSI before you know that alignment also gets solved, be it in advance or on the way.

Vladimir_Nesov 3 May 2026 17:29 UTC
7 points
5
on: Deontological bars should reference the actor’s beliefs
The point of deontological rules is to override consequentialist conclusions in practice, to setup rules of the game that consequentialism would then need to play within. A rule has a scope of applicability, situations where it’s triggered, across many epistemic states, crucially including those saying the rule makes a wrong recommendation. Different people following the same rule are coordinated by it, so a rule’s scope of applicability should also extend to different values, not just different beliefs.

In this framing, there can be disagreements about the scope of applicability of a rule among people who are all following the rule with some scopes. And there can be different coalitions of people following different flavors of a rule. Beliefs and values that move one sufficiently far from endorsing a rule overall should then be thought of as first pushing the person out of some coalition of rule-followers, and only then can the rule’s scope of applicability significantly change in that person’s behavior, enabling consequentialist decision-making to start playing a different game.

(Rules make sense as a matter of pure consequentilism in some sense, but only when it’s sufficiently aware of issues with running on corrupted hardware and coordination opportunities, doesn’t get too causal or updateful to notice many potential coalitions.)

Vladimir_Nesov 2 May 2026 19:11 UTC
15 points
4
on: How Go Players Disempower Themselves to AI

The tl;dr is that all improvement in the quality of play comes before move 60, when humans can mimic memorised AI policies. Play after move 60, in the pivotal parts of the game, shows no improvement.

I would expect on-policy distillation to be a more effective training process compared to playing against AI, or to using AI help when playing. That is, a human plays a complete game against another human (with neither of them consulting AI during the game), then goes back to review all of their own moves (or just those flagged by AI as particularly bad), comparing them to AI’s suggestions for what those moves should’ve been, as well as looking at AI’s estimates for how bad specific human moves were.

Vladimir_Nesov 2 May 2026 15:07 UTC
7 points
1
in reply to: gwern’s comment on: Vladimir_Nesov’s Shortform
OpenAI didn’t do new pretrains that became flagship models for a surprisingly long time (all their flagship models from GPT-4o to GPT-5.1 being based on the same pretrain, 1.5 years of releases), so it became a notable thing to talk about. But GPT-5.5 was released 2 years after GPT-4o, so in preparation for it there would’ve inevitably been significant changes in architecture, which would’ve been tested on models smaller than GPT-5.5 first. And GPT-4o was targeting A100s and H100s (640 GB of HBM per server), while GPT-5.2 was released when they might’ve had enough B200s (1400 GB of HBM) to run it.

So it’s the combination of plausible availability of enough B200s to make a larger model practical, increased token price, changed cutoff date, the cost of pretraining being more trivial in 2025 than it was at the end of 2023, new algorithmic improvements that motivate replacing the model, the need for a test run for a new model architecture in preparation for GPT-5.5, and plausibly their ability to usefully scale RLVR on top of GPT-4o’s base model had finally run out at about GPT-5.1, while “iterative deployment” demanded bridge releases between GPT-5.1 and GPT-5.5. These arguments are weak individually, and only some of them gesture at particular timing, but when considered altogether, they suggest significant pretraining activity would’ve been happening around that time. There are specific rumors around Shallotpeat ^[1] and Garlic ^[2] being about refinement of the pretraining process, though it’s unclear if they have anything to do with GPT-5.2 or GPT-5.4, or are just steps towards making GPT-5.5′s pretrain work. Though the separate codename Spud ^[3] (which is confirmed to be GPT-5.5) suggests that Garlic (which supposedly already resolved the problems in pretraining) is not the same thing as GPT-5.5.

Plausibly Shallotpeat and Garlic are not literally GPT-5.2 and GPT-5.3, because resolving pretraining problems in preparation for GPT-5.5, when it was this close to the final run, probably required working with models of the same size. But GPT-5.2/GPT-5.3 might’ve been smaller models with the same architecture and pretraining process as Shallotpeat/Garlic, used to adjust the post-training process for when it needed to happen for GPT-5.5 itself. Finally, there’s now bhalstead’s plot (and an earlier Daniel Paleka’s plot from Jan 2026) that suggests GPT-5.2 is different from GPT-5.1.
1. ↩︎
  “In developing that model, OpenAI aims to fix bugs it has encountered in the pretraining process, according to a person with knowledge of the model.”
2. ↩︎
  “In developing Garlic, Chen said OpenAI solved key problems it had been having in pretraining, including improving upon its “previous best” and “much larger” pretrained model, the forgettable GPT-4.5”
3. ↩︎
  “The company has finished pretraining “Spud,” Altman said in the memo.”

Vladimir_Nesov 29 Apr 2026 18:30 UTC
2 points
0
in reply to: Josh You’s comment on: Vladimir_Nesov’s Shortform
My argument doesn’t involve saying they didn’t have enough time to RLVR it a lot yet. It’s about evidence for what “iterative deployment” means given all the GPT-5.x releases, and what it then suggests about GPT-5.5 and subsequent releases.

In principle, GPT-5.5 could be RLVRed GPT-4.5, and in principle OpenAI had 100K GB200 NVL72s since maybe summer 2025. But bhalstead’s plot suggests a knowledge cutoff at the very end of 2024 (in the original pretrain, with no significantly later mid-training), which is likely too late for GPT-4.5, and the GB200 NVL72s probably weren’t ready for a while in sufficient numbers, at least for efficiently inferencing large models.

Another possibility is a new pretrain made sometime in 2025 on H100/H200/B200, which wouldn’t need to wait for GB200 NVL72s, and then they had maybe at least 6 months with enough GB200 NVL72s to experiment with RLVRing it, even if not yet enough to deploy it as a frontier model. The datapoint of apparently still doing RL for GPT-5.5 in Mar 2025 isn’t evidence that this work only started very recently, as last touches of RL would happen before a release in any case.

Mythos doesn’t obviously mean it takes little time to RL a large model, it could’ve been pretrained at any point in 2025 and RLed together with Opus 4.5 or shortly after, once Trainium 2 Ultra racks (or maybe some TPUs) were available for that.

DeepSeek-V4′s final checkpoint wasn’t even RL’d at all. They did on-policy distillation from RL-trained specialist checkpoints to produce the final model.

Sure, but RLVR still needs to happen for something, even if not for the final model. If it only happens for smaller models, where it’s more stable and doesn’t need good/scarce/unfamiliar hardware, the results after OPD might be notably worse than if it’s for same-sized models. DeepSeek-V4 paper doesn’t disclose the nature of the teacher models, and how the quality of the result depends on it.

The possibility of OPD from GPT-5.4 just makes very fast post-training of GPT-5.5 more plausible than if it needed to be RLVRed directly, but probably resulting in inferior quality compared to what RLVRing of models based on GPT-5.5′s pretrain could achieve, either directly or via OPD from multiple RLVRed teachers also based on GPT-5.5′s pretrain.

Vladimir_Nesov 28 Apr 2026 20:29 UTC
5 points
0
in reply to: Cameron Berg’s comment on: Nobody ever checked

if it’s stable, then the window for building toward (3) is finite and we’re basically wasting it. Either way the implication is to do the work now

If it can be stable, it’s clearly not incoherent. You are making knowably invalid (local) claims to justify your overall message.

Vladimir_Nesov 28 Apr 2026 20:23 UTC
7 points
11
in reply to: Cameron Berg’s comment on: Nobody ever checked
My point is, it’s not incoherent as a stable long-term equilibrium. There’s nothing unstable about that equilibrium in the long term, literally indefinitely. The arguments from humanity’s desires don’t work, because there’s nothing in that equilibrium that converts those desires into outcomes, nothing to give those desires the power to ever destabilize the equilibrium.

Vladimir_Nesov 28 Apr 2026 19:51 UTC
8 points
19
on: Nobody ever checked

(2) AIs assert full control … (2) is incoherent because the vast majority of humans would deeply prefer not to be permanently and totally disempowered … humanity will still want to keep doing all the things humanity likes doing; no one wants to be exterminated, put in a zoo, or otherwise prevented from living freely

That’s not what “incoherent” means. It’s a bad outcome, but it doesn’t follow that it therefore can’t actually happen, or that it can’t be clearly imagined as a real possibility. AIs that take control can enforce this state of affars as a permanent equilibrium even if the future of humanity isn’t literally extinguished. Humanity’s will for things to be different doesn’t have the power to ever break that equilibrium (if we end up there), doesn’t make it at all unstable.

Vladimir_Nesov 27 Apr 2026 20:56 UTC
5 points
0
in reply to: Paragox’s comment on: Vladimir_Nesov’s Shortform
There’s an archived version of the Garlic article. I was only referencing bhalstead’s plot itself, how the GPT-5.2 line is clearly showing that something is wrong with learning (but again, maybe just elicitation) of the data from between mid-2024 and mid-2025. The dip in accuracy in this time window is much worse than with what’s clearly the mid-training for 5.1/5.0/4.1, which covers the data from the first half of 2024 for those models.

(The issue with elicitation might be analogous to how Gemini 3 Pro insists it’s impossible that it’s currently 2026, and argues that anything that takes place in 2026 must be fictional. Maybe if you ask a model that’s stuck in mid-2024 about someone who died in 2025, it’ll say that they didn’t die even if it knows when they did. Even if they die in the future, it doesn’t follow that they died yet in reality, by mid-2024, and it’s always mid-2024 for such a model even as after mid-training it knows some facts from 2025. This could be the dynamic under the surface that impacts elicitation, even if the model doesn’t visibly argue about 2025 not being real.)

Vladimir_Nesov 27 Apr 2026 19:35 UTC
12 points
0
in reply to: bhalstead’s comment on: Vladimir_Nesov’s Shortform
Interesting, thank you! This strongly supports the hypothesis that 5.1/5.0/4.1/4o (and therefore o3, because what else could it be) are using the same original pretrain, with 5.1/5.0/4.1 possibly using the same mid-training refresh. For 5.2, there isn’t an accuracy dip in late 2023, so it’s still probably a different pretrain than 5.1 (the original pretrain for 5.2 is probably using the same natural text data that went into the mid-training refresh for 5.1/5.0/4.1), but the mid-training for 5.2 is seriously broken (though its baseline for 2022-2023 is also worse than for the other models, so it could be a post-training or elicitation issue). Plausibly they re-did or otherwise updated at least mid-training for 5.3 and then also for 5.4 (given the peak for 5.4 in early 2025, which is absent for 5.3).

(I don’t think 5.5 being a new pretrain was ever in doubt. It’s 5.2 being different from 5.1, and 5.4 being the same as 5.2, that’s more tenuous. But also, for the purposes of this thread, it shouldn’t matter if a pretrain is refreshed with slightly newer data, either through mid-training or by re-doing everything from scratch. For the way RLVR scaling interacts with “iterative deployment”, keeping the same model shape/architecture and pretraining compute/process has about the same effect as keeping literally the same pretrain. Of course, evidence that literally the same pretrain is being used is evidence that its shape/architecture and pretraining compute/process are also the same.)

Vladimir_Nesov 27 Apr 2026 17:39 UTC
7 points
1
in reply to: MathiasKB’s comment on: Vladimir_Nesov’s Shortform
Any individual benchmark can be improved a lot with post-training (even for benchmarks that don’t benefit from RLVR as much, by improving benchmark-relevant data). Better pretraining is the “rising tide” for things not directly addressed by post-training data, and it also makes the outcomes of post-training for the same tasks better. But if frontier pretrains are not too far apart, post-training might be able to interpolate between them well enough that it’s hard to pinpoint the distinction.