Vladimir_Nesov

Karma: 34,832

Vladimir_Nesov 5 Sep 2025 14:25 UTC
3 points
0
on: A Pitfall of “Expertise”

Still, now I know. [...] Do you see what I had to do? Can you imagine how it probably hurt?

It’s not a universal experience, thankfully. I think flow state is most useful for juggling pipelines for learning little things like that. Perhaps it’s worth cultivating an attitude where you are not an “expert” at anything, if these are the side effects.

Vladimir_Nesov 5 Sep 2025 1:09 UTC
2 points
0
on: In Defense of Alcohol

the risk of debilitating addiction

I worry the risk of occasional mild discomfort as a result of a very resistible slight addiction isn’t being priced in. To the extent that it’s permanent, even rare consumption would create the problem. Many people assume a sufficient buffer of willpower and habit to keep consumption at sane levels, but disregard the total cost in the long term of having to manage this constraint, compared to the alternative where even the slight addiction is completely absent.

Vladimir_Nesov 3 Sep 2025 22:53 UTC
18 points
10
in reply to: ryan_greenblatt’s comment on: ryan_greenblatt’s Shortform
I think most of the value in researching timelines is in developing models that can then be quickly updated as new facts come to light. As opposed to figuring out how to think about the implications of such facts only after they become available.

People might substantially disagree about parameters of such models (and the timelines they predict) while agreeing on the overall framework, and building common understanding is important for coordination. Also, you wouldn’t necessarily a priori know which facts to track, without first having developed the models.

Vladimir_Nesov 3 Sep 2025 18:16 UTC
25 points
4
on: All Exponentials are Eventually S-Curves
An exponent models things locally, at an appropriate level of detail for modeling them locally. An S-curve won’t actually be an S-curve, there will be a lot more data than that in the real thing, omitting the data specifying when the exponent slows down is no different. Simpler models are often more useful, even when you do realize they have a limited scope of applicability.

Vladimir_Nesov 3 Sep 2025 15:01 UTC
LW: 13 AF: 6
5
AF
on: Trust me bro, just one more RL scale up, this one will be the real scale up with the good environments, the actually legit one, trust me bro
Non-OpenAI pre-RLVR chatbots might serve as an anchor for how long it takes an AI company to turn an algorithmic idea into a frontier model, after it becomes a clearly worthwhile thing to do. Arguably only Anthropic managed to catch up to OpenAI, and it took them 1.5 years with Sonnet 3.5. Even Google never caught up after 2+ years, their first credibly frontier chatbot is Gemini 2.5 Pro, which is already well into RLVR (and similarly for Grok 4). So it seems reasonable to expect that it would take about 2 years for RLVR-based models to start being done well, somewhere in 2026-2027.

The IMO results probably indicate something about the current lower bound on capabilities in principle, for informally graded tasks such as natural language proofs. This is a lot higher than what finds practical use so far, and improvements in 2026-2027 might be able to capture this kind of thing (without needing the scale of 2026 compute).

Vladimir_Nesov 2 Sep 2025 18:45 UTC
4 points
2
on: Models vs beliefs
The use of models/theories is in their legibility, you don’t necessarily want to obey your models even when forming beliefs on your own. Developing and applying models is good exercise, and there is nothing wrong with working on multiple mutually contradictory models.

Framings take this further, towards an even more partial grasp on reality, and can occasionally insist on patently wrong claims for situations that are not central to how they view the world. Where models help with local validity and communication, framings help with prioritization of concepts/concerns, including prioritization of development of appropriate kinds of models.

Neither should replace the potentially illegible judgement that isn’t necessarily possible to articulate or motivate well. That seems to be an important failure mode that leads to either rigid refusal to work with (and get better at) the situations that are noncentral for your favored theories, or to deference to such theories even where they have no business having a clue. If any situation is free to spin up new framings and models around itself, even when they are much worse than and contradictory to the nearby models and framings that don’t quite fit, then there is potential to efficiently get better at understanding new things, without getting overly anchored to ways of thinking that are much more familiar or better understood.

Vladimir_Nesov 2 Sep 2025 15:47 UTC
5 points
0
in reply to: Thomas Kwa’s comment on: Vladimir_Nesov’s Shortform
I guess the cost-quality tradeoff makes AI progress even better described as that of a normal technology. As economies of scale reduce cost, they should also be increasing quality (somewhat interchangeably). It’s just harder to quantify, and so most of the discussion will be in terms of cost. But for the purposes of raising the ceiling on adoption (total addressable market), higher quality works as well as lower cost, so the lowering of costs is directly relevant.

In this framing, logarithmic improvement of quality with more resources isn’t an unusual AI-specific thing either. What remains is the inflated expectations for how quality should be improving cheaply (which is not a real thing, and so leads to the impressions of plateauing with AI, where for other technologies very slow quality improvement would be the default expectation). And Moore’s law of price-performance, which is much faster than economic growth. The economies of scale mostly won’t be able to notice the growth of the specific market for some post-adoption technology that’s merely downstream of the growth of the overall economy. But with AI, available compute would be growing fast enough to make a difference even post-adoption (in 2030s).

Vladimir_Nesov 2 Sep 2025 1:46 UTC
13 points
0
in reply to: ryan_greenblatt’s comment on: Vladimir_Nesov’s Shortform
Exponential increase in total economic value is not specific to AI, any new tech is going to start exponentially (possibly following the startups championing it) before it gets further on the adoption S-curve. The unusual things about AI is that it gets better with more resources (while most other things just don’t get better at all in a straightforward scaling law manner), that the logarithm of resources thing leaves the persistent impression of plateauing despite not actually plateauing, and that even if it runs out of the adoption S-curve it still has Moore’s law of price-performance to keep fueling its improvement. These unusual things frame the sense in which it’s linear/logarithmic.

If the improvement keeps raising the ceiling on adoption (capabilities) fast enough, funding keeps scaling into slightly more absurd territory, but even then it won’t go a long way without the kind of takeoff that makes anything like the modern industry obsolete. After the exponential phase of adoption comes to an end, it falls back to Moore’s law, which still keeps giving it exponential compute to slowly keep fueling further progress, and in that sense there is some unusual exponential-ness to this. Though probably there are other things with scaling laws of their own that global economic growth (instead of Moore’s law) would similarly fuel, even slower.

Vladimir_Nesov 1 Sep 2025 20:36 UTC
58 points
9
on: Vladimir_Nesov’s Shortform
It seems more accurate to say that AI progress is linear rather than exponential, as a result of being logarithmic in resources that are in turn exponentially increasing with time. (This is not quantitative, any more than the “exponential progress” I’m disagreeing with^[1].)

Logarithmic return on resources means strongly diminishing returns, but that’s not actual plateauing, and the linear progress in time is only slowing down according to how the exponential growth of resources is slowing down. Moore’s law in the price-performance form held for a really long time; even though it’s much slower than the present funding ramp, it’s still promising exponentially more compute over time.

And so the progress won’t obviously have an opportunity to actually plateau, merely proceed at a slower linear pace, until some capability threshold or a non-incremental algorithmic improvement. Observing the continued absence of the never-real exponential progress doesn’t oppose this expectation. Incremental releases are already apparently making it difficult for people to notice the extent of improvement over the last 2.5 years. With 3x slower progress (after 2029-2032), a similar amount of improvement would need 8 years.
1. ↩︎
  The METR time horizon metric wants to be at least exponential in time, but most of the other benchmarks and intuitive impressions seem to quantify progress in a way that better aligns with linear progress over time (at the vibe level where “exponential progress” usually has its intended meaning). Many plots use log-resources of various kinds on the horizontal axis, with the benchmark value increasing linearly in log-resources, while it’s not yet saturated.
  
  Perhaps another meaning of “exponential progress” that’s real is funding over time, even growth of individual AI companies, but that holds at the start of any technology adoption cycle, or for any startup, and doesn’t need to coexist with the unusual feature of AI making logarithmic progress with more resources.

Vladimir_Nesov 30 Aug 2025 21:00 UTC
2 points
0
in reply to: mako yass’s comment on: MakoYass’s Shortform
The point is that people shouldn’t be stakeholders of everything, let alone to an equal extent. Instead, particular targets of optimization (much smaller than the whole world) should have much fewer agents with influence over their construction, and it’s only in these contexts that preference aggregation should be considered. When starting with a wider scope of optimization with many stakeholders, it makes more sense to start with dividing it into smaller parts that are each a target of optimization with fewer stakeholders, optimized under preferences aggregated differently from how that settles for the other parts. Expected utility theory makes sense for such smaller projects just as much as it does for the global scope of the whole world, but it breaks normality less when applied narrowly like that than if we try to apply it to the global scope.

The elephant might need to be part of one person’s home, but not a concern for anyone else, and not subject to anyone else’s preferences. That person would need to be able to afford an elephant though, to construct it within the scope of their home. Appealing to others’ preferences about the would-be owner’s desires would place the would-be owner within the others’ optimization scope, make the would-be owner a project that others are working on, make them stakeholders of the would-be owner’s self, rather than remaining a more sovereign entity. If you depend on the concern of others to keep receiving the resources you need, then you are receiving those resources conditionally, rather than allocating the resources you have according to your own volition. Much better for others to contribute to an external project you are also working on, according to what that project is, rather than according to your desires about it.

Vladimir_Nesov 30 Aug 2025 14:05 UTC
4 points
−2
in reply to: mako yass’s comment on: MakoYass’s Shortform
I think assuming the whole world as the optimization scope is a major issue with how expected utility theory is applied to metaethics. It makes more sense if you treat it as a theory of building machines piecemeal, each action adding parts to some machine, with the scope of expected utility (consequences of actions) being a particular machine (the space of all possible machines in some place, or those made of particular parts).

Coordination is then a study of how multiple actors can coordinate assembly of the same shared machine from multiple directions simultaneously. The need to aggregate preferences is a comment on how trying to build different machines while actually building a single machine won’t end coherently. But also, aggregating preferences is mostly necessary for individual projects, rather than globally, you don’t need to aggregate preferences of others as they are talking about yourself, if you yourself are a self-built machine with only a single builder. Similarly, you don’t want too many builders for your own home. Shared machines are more of a communal property, there should be boundaries defining the stakeholders that get to influence the preference over which machine is being built in a particular scope.

Vladimir_Nesov 29 Aug 2025 17:24 UTC
3 points
0
on: AI #131 Part 2: Various Misaligned Things

Their capacity is still miles behind domestic demand, their quality still lags far behind Nvidia, and of course their capacity was going to ramp up a lot over time as is that of TSMC and Nvidia (and presumably Samsung and Intel and AMD). I don’t get it.

Does anyone seriously think that if we took down our export controls, that Huawei would cut back its production schedule? I didn’t think so.

As with Intel, you need to distinguish fabs from chip (and system) design, very different activities. Having (multiple) real customers is plausibly crucially important for a fledgling/struggling fab company (they let the fab company make the process of being a customer better, and set the right priorities), and it’s probably a major reason Intel Foundry is dying. Intentions to ramp up a lot or subsidies and vague domestic demand don’t necessarily help that much (it’s not necessarily centrally voluntary and non-fake demand, given the current product). If you have a somewhat worse product as a fab company, you don’t get comparable interest in it, and can’t get those multiple real customers.

Vladimir_Nesov 29 Aug 2025 16:26 UTC
4 points
−2
on: AI #131 Part 1: Gemini 2.5 Flash Image is Cool

The AI takeover is a default.

The takeover framing is deeply misleading, because normally the thing taken over is approximately unchanging, with one power taking it from another. With AI, the thing taken over is not (merely) the human world, and so the dynamic and risks are completely different. AI doesn’t need to intrude on current human interests at all (even though it might), and can still take over the Future (much greater than the current human world), at which point human interests can become irrelevant to that Future, with no takeover in the interim.

Vladimir_Nesov 27 Aug 2025 21:38 UTC
5 points
0
in reply to: aog’s comment on: Slowdown After 2028: Compute, RLVR Uncertainty, MoE Data Wall
The question is total addressable market, and the current tech giants are the anchor for what capturing something mildly useful for everyone in the whole world gets you, which is on the order of $100bn per year. If AIs don’t get much better than they are now, it’s plausible this is where the winning AI companies land for a while (at least they probably don’t get to trillion dollar capex budgets). If AIs do get much better, then the anchor shifts to some portion of the global job market, which is in tens of trillions per year.

For now, a $200bn project every two years to build 5 GW worth of datacenters with the newest compute hardware (for a single AI company) seems plausible in a steady state ($140bn compute hardware, $60bn non-compute infrastructure). At which point it might slowly get to 10 GW in 2030s as the longer-term non-compute infrastructure no longer needs to be built anew every two years together with the compute hardware (once the total power and cooling requirements stop changing too much and it becomes possible to reuse the older datacenter campuses and only refresh the compute hardware).

But even for the 5 GW datacenter campuses, there isn’t currently enough talk about building them in 2028 to be sure they’ll be built on-trend, it sounds like 2 GW in 2028 is more likely and the 5 GW sites (serving a single AI company) will only appear in 2029-2031. Though 1 GW in 2026 also wasn’t clearly foreshadowed back in 2023, and 5 GW in 2028 is both on-trend and in principle financially plausible if growth continues, so maybe it still happens.
What links here?
- Vladimir_Nesov's comment on Vladimir_Nesov’s Shortform by Vladimir_Nesov (1 Sep 2025 20:36 UTC; 58 points)

Vladimir_Nesov 25 Aug 2025 22:03 UTC
3 points
0
in reply to: Said Achmiz’s comment on: Banning Said Achmiz (and broader thoughts on moderation)
That seems hard to judge from anything empirical, you’d need to compare with the counterfactual where there is little difficulty in distinguishing and so good tentative takes don’t need to live in squalor among piles of worthless nonsense (especially well-presented “high effort” worthless nonsense). So I don’t see how it can possibly be clearly false, and similarly I don’t see how it can possibly be clearly true, since it has to rely on low-legibility intuitive takes about unobservable counterfactuals.

Also, the problems from the difficulty to distinguish are both on the side of the authors (in the form of incentives) and on the side of the readers (in the form of low availability of good content of this type, and having to endure the worthless nonsense without even being able to know if it actually is worthless nonsense).

The difficulty to distinguish from worthless nonsense is already too much of a punishment

Empirically, this is clearly false. The track record of LW in the past ~8 years makes this very clear.

Vladimir_Nesov 25 Aug 2025 19:42 UTC
5 points
2
in reply to: Said Achmiz’s comment on: Banning Said Achmiz (and broader thoughts on moderation)

The way that we can distinguish between this possibility, and the possibility that there is something there but it’s difficult to verbalize or to characterize coherently, is precisely via discussion, conceptual analysis, examination of intent behind word choices, examination of examples (or trying to think of examples), etc.

It’s not worth making most posts where implied central points are not coherently understood by the author. But some things that look similarly are gesturing at fruitful puzzles, which are too difficult for the author to solve by the time they’ve written the post, or possibly ever. This shouldn’t of course involve the author claiming to have a coherent picture already.

The incentives should carve out a niche for this kind of communication, acknowledging practical impossibility to distinguish. The difficulty to distinguish from worthless nonsense is already too much of a punishment, so any incentives should actually want to point the other way, possibly on orthogonal or correlated considerations that can actually be resolved in practice.

Vladimir_Nesov 25 Aug 2025 15:11 UTC
1 point
2
on: Arguments About AI Consciousness Seem Highly Motivated And At Best Overconfident

Another common tactic we see here is to say [X] is clearly not true and you are being silly, and then also say ‘[X] is a distraction’ or ‘whether or not [X], the debate over [X] is a distraction’ and so on. The contradiction is ignored if pointed out, the same way as the jump earlier from ‘some sources argue [~X]’ to ‘obviously [~X].’

In a framing-aware setting, this tactic can be healthy. A framing specifies which concerns are relevant/salient, and a collection of relevant/salient concerns can define a framing. It’s a way of thinking about situations, and the same situation can be understood under multiple different ways of thinking, some a better fit than others for the situation, and some a better way of thinking about this situation for the purposes of some overarching goal.

So what the quote could be saying is that there is some framing [f] that doesn’t see [X] as a relevant/salient consideration. In particular this suggests that any framing [g] that does see [X] as relevant/salient is significantly different from [f] and is probably suboptimal for whatever purposes that [f] is good for. So [X] is a distration in the sense that even thinking about it at all directs you into the framing [g] and away from the framing [f], and that holds whether [X] or [~X] because this is an observation about relevance of [X], not an observation about its truth.

In this context, a proponent of [f] also saying that [~X] or that obviously [~X] might be making a claim about [f]‘s stance on [X] that it takes half-heartedly without considering it relevant/salient, and [f] produces it in a necessarily somewhat uninformed manner, perhaps with mild annoyance at being asked about something irrelevant to [f], and also thereby feeding competing frames that are not [f], that might do worse at [f]’s purpose.

So the error is reporting [f]‘s stance on the out-of-scope [X] (for [f]) with an implied high confidence as a claim about reality. It could well be that [f] does claim [~X] with high confidence (though that’s already suspicious since [X] in not relevant for the way [f] thinks about the world), but this should additionally have huge model uncertaintly error bars when considering that [f] might be producing bad models for considering [X], and by the time you convert credence from [f]’s internal poorly informed impression to a language of someone who considers [X] relevant, there shouldn’t be much certainty left on [~X]. So it’s more a failure of communication than an error of reasoning, though it would become an error of reasoning when a proponent of [f] tries to communicate its conclusions to their own other understandings that are not within the purview of [f], even in taking any real world actions.

Of course, being locked in a framing that lacks lightness is no good, it should be possible for any important [X] to push for either changes in [f] or for development of some other framing [g] that does take [X] seriously. That doesn’t necessarily need to damage the power of [f] or change [f], as specialized framings are very useful, but people shouldn’t be locked into specialized framings, losing ability to look at anything these framings wouldn’t consider. For anything at all, including claims that are obviously false, there is a framing waiting to develop that does consider it as relevant/salient.
What links here?
- Vladimir_Nesov's comment on Models vs beliefs by Adam Zerner (2 Sep 2025 18:45 UTC; 4 points)

Vladimir_Nesov 24 Aug 2025 6:52 UTC
7 points
2
in reply to: Marcio Díaz’s comment on: Marcio Díaz’s Shortform
Being slightly and illegibly unkind is somewhat effective at driving people away (or even motivating a change in behavior, but that’s harder) without too much drama (there are ways of expressing this in person, but online you need some UI feature to substitute), which is occasionally very useful. With downvoting, this doesn’t happen that much when contributions tend to be good, and happens often when not, so directionally it should be working well.

And the alternative to explaining-rather-than-downvoting (which is slightly inconvenient and therefore rarely happens) is mostly not providing any feedback, which in the long run lets the site content drift where the readers wouldn’t want it to.

Vladimir_Nesov 24 Aug 2025 6:18 UTC
2 points
0
in reply to: Alicorn’s comment on: Banning Said Achmiz (and broader thoughts on moderation)
(Maybe you misread what I intended about the Said/Duncan conflict? The wording in your comment seems a bit incongruous under the reading where you didn’t, sorry if I’m overthinking this. My point was that the outcome where Duncan mostly left was favorable in its longer term impact, at least so far, due to the norm influence that’s not necessarily even intended by Duncan himself, and so would be hard to argue or even know how to correctly attribute responsibility for. This is the largely illegible thing where Zack made a bit of a headway elucidating some aspects of it. But the impact of Said leaving seems unclear either way, while at the same time any issues seem more like the kind of thing he should’ve been able to fix and that’s therefore easier to attribute to his own decisions...)

Vladimir_Nesov 24 Aug 2025 1:21 UTC
3 points
0
in reply to: J Bostock’s comment on: lcmgcd’s Shortform

For example drawing from the distribution of GPT-4-base trajectories probably doesn’t kill you. … The real issues occur when you need the AI to do something difficult, like “Save the world from OpenBrain’s upcoming Agent-5 release”.

An LLM is still going to have essentially everything in its base distribution, the trajectories that solve very difficult problems aren’t going to be absurdly improbable, they just won’t ever be encountered by chance, without actually doing the RL. If the finger is put on the base model distribution in a sufficiently non-damaging way, it doesn’t seem impossible that a lot of concepts and attitudes from the base distribution survive even if solutions to the very difficult problems move much closer to the surface. Alien mesa-optimizers might take over, but also they might not, and the base distribution is still there, even if in a somewhat distorted form.