Vaniver

• Vaniver 7 Jun 2026 5:05 UTC

  4 points

  2

  in reply to: AnnaSalamon’s comment on: Com­ment on “Ban­ning Said Ach­miz”

  But he says the tension is fruitful, and that cool projects typically have at least two non-identical ideals that are both at least partially aimed at, in a fruitful tension, with development over time into how to reconcile them in practice.

  Incidentally, I think it is helpful to view epistemic rationality and instrumental rationality as two non-identical ideas that have fruitful tension with each other. Self and No-Self was in part an attempt to point at how it makes sense to deepen them in connection with each other, rather than becoming unbalanced.

• Vaniver 2 Jun 2026 16:39 UTC

  13 points

  5

  in reply to: Raemon’s comment on: Lighthaven East—A Fea­si­bil­ity Study

  Either you need to find a way to make that weirdness work in DC, or you need to find a different “source of soul.” Naively mashing the vibe into DC probably wouldn’t work.

  My first guess here is Art Deco. Ornamentation instead of minimalism to demonstrate willingness to make commitments /​ be weird; the good taste of a century ago instead of contemporary envelope-pushing to demonstrate normalcy /​ ability to be responsible for something.

  Like I think the deal has to be some virtuous spiral involving truth, beauty, and power.

• Vaniver 2 Jun 2026 3:41 UTC

  33 points

  11

  in reply to: Zack_M_Davis’s comment on: Com­ment on “Ban­ning Said Ach­miz”

  The reason it seems like an intuitively surprising claim to me is because—as a non-specialist, I thought the standard explanation for the Royal Society’s success was, um, Science?

  This explanation is on the wrong level of abstraction. From my point of view, the whole question is “How did this group of humans manage to do science together?” It wasn’t by embroidering “Science!” onto their banners; there were lots of details to their organization.

  (And the practice of science has changed substantially, over the years, such that even a practicing scientist today might be far out of touch with the underpinnings of the true science. Comparing organizations, comparing eras, asking which is better—this takes scholarship and empiricism to figure out.)

  Like, on the simplest level, Nullius in Verbia cannot be the basis of science because of limited human observational capacity. (Do you think the Earth has magma inside it? Is that because you checked, or because someone told you?) The actual basis of science has to be something more like “take people’s word only on clearly laid-out chains of checkable inferences and observations, and careful observation of their character”.^[1]

  Crocker’s rules is obviously consilient with the philosophical vision. “Anyone is allowed to call you a moron and claim to be doing you a favor,” “[w]hich, in point of fact, they would be,” says the page. (You see, because having an accurate map is instrumentally convergent for rational agents: if I were an idiot, I would want to know about it, because that might have decision-relevant consequences.)

  Crocker’s rules erode the foundation on which they rest, in a way that I suspect Eliezer learned by experience. If it is costly to call you a moron, then people deciding whether or not to do it will have to determine whether or not it’s worth making the bet, and to the extent they have skill at telling whether or not you’re a moron, they’ll only call you out when it’s correct. But if it’s too costly, you miss out on corrections you would’ve want to have; the only way to get all of those is by making it free, and thus all bets worthwhile.

  But all bets? The only reason to tell an unknown they’re a moron is because they have made a mistake; there are many reasons to call a celebrity a moron which do not depend on the celebrity having made a mistake. Crocker’s Rules liberates the autistic, but it also attracts the cruel and meanspirited to you, the sneerers who delight in you having abandoned a sort of social self-defense. At some point the signal from criticism drops into the negative, and you find yourself missing out again, because it is no longer worthwhile to pay attention to the pile of messages.

  [This is why, if you do follow Crocker’s Rules, it primarily makes sense to do so for private communications; there’s less reward for sending someone hatemail than there is for writing hateful comments.]

  That is--

  Of course, humans can’t embody the ideal, but it’s important that there is an ideal, and claims that humans shouldn’t even try to better approximate the ideal because it’s supposedly impossible don’t comport with the philosophical vision I learned from the texts.

  I think it would be different if we were saying “look, Said hurts our feelings and that’s not where we’d like to spend our limited pain tolerance.”^[2] Instead we’re saying things like the preceding sections, that “yeah I can see why people might think that Crocker’s Rules make sense, but actually we think the society where people expect that other people are (or should be) following Crocker’s Rules is worse than one where they don’t.”

  That is, the ideal is actually different based on your goals and situation. The Tao of a lone scholar is not the same as the Tao of a participant in a university or a scene; arguments that you receive a benefit in some situation from following a policy do not constitute a complete argument for that policy.

  You say that so casually! Okay, sure, humans can’t put correctness before all other concerns. But that’s, like, non-normative, right?

  I think the thing that often is going on is that someone will complain that Said is annoying, or that they didn’t like his speech act along some layer other than correctness. This will then be interpreted by you or Said as being about correctness—the deflection of legitimate concerns rather than addressing them—which will then be viewed by the someone as deflection of their legitimate concerns (about something other than correctness). Unsurprisingly, little progress is made from that starting point.

  To return to the point of instrumental convergence—many things are instrumentally convergent for rational agents, like continued survival, or other agents thinking highly of them, and so on. A thing that we need for a proper intellectual scene is more like sportsmanship, in that people want the truth to win socially, or are identifying with the sport instead of with their team. A professor who suppresses papers critical of his contribution to an academic field is not being irrational—he is pursuing a plan that achieves his values based on an accurate understanding of the world—he is instead being evil or anti-social, in that he’s putting his wants above humanity’s shared understanding; we wish he was operating on different values.

  I think this is the layer where you need to make your case—that Said was pushing people towards the right sort of sportsmanship—and I think this was not true on net.^[3]

  [Edit] On reflection I think this section might be confusing and I should try to lay out my reasoning more clearly. I think one of the arguments for Said being fine is that it is the user’s job to have an emotional orientation where they like Said’s style of criticism, and that’s because that emotional orientation is both good for them personally and also good for society. I buy the argument that the emotional orientation has benefits for society—for example, I think it’s better if the professor has it—but I think the question is “which environment causes there to be more of that orientation?” and I think your plan is “require it and let them sort out how it’s produced” in a way that I think is trying to simplify the world so that you neither need to figure out whether your plan is efficacious or put in any of the work that would cause the system to be better.

  1. ^

     Like, I’m sticking my neck out here on the Royal Society analogy, and doing it primarily on the strength of one historian. If it turns out that my understanding of that historian’s work is confused, or the work is discredited, so too my point, and this is bad for my overall credibility.

  2. ^

     The closest to this is the “limited moderation time” argument, but I’m not sure time is actually the most important resource there.

  3. ^

     I might buy that Said was operating in a way that would work well in a culture already high in this sort of sportsmanship, but that’s different from causing the culture to have more of it.

• Vaniver 31 May 2026 2:15 UTC

  80 points

  56

  on: Com­ment on “Ban­ning Said Ach­miz”

  Responding to the section about my argument (and a few related parts):

  You title it “The Founding Values Have Not Been Refuted.” But I never set out to refute the founding values! We prioritize the founding values a bit differently; perhaps from your perspective it looks like I’m dethroning the central one. But it really seems relevant to me that I am the one arguing Eliezer’s position from 13 years ago. To summarize my thoughts on Said and your arguments here, I might as well quote him from earlier, “If you fail to achieve a correct answer, it is futile to protest that you acted with propriety.”

  Vaniver then claims that the new culture is the more sophisticated one (analogous to FDT), but the justification he offers is fundamentally lacking if not absent.

  I don’t justify it in that comment; the justifications are spread across many comments and many posts and several authors. I would attempt to justify it here, but on reflection I think that’s putting the cart in front of the horse. Cultures are package deals which have many impacts, both positive and negative; no real culture is born from picking a justification and then optimizing purely for that. They are grown, negotiated, and cultivated across many people and many actions.

  To continue hammering on my example of the Royal Society, the thing that they wanted to do was understand the world better. They succeeded much more than others before them. Why? Maybe they had more geniuses; maybe they had more wealth; maybe they had better social technology. It seems to me like one of the contributors here is that they believed that social graces were important, and that managing the feelings of the participants was important. Not the most important thing—they weren’t a social club first and foremost—but not banned from consideration, a sort of anti-trump that loses every competition it enters. They wanted people to stay engaged in the project, to keep supplying data and attending meetings, and they thought seriously about how.

  But that’s not what the Ideological Turing Test (henceforth ITT) is for.

  I am using it to answer the question: “which of these views incorporates the other?”. I think it’s a valid tool for that.

  Now, is that question relevant? I was replying to parts of Said’s comment where he objected to habryka’s framing of the ban post:

  This phrasing assumes that there’s something to “understand” (and which I do not understand), and something which I should wish to “learn” (and which I have failed, or have not tried, to learn). This, of course, begs the question. The unambiguous reality is that I have disagreements with the LW moderation team about various things (including, as is critical here, various questions about what are proper rules, norms, and practices for a discussion forum like this one).

  ...

  But by describing the situation as one in which he has some (presumptively correct) understanding, which remains only for him to impart to me, and some (presumptively useful) skill, which remains only for me to learn, @habryka attempts to sidestep the need to make his case.

  I agreed that it was a disagreement about principles, and that it wouldn’t be correct to just say “we’re more sophisticated, therefore we’re right.” The way I’d put it today is that including all the same principles in your calculation and more doesn’t give you a strategy-stealing argument in the way that including all the same information and more does, because it doesn’t mean you are setting the prices correctly. Said and Zack appear to think some things should have 0 cost, and habryka and I think those things have a cost that cannot be ignored; the question is about what price the culture should put on them.

  I think this does point to what I find frustrating about this conversation, however; I think we never managed to find cruxes because you thought our charges were inadmissible, and rounded them into something different and more base. (We’ll return to this point.)

  I have to question whether Vaniver read past the title of “Critic Contributions Are Logically Irrelevant”

  So far, I’ve read everything you’ve written on LW moderation issues, because I view it as my responsibility to.

  I think I adequately handled the case of objections not intended as logical criticisms in the final section

  I don’t think so. Or, that is, I think your post spends nearly two thousand words to investigate a simple puzzle, badly. habryka and Duncan complain about the hedonic and reputational effects of a style of behavior. “But what does this have to do with correctness?” you ask, and spend 1300 words demonstrating that, as the alert reader might have anticipated immediately, it is not about putting correctness before all other concerns. Therefore it must be about something else—perhaps, for example, ad revenue.

  In a fit of Said-like politeness, I comment to ask why you don’t view comments as speech acts, a category which naturally resolves the puzzle. (Said replies that it’s not unreasonable to, and that most comments are primarily about the information content—which, while true, doesn’t obviate that we’re talking about the cases where it’s not true.)

  This time, thankfully, you cut to the chase:

  The relevance of people having objections about commenters that don’t make sense as logical criticisms is that if the objectors have their way, that establishes a precedent of the forum being a place where logical criticism is dismissed in favor of other concerns, which might be desirable for a social or hobby group whose only mission is for its members to have a fun time, but is contrary to the mission of an intellectual forum like Less Wrong purports to be.

  We were open about changing the culture and the discourse norms at the very beginning! One of the goals of LW 2.0 has always been to make posting fun again.

  I’m not sure how to say this more clearly, but we disagree about how to accomplish the mission of an intellectual forum like Less Wrong purports to be. It’s the same disagreement as the interpretation of the Royal Society or Socrates. We think it advances instead of betrays our Sacred Mission to enforce a minimum standard of social graces. It is easier for you to pretend that this is about us seeking ad revenue, apparently.

  I don’t think your values could build Less Wrong. (Obviously Said can build websites, and has built other forums.) You’re welcome to try, and to put your ideas into practice, and see what comes of them.^[1]

  Vaniver’s suggestion that I reread “Feeling Rational” is a non sequitur.

  The sequitur is that in the post EY argues that a popular belief about rationality is that rationality opposes all emotion, but this isn’t how probability theory works, and instead the question is whether or not emotions are concordant with reality. Rational emotions are downstream of models that correspond to reality; irrational emotions are downstream of models that don’t correspond to reality. When someone finds one of Achmiz’s comments hurtful, is it rational for them to do so? Well, that depends on whether or not the models underlying their hurt correspond to reality.

  [Incidentally, one of the reasons why I think this isn’t “the weaponization of emotional harm” is that we’re not just measuring emotional outbursts or tears or whatever, but instead trying to figure out how it connects to our goals for LessWrong, and sometimes the hurt seems real and relevant and sometimes it fails on either count.]

  A site dedicated to advancing the art of rationality shouldn’t go out of its way to silence people with high values of the trait in order to accommodate people with low values of the trait.

  I think you’re mistaking defense and offense. I’m not the one being silenced here, and it’s not Said’s fortitude we’re complaining about.

  That is, yes, fortitude is a virtue, as is readiness of comprehension. But it is likewise a virtue to be smooth instead of rough, and clear instead of confusing. Someone’s whose posts are confusing and incoherent may find themselves managed out—either by the karma system, or getting rate-limited, or not accepted as a new user—even if they readily understand all the posts and comments other people write.

  Is it a good thing that Said requires more fortitude in those around him than the typical poster? You could imagine a school having an evil potions master in order to cause its students to develop a certain kind of resilience, but that’s not what’s going on here. Instead we had years of asking and telling Said to be easier to deal with, and eventually told him that if he was going to keep it up, he would have to do it elsewhere.

  But the mere observation that present-day Vaniver possesses “the concept of emotional tallness” doesn’t explain what the younger Vaniver was allegedly wrong about!

  Younger Vaniver was using his personal experience—and the expectations downstream primarily of that, and advice he should have reversed—rather than empiricism, such as by polling others or doing interviews. He wasn’t attuned to the questions of what is good for the site overall, rather focused on the momentary turn-by-turn of the conversation. “Wait, someone in a position of power doing something because it’s fun or unfun?” he said with a look of disapproval, and was not tracking that how much and what Eliezer was posting was downstream of what sort of an environment LessWrong was, and that it was smoother and more rational to work inside of those dynamics rather than attempt to force them into a shape they were not, and that commenters were being shaped by the culture that they were shaping.

  Now, is this being corrupted by social pressure? I mean, you could say that, in the same way that a lot of my hygiene habits are the result of social pressure exerted on me, but that doesn’t mean it’s corruption that I shower regularly. There is an actual tension between being able to work with anyone and cultivating a circle of contacts where people can trust that someone being your friend means they’ll likely be their friend too.

  As a grown-up on an intellectual discussion forum, it’s not other people’s job to manage your feelings.

  Note this is a one-way arrow. I don’t think I ever viewed it as other people’s job to manage my feelings, but I did think I had some duty to other people to manage their feelings. (Otherwise, why be polite?) It was a limited duty—their feelings could be disproportional—but the fundamental question was not whether I was justified in losing but whether I got what I wanted.

  That is, in the Sequences era, it was understood that even if you don’t have any persistent and demanding critics in real life, you should try to simulate them.

  In the Sequences era, it was also understood why our kind can’t cooperate. We have been trying to fix that, with our new culture.

  1. ^

     To be clear, I don’t think we could build Less Wrong either; I think EY deserves that credit. I don’t think we could even have built LW 2.0 without the inheritance of the URL and The Sequences.

  What links here?

  • AnnaSalamon's comment on Com­ment on “Ban­ning Said Ach­miz” by Zack_M_Davis (31 May 2026 21:26 UTC; 16 points)

• Vaniver 22 May 2026 22:04 UTC

  6 points

  0

  in reply to: JohnWittle’s comment on: Con­clave 1492

  I think we’ll have capacity for some NPCs like this.

• Vaniver 22 May 2026 22:03 UTC

  3 points

  0

  on: Con­clave 1492

  Signups are closing Monday, May 25th.

• Vaniver 22 May 2026 21:48 UTC

  3 points

  0

  in reply to: Drake Morrison’s comment on: Con­clave 1492

  Thanks for the link; I thought I had made this a linkpost but maybe that… doesn’t work anymore?

• Vaniver 11 May 2026 21:17 UTC

  2 points

  0

  in reply to: Roger Scott’s comment on: In­tel­li­gence Dis­solves Privacy

  The main answer is “responding at all is better than not responding”. Yes, I’m aware that governments have made lots of terrible decisions over the years—it’s not obvious to me that if we had the Bush-created pandemic preparedness office at the beginning of COVID (rather than it being dissolved earlier by Trump), they would have made things better instead of worse—but from my vantage point it is obvious that Operation Warp Speed was good and the FDA being slow (both on testing earlier and on approving the vaccine) was bad. In many contexts, velocity is a virtue on its own.

• Vaniver 9 May 2026 20:42 UTC

  5 points

  0

  in reply to: kman’s comment on: In­tel­li­gence Dis­solves Privacy

  I think this is easiest to see in biorisk contexts. A fast government is more able to process new information, identify new threats and respond appropriately, and develop new capacities as required. Like, we’re still ~2 years in to discussion of mirror life with no ban /​ official government designation; with the current rate of technological progress, that’s ok but I think it wouldn’t be if technological progress were running ten times faster. (And that’s just saying “this isn’t ok”, not even developing new monitoring systems or countermeasures.)

• Vaniver 5 May 2026 5:40 UTC

  29 points

  16

  in reply to: Neel Nanda’s comment on: Ir­re­triev­abil­ity; or, Mur­phy’s Curse of Oneshot­ness upon ASI

  However, I feel like arguing that AI safety could be an incredibly hard problem is a way less extreme position than the one Eliezer seem to actually hold.

  I mean, my take on this is that around two decades ago Eliezer thought AI safety could be an incredibly hard problem, and then spent a lot of time checking, and now has lots of reasons to believe that it is an incredibly hard problem, and those reasons are spelled out elsewhere, with this post just trying to point at the problem of irretrievability.

• Vaniver 5 May 2026 3:31 UTC

  8 points

  0

  on: Tak­ing woo se­ri­ously but not literally

  Copying my comment from Substack:

  I don’t have a similarly crisp model for chakras. I don’t think they involve anything that we’d consider supernatural or as requiring exotic physics. I think they do involve emotional expression and issues, expressed in different areas of the body, manipulable in various beneficial ways. But I don’t know what the extent of those ways is, or what all the ways are. Quite possibly there’s more to it than this, that I don’t know about yet.

  I think there’s two levels of detail here in the model: the method, and the content. For Tarot, the method is “random access to a library of perspectives that give you the ability to unstick yourself /​ train thinking” and then the content involves, like, seventy-eight different elements of the distribution, which then might have deeper models of why 78 and why _those_ 78 and so on.

  For chakras, I think the method feels nearly similarly crisp to me. Humans have a mostly-but-not-completely shared bodyplan; thus it’s not that surprising if they have a mostly-but-not-completely shared introspective experience of the physiological components of their emotions.

  The content involves why seven chakras, why those specific spots, why those specific emotions, etc.; I don’t understand that particularly well (I don’t understand the body particularly well) but it’s not like I could generate the Tarot deck from scratch either!

• Vaniver 5 May 2026 3:22 UTC

  4 points

  0

  in reply to: Roger Scott’s comment on: In­tel­li­gence Dis­solves Privacy

  Yep; I think we are entering a technological regime where slow government immediately leads to human extinction, or the replacement of that government with a fast government. While it lasted, slow government had some upsides, but I think it makes sense to change with the times.

  (For example, in a lot of classic cyberpunk literature, the old governments of the world are still around, just irrelevant compared to the corporations running important parts of the important cities.)

• Vaniver 5 May 2026 3:11 UTC

  58 points

  46

  in reply to: ryan_greenblatt’s comment on: Ir­re­triev­abil­ity; or, Mur­phy’s Curse of Oneshot­ness upon ASI

  I found this surprising. Why do you think this? All of these posts/​comments seemed pretty reasonable to me. I don’t see how they are strawmanning the point in this post?

  So, take Paul’s quote, where he suggests that Eliezer sometimes says that “you can’t learn anything about alignment from experimentation and failures before the critical try.” I think Eliezer doesn’t say this? I think it’s possible to read Eliezer as saying this, or for his previous framings to make it harder to rule out that interpretation. But like with the WWI → WWII example, the question is not whether you learn anything about alignment but whether you learn enough about alignment, and I think Eliezer has always been focused on the question of “enough” and swapping that out for “anything” is a central example of strawmanning.

  Sam’s and Joe’s examples seem to be in the same vein. If Alice asks “will we sell enough paintings to cover rent this month?” and Bob responds “Alice is downplaying the importance of how earning revenue allows us to pay rent”, it is clear that Bob has made some mistake here. The question is how the numbers compare, not whether or not there’s a mechanism by which learning will work.

  I actually mostly like Buck’s quote /​ don’t think it’s doing the strawmanning. Note that Eliezer responded to it in context like so:

  You are now arguing that we will be able to cross this leap of generalization successfully. Well, great! If you are at least allowing me to introduce the concept of that difficulty and reply by claiming you will successfully address it, that is further than I usually get.

  Nevertheless, I think the same sort of disagreement is present, where Buck argues that it might not be different in the relevant respects, but I think it’s still worth asking whether it is different in enough respects, and the Eliezer-ish view is that one difference is probably enough to be fatal and Buck’s optimism seems predicated on either a much higher threshold or a much lower prior on difference for each respect.

• Vaniver 2 May 2026 6:10 UTC

  6 points

  0

  in reply to: axel_sdq’s comment on: ax­el­core’s Shortform

  I didn’t use a space back in 2015, but Eliezer did use the version with a space in 2009. So I think this rebrand happened a long time ago.

• Vaniver 23 Apr 2026 6:09 UTC

  9 points

  2

  in reply to: Matrice Jacobine’s comment on: (Re)in­tro­duc­tion of a ra­tio­nal­ist dragon, and clar­ifi­ca­tions on Ziz’s character

  Yes, but I think it matters what the letters were about and what the murders were about. Minor (the man from my example) was paranoid about the Irish but reliable about quotations and the meanings of words. I don’t think it would have made sense to publish his writings about the Irish.

• Vaniver 22 Apr 2026 4:32 UTC

  36 points

  3

  on: In­tel­li­gence Dis­solves Privacy

  See Kelsey Piper’s discovery that Opus 4.7 can reliably identify her as the author of unpublished text.

• Vaniver 14 Apr 2026 7:07 UTC

  2 points

  0

  in reply to: RobertM’s comment on: Re­turns to intelligence

  No, it’s definitely not, those are b and B.

  (I confused myself and then unconfused myself. I think B, J, L, and S in the widget correspond to 1, 3, 5, and 6 out of the pentacube pairs, but I don’t think Z and T correspond to 2 or 4. a /​ A is 2, which is why I couldn’t find the correspondence. ZR is just SL.)

• Vaniver 13 Apr 2026 20:33 UTC

  2 points

  0

  on: Re­turns to intelligence

  If you want to play around with the puzzle yourself, here is a widget.

  I suspect I am confused, but which shape in the widget corresponds to the A block in Drake’s solution?

• Vaniver 9 Apr 2026 18:03 UTC

  3 points

  0

  in reply to: dominicq’s comment on: Do not be sur­prised if LessWrong gets hacked

  It’s just very good at finding exploits in source code. I don’t expect that you can simply point Mythos towards the lesswrong.com domain and tell it “you’re in a CTF, hack this site”—finding vulns in source code is a different type of activity.

  Note that lesswrong.com is open source, which can be easily found by googling “lesswrong.com github”.

• Vaniver 9 Apr 2026 15:44 UTC

  4 points

  0

  in reply to: iamthouthouarti’s comment on: Hedg­ing and Sur­vival-Weighted Planning

  Eh, it’s sort of hard to talk about the overall future? That is roughly my confidence level of doom contingent on, like, Anthropic doing RSI starting in the next year. But that happening feels like something that is more like a 10-20% chance, and it’s harder to estimate what the doom probabilities will look like as we get further and further into the future (in part because something will mean we don’t get RSI soon, and it’s not obvious how that impacts further development).