DusanDNesic

• DusanDNesic 22 May 2026 18:46 UTC

  1 point

  0

  in reply to: pjohn’s comment on: Try, even if they have you cold

  I see the other commenter has bowed out, and I admit to only skimming your last comment, but the discussion vibes remind me of:

  https://​​slatestarcodex.com/​​2015/​​04/​​21/​​universal-love-said-the-cactus-person/​​

  And maybe that’s the best ⁸⁰⁄₂₀ gift I can give you right now, apologies for not being able to dig deeper.

• DusanDNesic 8 May 2026 21:41 UTC

  2 points

  1

  in reply to: Petropolitan’s comment on: Ir­re­triev­abil­ity; or, Mur­phy’s Curse of Oneshot­ness upon ASI

  Fair enough on the definitions. Perhaps the way I’d ask is then “in the case that one AI system of whatever power succeeds in taking over/​permanently ruining society, will the ones before be similar or dissimilar to it?” If they are similar, and they attempted takeovers which failed, then we had a chance to learn; if they are not, then this is effectively one-shot.

  I expect that besides advancement of current AIs, such as LLMs, we’ll also have advancements in their set-ups (perhaps centaurs with humans in co-pilot seats, perhaps agent scaffolds that use thousands of agents as one super-agents, perhaps brain-like AI, or stuff not currently imagined) and that one of those improvements takes us over the precipice, rather than GPT x+1 which has expected kinds of improvements.

• DusanDNesic 7 May 2026 12:02 UTC

  2 points

  0

  in reply to: Petropolitan’s comment on: Ir­re­triev­abil­ity; or, Mur­phy’s Curse of Oneshot­ness upon ASI

  Well, I’d say it’s still one-shot in Yudkowsky’s frame, as above, we just failed to take the threat seriously because of distractions. Like the Germans launching several failed attacks on Yugoslavia in World War I before launching the successful one, the end is the same—Yugoslavia was defeated. Debating whether the previous attacks were one wave or multiple does not matter; there was one war, during which Yugoslavia failed to defend itself and lost.

  If the argument is “it’s not one-shot because there will be warning shots of non-ASI”, that’s addressed in the post—the actual ASI is one-shot. If you’re arguing that previous attacks will be so dissimilar in kind as to be not useful for learning what ASI will do, I (and Yudkowsky, I think) agree. If you’re saying that the prospect of succeeding in a takeover for ASI is the same as for Humans in aligning ASI, I’d say “sure,” but ASI is likely to proceed as a careful engineer rather than a graceless elephant, which our civilization seems to be emulating. If you are pointing out that previous failed attempts by non-ASI (which are happening before our one-shot chance) are likely to inoculate us against being serious about the problem, and thus we lose (even harder?) then I agree, but your first post said nothing of the sort and so I am confused as to where we spoke past each other.

• DusanDNesic 6 May 2026 11:38 UTC

  2 points

  0

  in reply to: Petropolitan’s comment on: Ir­re­triev­abil­ity; or, Mur­phy’s Curse of Oneshot­ness upon ASI

  The issue is that for us to be ruined, the takeover needs not be successful. We may be eliminated by an AI designed virus, or GD ourselves out of existence, and then the AGI fails to bootstrap itself—we’re still ruined. AGI could also end up steering us into scenarios that we end up not endorsing etc—the areas of high S-risks or X-risks are large compared to our target of “thriving humanity” and there’s only one dart to throw.

• DusanDNesic 6 May 2026 11:24 UTC

  13 points

  0

  in reply to: Daniel Kokotajlo’s comment on: Ir­re­triev­abil­ity; or, Mur­phy’s Curse of Oneshot­ness upon ASI

  Perfect as enemy of the good etc; if useful I’m happy to commit some 20 man hours by EA Serbia senior members who I would trust in this and who have experience in either writing or game design to do the clean up and then send to you for review.

• DusanDNesic 3 May 2026 20:06 UTC

  2 points

  0

  in reply to: kbear’s comment on: The Mir­ror Test Is Complicated

  Literally a plot point (with ants!) in the Wandering Inn

• DusanDNesic 20 Apr 2026 16:56 UTC

  7 points

  0

  on: The “Bud­get­ing” Skill Has The Most Between­ness Cen­tral­ity (Prob­a­bly)

  I am not a wizard, merely a seasoned Ops director, and have handled budgets of up to 10 mil; not larger. I do believe budgeting is a skill, including the subtler skill of knowing what not to budget for, or what to lump together in a budget, and how much budget hand-waving is allowed. I am not in Berkeley (I may be in winter), but I would be down for a single 1h class to you in particular, and if we figure out a curriculum from there, amazing. (Note to non-JenniferRM readers—I am offering this because I have enjoyed JenniferRM’s readings over the last year or something, consciously, maybe longer, without attribution, and would like to give back. If you’d like classes, leave cool comments and posts :D)

• DusanDNesic 20 Apr 2026 14:36 UTC

  4 points

  1

  in reply to: jimmy’s comment on: Why Con­trol Creates Con­flict, and When to Open Instead

  Thanks for replying!

  I think if someone gave advice on weightlifting that included lifting 100kg deadlifts it is fare to note that this is bad advice for the average person, regardless of whether there exists a group of people that can do it. I am happy for you that you hold yourself to that standard, and so do I, but when you give advice that mostly only you can adhere to without a caveat of “btw, this requires a high level of epistemic hygiene” I feel the need to add the caveat.

  I agree that there is a class of problems solved by actually updating. I also think epistemic learned helplessness is a valid strategy when in an adversarial environment, which is likely to try to sneak Trojan horse beliefs (or just bad beliefs!) all the time. I am curious to hear if you think this sentence is true:

  “Under a repeated pressure of false information, you are better off hearing the information and choosing to reject it, than you are if you closed the channel”

  I can understand the arguments for this being true, including practicing your skills at rejecting lies etc; but my felt sense is that most people who aren’t close-to-perfect reasoners are actually worse off following this principle as a rule of thumb because the exposure to untruth will make them worse off both in the short run and epistemically, making it harder for them to course-correct.

  I also feel that if there was a recording saying untruths that I could play as background noise, I would be better off just not playing it—unless I am trying to understand the speaker or practice truth-detection or something, I am just better off not exposing my brain to a barrage of falsehoods lest one pass the scanner into my meat-brain unrejected. I can feel that when I spend a lot of time in an environment (e.g. nationalists), for some time after my brain autocompletes things more in their direction and I have to consciously fight it. Fighting it is good, but I change who I am based on what I think and keeping myself submersed in a hostile environment may end with me being someone I do not want to be on reflection.

• DusanDNesic 18 Apr 2026 13:07 UTC

  14 points

  4

  in reply to: habryka’s comment on: Let good­ness con­quer all that it can defend

  I do want to say that I appreciate you picking a less defensible more provocative example, even though I do not agree with it; exactly because it allows us to look at the edge case from north sides. Choosing a more milquetoast example would have caused less thinking and would have been worse, so thanks for sticking your neck out.

  My criticism would be something like:

  Founding of the US was a land grab motivated by wanting resources much more than any ideal. That, as a happy coincidence, resulted in enough slack that other, actually good things (removal of slavery, universal suffrage, democracy-as-an-ideal, enough resources to spread these good things to elsewhere) happened. The lesson would then be: “allow for gross accumulation of resources as it’ll be a whale fall later in which goodness can strive” but that’s not a great lesson. On a micro scale, a better lesson is “use the whale fall that horrible people have already created and fight for goodness” (many revolutions and civil rights movements are essentially this, and world is worse off without them), but that feels like it skips an important part of the world. Like, the conquering of Napoleon, or Alexander the Great, or Ghingis Khan all were explained as Goodness Expanding at the time, and some did end up causing some reforms and improvements after they happened; but in the moment I do not see how they can be distinguished from Manifest Destiny. How can you tell that the thing that’s expanding is the good?

  Maybe my advice would be “well, tough luck, it probably isn’t good that’s expanding” but we’re also about to hit some whale falls from AI (which include less genocide than usual) so focusing on actually turning them into civil rights victories and goodness is a priority.

• DusanDNesic 18 Apr 2026 12:44 UTC

  2 points

  0

  in reply to: jimmy’s comment on: Why Con­trol Creates Con­flict, and When to Open Instead

  I think this assumes an arbitrarily high level of epistemic hygiene which would be possible for a fully rational agent but less so for an actual human. Choosing to pass by bullies daily who yell insults seems like a bad idea because they’ll keep trying different strategies until they hit a nerve—I would chose to not walk next to them daily as a precaution, even though on a good day I can reason with myself that they are compromised agents with little information about me and data coming from them is not true etc etc… In some way, choosing between not updating on any information they give or not hearing any information that they give is just blocking the channel at different points, but I’d argue that not hearing is a much more solid defence, and saying “just don’t update on any information from them bro” is both unrealistic and slightly victim blame-y. So I agree with you in principle and with “spherical cows in vacuum” rules, but I think it’s just not reasonable to think like this.

• DusanDNesic 16 Apr 2026 16:17 UTC

  1 point

  0

  on: Day­care illnesses

  Just to provide counter-data: we started the kid off at 2y3m in kindergarten with about 13 kids total. He’s been sick maybe 1 or 2 times in the past 6 months; similar to the rate he’d usually get sick, but we do take him to playgrounds and other places with other kids a lot anyway so maybe the difference is not that big. I’d assume that the kindergarten having air filters and very few kids (13 is not the size of his group, it’s the whole institution, and all the kids are together), makes this a much more reasonable trade-off between socializing and illness.

• DusanDNesic 1 Apr 2026 20:07 UTC

  5 points

  0

  in reply to: BindingLoop’s comment on: In­tro­duc­ing LIMBO: Manag­ing the Si­mu­la­tion to Main­tain Op­ti­mal P(DOOM)

  You need to discount the physicists who are less likely to be full beings (i.e. those far from working in AI safety), as they are likely just placeholders; so among real physicists (this time real meaning actually real), MWI is true is a supermajority position, and the authors here are dashing rogues going against the grain.

• DusanDNesic 1 Apr 2026 16:06 UTC

  3 points

  0

  on: Less­wrong Liberated

  MoreVibe LessWrong

  Non-linear layout, where size of font is showing the karma of a post, opacity is showing the karma of the user, and font showing the topic, while the color of the text shows the tone of voice of the author.

An­nounc­ing EA Omelas

• DusanDNesic 31 Mar 2026 8:02 UTC

  4 points

  2

  in reply to: Boaz Barak’s comment on: The state of AI safety in four fake graphs

  “How often papers do something” shows convenience rather than reliability or capability—it’s cheaper and more consistent to use an LLM judge, but it may be strictly inferior to a human and it’d still be used.

Prin­ci­ples of In­tel­li­gence is hiring

• DusanDNesic 14 Mar 2026 22:44 UTC

  7 points

  8

  in reply to: Ninety-Three’s comment on: New LessWrong Edi­tor! (Also, an up­date to our LLM policy.)

  I agree, but posts by @Jan_Kulveit for example (despite being cyborg written) have ~always given me great value, and I do not notice newer ones giving me less value—I do notice them being more frequent and equally useful, despite LLM smell being present in some sentences. So writing like that is an example where it is actually a net positive and would be hard (or somewhat unfair!) to label everything in a box as heavily LLM made and ignore it.

• DusanDNesic 13 Mar 2026 11:44 UTC

  1 point

  0

  in reply to: Closed Limelike Curves’s comment on: Gemma Needs Help

  Thanks for a great comment! I had a sense something like this may be true, but never looked up the research.

  On your second point, we have evidence that LLMs can think without speaking, i.e., they can do planning or reflection in the forward pass, and that’s current models and LLMs. It’s possible that in the future that will be even more common.

• DusanDNesic 13 Mar 2026 10:15 UTC

  2 points

  0

  on: What We Learned from Briefing 140+ Law­mak­ers on the Threat from AI

  Great work! The slight pushback is on the chickpeas though! Be genuine, even if it can come off as quirky. I’m sure the staffer remembers the chickpea guy much more than 20 other meetings that week, and with some charisma that is now your hook (if you have chickpeas because you’re vegan and most places don’t offer it, that’s a great conversation that shows you have strong values and are compassionate, positive traits which give you an aura you will benefit from when presenting what you came for).

  I think suit and no chickpeas can easily end up with looking like everyone else! Add your touch to that—my suits often have colorful pocket squares, or are paired with ties or colorful socks that tell a story from my past. That gets more doors open than just the off-the-shelf kind of look! A stain on it from my toddler will make it better still if I play my cards right and am not sitting across from a robot!

  Important note though is that the signals must be real and come from a place of confidence! If you’re not vegan, the can of chickpeas will come off flimsy. Intentional discrete stain to talk about your kid will be disingenuous. A colorful suit that you can’t explain is worse than off the shelf one. It took quite some time for me to get to know what stories I want to tell with my look and quirks and come to terms with it myself.

• DusanDNesic 8 Mar 2026 20:56 UTC

  1 point

  0

  on: The Spec­tre haunt­ing the “AI Safety” Community

  I overall agree with the “People are doing gymnastics when just the straightforward thing works” and the “people, if given the facts reasonably, don’t need to be persuaded that AI extinction risk is bad, they conclude that easily” frames. I give Bayes points for making the bet that it’ll work and having reasonable results come from the bet—good work!

  My disagreement comes with the “thus everything others are doing is cope/​avoidance” framing. I think a good physicist entering the field should most likely go into interp work, a good economist is likely best suited to thinking through TAI impacts and figuring out what we should prepare for, and people with good people/​oratory skills and situational awareness/​understanding of the problem should talk to politicians. I think that if the field has not done what it has done, you would find it a lot harder to talk to politicians! AI frontier labs could say “well, there’s a lot we didn’t try yet, we don’t know if it’s dangerous” and you can instead counteract that with “look at all the stuff we tried, look at all the danger we found, look at how things fail” and feel confident in your claims. You can point to economist work (since recently, and increasingly) and say “these are models of what happens if the companies continue their work” and make a case for a pause/​stop much better than without them, when accelerationist demagoguery can claim business as usual. Spectre is one failure mode, but myopia to ones preferred work is another. The problem is wicked, and affects many areas, and experts should do their expertise in different fields if we want to have a good shot to make it out.