Dalmert

• Dalmert 22 Nov 2025 4:15 UTC

  4 points

  0

  in reply to: StartAtTheEnd’s comment on: Dom­i­nance: The Stan­dard Every­day Solu­tion To Akrasia

  You might find this concept helpful to look into as have I: Pervasive Drive for Autonomy.

• Dalmert 18 Oct 2025 15:37 UTC

  4 points

  1

  on: Us­ing Bayes’ The­o­rem to de­ter­mine Op­ti­mal Protein Intake

  I’ve been thinking of doing more things like this, however, I wonder about this part:

  • P(E | H1) = 0.2 (if 150g is enough, poor recovery is unlikely)

  • P(E | H2) = 0.7 (if 170g is needed, poor recovery is likely if you only ate 150g)

  Any reason why these particular conditional probabilities are chosen here -- 0.2 and 0.7?
  

  Would there be any principled way to update these probabilities while new evidence rolls in, and maybe even start both from 0.5? I think for simple observation our set of formulae might be under-constrained, so maybe we’d need to incorporate other stream of evidence to constrain it enough?

• Dalmert 14 Oct 2025 4:29 UTC

  2 points

  1

  on: Dalmert’s Shortform

  ‘Shower thought: If Christianity is true, then being aborted is like winning the lottery’
  

  Think about it—you start to exist without consciousness and you die without consciousness. You never experience any form of suffering, because you are not physically able to feel the pain. You are going straight to heaven, because you have no opportunity to sin in any way. You don’t even have a birth sin because well… you weren’t born. You never experience human existence, you are just “born” in heaven and live in the perfect world from the beginning—like Adam and Eve would have if they hadn’t sinned. In Christianity, that’s the best position you could be in.

  But then, there’s also the darker side of it. If you want to create the highest amount of those sinless sufferless beings, you could… get abortions as frequently as you can. The problem is, it will send YOU to hell. In that scenario the woman who would do that would be an ultimate martyr, who would sacrifice her own soul to create many infinitely happy children...

  Source: /​u/​Uszanka from reddit (lightly edited mostly to clean up typos and likely ESL quirks)

  This thought is short and sweet. Saw it and figured others here might appreciate this view as well. This was a to-me-novel way to look at what might be a somewhat fundamental incoherence at the core of Christianity, maybe some of you will be inclined to argue against.

  This also ties into something I’ve been thinking about lately, that there might be very few “true” christians who do not just believe if belief, but truly fully believe what they preach. Maybe with very heavy compartmentalization and low enough self-reflection it can happen, but the concept of heaven seems to cause some issues.

  Here is another: if someone truly believes in heaven, they should take much fewer steps—than what their revealed preferences betray—to avoid dying. On a long enough time horizon, evolution might make sure that, of heaven, only belief-in-belief can persist, the others select themselves out.

• Dalmert 12 Sep 2025 14:40 UTC

  1 point

  0

  in reply to: dr_s’s comment on: Sense-mak­ing about ex­treme power concentration

  Thanks for your response, can I ask the same question of you as I do here in this cousin comment?

• Dalmert 12 Sep 2025 14:27 UTC

  4 points

  0

  in reply to: Fabien Roger’s comment on: Sense-mak­ing about ex­treme power concentration

  Reference class tennis, yay!

• Dalmert 12 Sep 2025 14:20 UTC

  1 point

  0

  in reply to: cousin_it’s comment on: Sense-mak­ing about ex­treme power concentration

  I think I see somewhat where you are coming from, but can you spell it out for me a bit more? Maybe through describing a somewhat fleshed out concrete example scenario all the while I can acknowledge that this is just one hastily put together possibility of many.

  Let me start by proposing one such possibility but feel free to start going in another direction entirely too. Let’s suppose the altruistic few put together sanctuaries or “wild human life reserves”, how might this play out after this? Will the selfish ones somehow try to intrude or curtail this practice? By our scenarios granted premises, the altruistic ones do wield real power, and they do use some fraction of it to maintain this sanctum. Even if the others are many, would they have a lot to gain by trying to mess with this? Is it just entertainment, or sport for them? What do they stand to gain? Not really anything economic or more power, or maybe you think that they do?

  What links here?

  • Dalmert's comment on Sense-mak­ing about ex­treme power concentration by rosehadshar (12 Sep 2025 14:40 UTC; 1 point)

• Dalmert 12 Sep 2025 11:33 UTC

  3 points

  2

  in reply to: cousin_it’s comment on: Sense-mak­ing about ex­treme power concentration

  There is one counterargument that I sometimes hear that I’m not sure how convincing I should find:

  1. AI will bring unprecedented and unimaginable wealth.

  2. More than zero people are more than zero altruistic.

  3. It might not be a stretch to argue that at least some altruistic people might end up with some considerable portion of that large wealth and power.

  4. Therefore: some of these somewhat-well-off somewhat-altruists^[1] will rather give up little bits of their wealth^[2] and power than to see the largest humanitarian catastrophe ever unfold before their eyes, in no small part due to their inaction playing a central role, especially if they have to give up comparatively so little to save so many.

  Do you agree or disagree with any parts of this?

  p.s. this might go without saying but this question might only be relevant if technical alignment can be and is solved in any fashion. With that said I think it’s entirely good to ask this question lest we find ourselves in a world where we clear one impossible seeming hurdle and still find ourselves in a world of hurt all the same.

  1. ^

     This only needs there to exist something of a pareto frontier of either very altruistic okay-offs, or well-off only-a-little-altruists, or somewhere in-between. If we have many very altruistic very-well-offs, then the argument might just make itself, so I’m arguing in a less convenient context.

  2. ^

     This might truly be tiny indeed, like one one-millionth of someone’s wealth, truly a rounding error. Someone arguing for side A might be positing a very large amount of callousness if all other points stand. Or indifference. Or some other force that pushes against the desire to help.

• Dalmert 5 Jun 2025 3:27 UTC

  12 points

  6

  in reply to: rain8dome9’s comment on: Truth or Dare

  > The low-trust attractor starts to bend other people into reciprocal low-trust shapes, just like a prion twisting nearby proteins.

  Convincing people using your actions sounds disgusting!

  
  Could you expand on what you mean here? I’m not sure I or others followed you. Perhaps you mean what you say sarcastically?
  
  (Formatting wise: not sure how to quote a quote here, perhaps someone knows?)

• Dalmert 23 May 2025 18:26 UTC

  1 point

  0

  on: De­fault his­tory is dead wrong

  I’m not sure why you claim that narrative as default, maybe we could try to look at some stats on how many people hold that world view in the general population. I’m aware of plenty that hold the opposite; directionally, kind of: noble savage/​romantic medievalist/​conservative/​good old days/​nostalgic/​cynical. Let’s pick noble savagism vs current-day Steven Pinkerism. There are plenty of people who would full-throatedly try to crown one or the other as supreme. I’m more epistemically reserved, without having spent too much looking into it yet—which is to say, you, the general you, feel free to take this as an invitation to make your case—though be advised, I might read your answer with my goggles of motivated-reasoning-suspicion donned given the forum we are here steeped.

  I—and I think all—must grant that this conversation cannot be had value-neutral. So I wouldn’t be terribly surprised if the answer is multi-context dependent, and X% of people might truly prefer one and Y another.

• Dalmert 15 May 2025 3:24 UTC

  14 points

  0

  on: Eliezer and I wrote a book: If Any­one Builds It, Every­one Dies

  Is an audiobook version also planned per chance? Could preordering that one also help?

  Judging from Stephen Fry’s endorsement and, as I’ve seen, his interest in the topic for some time in general, perhaps a delightful and maybe even eager deal could be made where he narrates? Unless some other choice might be better for either party of course. And I also understand if negotiations or existing agreements prevent anyone from confirming anything on this aspect, I’d be happy to hear whether the audio version is planned/​intended to begin with and when if that can be known.

• Dalmert 12 May 2025 5:48 UTC

  6 points

  0

  on: a con­fu­sion about prefer­ence orderings

  I might be missing something that’s written on this page, including comments, but if not, here is a vague understanding of mine of what people might fear regarding money pumps. I’m going to diverge from your model a bit, and use the concept of sub-world-state, denoted A’, B’, and C’, which includes everything related to the world that can be preferred except for how much money you have, which I handle separately in this comment.

  A’ → B’ → C’ → A’ preferences hold in a cycle.

  $M_{less}$ → $M_{more}$ preference also holds for money had.

  I think, agents, either intrinsically, or instrumentally, (have to) simplify their decisions by factoring them at each timestep.

  So they ask themselves:

  Do I prefer going from A’ → B’ more than having $M_{less}$ → $M_{more}$ , more concretely $M_{\Delta -1}$ → $M_{\Delta 0}$ ?

  In this example, the non-money preference is strong, so the answer is clearly yes.

  Even if the agent plans ahead a bit, and considers:

  Do I prefer A’ → B’ → C’ more than having $M_{\Delta -2}$ → $M_{\Delta 0}$?

  The answer will still be a clear yes.

  The interesting question is, what might someone who fears money pumps say an agent would do if it occurs to them to plan ahead enough and consider:

  Do I prefer A’ → B’ → C’ → A’ more than having $M_{\Delta -3}$ → $M_{\Delta 0}$?

  According to both this and your formalisms, this agent should clearly realize that they much prefer $M_{\Delta -3}$ → $M_{\Delta 0}$ and stay put at A’. And I think you are correct to ask whether planning is allowed by these different formalisms, and how it fits in.

  I think concerns come in two flavors:

  One is how you put it: if the agent is stupid, (or more charitably, computationally bounded, as we all are) they might not realize that they are going in circles and trading away value in the process for no (comparable(?)) benefit to themselves. Maybe agents are more prone to notice repetition and stop after a few cycles, since prediction and planning are famously hard.

  The other concern is what we seem to notice in other humans and might notice in ourselves as well (and therefore might in practice diverge from idealized formalisms): sometimes we know or strongly suspect that something is likely not a good choice, and yet we do it anyway. How come? One simple answer can be how preference evaluations work in humans, if A’ → B’ is strongly enough preferred in itself, knowing or suspecting A’ → B’ → C’ → A’ + $M_{\Delta -3}$ → $M_{\Delta 0}$ might not be strong enough to override it.

  It might be important that, if we can, we construct agents that do not exhibit this ‘flaw’. Although one might need to be careful with such wishes since such an agent might monomaniacally pursue a peak to which it then statically sticks if reached. Which humans might dis-prefer. Which might be incoherent. (This has interested me for a while and I am not yet convinced that human values do not contain some (fundamental(?)) incoherence, e.g. in the form of such loops. For better or for worse I expanded a bit on this below, though not at all very formally and I fear less than clearly.)

  So in summary, I think that if an agent

  1. has static preferences over complete world states

  2. is computationally boundless (enough), plans, and

  3. does not ‘suffer’ from the kind of near-term bias that humans seem to

  then it cannot be made worse by money pumps around things it cares about.

  I think your questions are very important to be clear on as much as we can, and I only responded to a little bit of what you wrote. I might respond to more, hopefully in a more targeted and clear way, if I have more time later. And I also really hope that others also provide answers to your questions.

  ---

  Some bonus pondering is below, much less connected to your post just felt nice to think through this a little and perhaps invite others’ thoughts on as well.

  Let’s imagine the terminus of human preference satisfaction. Let’s assume that all preferences are fulfilled, importantly enough: in a non-wire-headed fashion. What would that look like, at least abstractly?

  a) Rejecting the premise: All (human) preferences can never be fulfilled. If one has n dyson spheres, one can always wish for n+1. There will always be an orderable list of world states that we can inch ever higher on. And even if it’s hard to imagine what might someone desire if they could have everything we currently can imagine—by definition—new desires will always spring up. In a sense, dissatisfaction might be a constant companion.

  b) We find a static peak of human preferences. Hard to imagine what this might be, especially if we ruled out wireheading. Hard to imagine not dis-preferring it at least a little.

  c) A (small or large) cycle is found and occupied at the top. This might also fulfill the (meta-)preference against boringness. But it’s hard to escape that this might have to be a cycle. And if nothing else we are spending negentropy anyway, so maybe this is a helical money-pump spiral to the bottom still?

  d) Something more chaotic is happening at the top of the preferences, with no true cycles, but maybe dynamically changing fads and fashions, never much deviating from the peak. This is hard to see how or why states would be transitioned to and from if one believes in cardinal utility. This still spends negentropy, but if we never truly return to a prior world-state even apart from that, maybe it’s not a cycle in the formal sense?

  I welcome thoughts and votes on the above possibilities.

• Dalmert 16 Mar 2025 4:51 UTC

  3 points

  0

  on: Any-Benefit Mind­set and Any-Rea­son Reasoning

  I think this is a nice write-up, let me add some nuance in two directions:
  
  Indeed these are quick-and-dirty heuristics that can be subpar, but you may or may not be surprised just how often decisions don’t reach even this bar. In my work, when we are about to make a decision, I sometimes explicitly have to ask: do we have even a single reason to pick the option that we were about to pick over one or more others? And I find myself saying that (one of) those other options actually have reason(s) for us to pick them—I didn’t bring up the question for nothing after all.
  
  In these cases I could argue that we upgraded from no-reason deciding to at least any-reason deciding. (If we even did, in some contexts it’s not unheard of that the answer to the above is something along the lines of “I cannot name any reasons but I still want to pick the first option.”)
  
  This is how we can cross from lower sophistication to the middle. However, there are perils of going ever higher: once we have identified at least one set of opposing reasons, we cross into a regime that can be immensely costly: how to weigh reasons against each other, especially when people disagree. And I’d argue that people in general are quite bad at doing this, hence why this can take up a lot of resources and have results with questionable arbitrariness.
  
  Of course this has to all balance in how important the decision even is and how much effort, if any, should be extended towards it. And I think humans are quite bad at judging this too but we do approximate it somewhat at least with large variance.
  
  Thank you for naming these patterns!

• Dalmert 31 Dec 2024 0:51 UTC

  19 points

  0

  in reply to: Dalmert’s comment on: Open Thread Win­ter 2024/​2025

  I have found it! This was the one:

  https://​​www.lesswrong.com/​​posts/​​qvNrmTqywWqYY8rsP/​​solutions-to-problems-with-bayesianism
  
  Seems to have seen better reception at: https://​​forum.effectivealtruism.org/​​posts/​​3z9acGc5sspAdKenr/​​solutions-to-problems-with-bayesianism

  The winning search strategy was quite interesting as well I think:

  I took the history of all LW articles I have roughly ever read, I had easy access to all such titles and URLs, but not article contents. I fed them one by one into a 7B LLM asking it to rate how likely based on the title alone the unseen article content could match what I described above, as vague as that memory may be. Then I looked at the highest ranking candidates, and they were a dud. Did the same thing with a 70B model, et voila, the solution was near the top indeed.

  Now I just need to re-read it if it was worth dredging up, I guess when a problem starts to itch it’s hard to resist solving it.

• Dalmert 30 Dec 2024 7:47 UTC

  5 points

  0

  on: Open Thread Win­ter 2024/​2025

  Hey, can anyone help me find this LW (likely but could be diaspora) article, especially if you might have read it too?
  
  My vague memory: It was talking about (among other things?) some potential ways of extending point estimate probability predictions and calibration curves. I.e. in a situation where making a prediction in one way affects what the outcome will be, i.e. if there is a mind-reader/​accurate-simulator involved that bases its actions on your prediction. And in this case, a two dimensional probability estimate might be more appropriate: If 40% is predicted for event A, event B will have a probability of 60%. If 70% for event A, then 80% for event B, and so on, a mapping potentially continuously defined for the whole range. (event A and event B might be the same.) IIRC the article contained 2D charts where curves and rectangles were drawn for illustration.
  
  IIRC it didn’t have too many upvotes, more like around low-dozen, or at most low-hundred.
  
  Searches I’ve tried so far: Google, Exa, Gemini 1.5 with Deep Research, Perplexity, OpenAI GPT-4o with Search.
  
  p.s. if you are also unable to put enough time into finding it, do you have any ideas how it could be found?

• Dalmert 23 Dec 2024 11:24 UTC

  4 points

  0

  on: Hire (or be­come) a Think­ing As­sis­tant /​ Body Double

  I’m interested in variants of this from both sides. Feel free to shoot me a DM and let’s see if we can set something up.
  
  I haven’t had a good label to put on things like this but I’ve gravitated towards similar ways of work over the last 10-20 years, and I’ve very often found very good performance boosting effects, especially where compatibility and trust could be achieved.

• Dalmert 7 Jul 2024 11:06 UTC

  18 points

  0

  on: Reflec­tions on Less Online

  If anyone reading this feel like they missed out, or this sparked their curiosity, or they are bummed that they might have to wait 11 months for a chance at something similar, or they feel like that so many cool things happen in North America and so few things in Europe, (all preceding “or”s are inclusive) then I can heartily recommend you to come to LessWrong Community Weekend 2024 [Applications Open] in Berlin in about 2 months over the weekend of 13 September. Applications are open as of now.

  I’ve attended it a couple of times so far, and I quite liked it. Reading this article, it seemed very similar and I begun to wonder if LWCW was a big inspiration for LessOnline, or if they had a common source of inspiration. So I do mean to emphasize what I wrote in the first paragraph: if you think you might like something as described here then I strongly encourage you to come!

  (If someone attended both then maybe they can weigh in even more authoritatively whether my impression is accurate or if more nuance would be beneficial.)

• Dalmert 14 Sep 2023 0:38 UTC

  3 points

  2

  on: Sum-thresh­old attacks

  In a not-too-fast and therefore requisitely stealthy ASI takeover scenario, if the intelligence explosion is not too steep, this could be a main meta-method by which the system gains increasing influence and power while fully remaining under the radar and avoiding detection until it is reasonably sure that it can no longer be opposed. This could be happening without anyone knowing or maybe even being able to know. Frightening.

• Dalmert 17 Mar 2023 10:50 UTC

  3 points

  1

  on: AI: Prac­ti­cal Ad­vice for the Worried

  The employees of the RAND corporation, in charge of nuclear strategic planning, famously did not contribute to their retirement accounts because they did not expect to live long enough to need them.

  
  Any sources for this? I tried searching around without avail yet, which is surprising if this is indeed famously known.

• Dalmert 21 Feb 2023 18:51 UTC

  1 point

  0

  on: Per­sonal pre­dic­tions for de­ci­sions: seek­ing insights

  I expect that until I find a satisfactory resolution to this topic, I might come back to it a few times, and potentially keep a bit of a log here of what I find in case it does add up to something. So far this is one of the things I found:
  
  https://​​www.lesswrong.com/​​posts/​​JnDEAmNhSpBRpjD8L/​​resolutions-to-the-challenge-of-resolving-forecasts
  
  This seems very relevant to a part of what I was pondering about, but not sure how actionable are the takeaways yet.

• Dalmert 21 Feb 2023 7:53 UTC

  2 points

  1

  on: Medlife Cri­sis: “Why Do Peo­ple Keep Fal­ling For Things That Don’t Work?”

  I strong-upvoted this, but I fear you won’t see a lot of traction on this forum for this idea.
  
  I have a vague understanding of why, but I don’t think I heard compelling enough reasons from other LWers yet. If someone has some, I’d be happy to read them or be pointed towards them.
  
  I value empiricism highly, i.e. putting ideas into action to be tested against the universe; but I think I’ve read EY state somewhere that a superintelligence would need to perform very few or even zero experiments to find out a lot (or even most? all?) true things about our universe that we humans need painstaking effort and experiments for.

  Please don’t consider this very vague recollection as anywhere close to a steelman.

  I think this was motivated by how much bits of information can be taken in even with human-like senses, and how a single bit of information can halve a set of hypotheses. And where I did not see sufficient motivation for this argument for yet: this can indeed be true for very valuable bits of information, but are we assuming that any entity will easily be able to receive those very valuable bits? Surely a lot of bits are redundant and give no novel information, and some bits are very costly to attain. Sometimes you are lucky if you can even just so much as eliminate a single potential hypothesis, and even that is costly and requires interacting with the universe instead of just passively observing it.
  
  But let’s hear it from others!
  
  (I’m not sure if this spectrum of positions have any accepted names, maybe rationalist vs empiricist?)