enterthewoods

• enterthewoods 29 Mar 2026 3:36 UTC

  3 points

  0

  on: [Valence se­ries] 2. Valence & Normativity

  Coming across this series years later, but loving it so far!

  I’m no expert on meta-ethics either (nor neuroscience), but I disagree slightly with the meta-ethical takes at the end. In particular, I expect that moral reasoning processes converge a decent amount among most humans (leaving aside true sociopaths) under ideal conditions, although maybe not all the way. In particular, I think I could predict with high confidence the broad strokes of morality in 2100 or 2200, conditional on a few things like humanity’s survival and the arrival of ASI. If I’m correct on this, I think that’s mild evidence for convergence. And I think this causes me to put more weight on something like the importance of a Long Reflection period.

  I think we have different intuitions here because I am putting somewhat more weight on a lot of (bad, in my view) moral reasoning being (1) reflectively inconsistent with itself, or (2) based on a fundamentally confused understanding of the universe.

  Examples of “bad” moral reasoning I expect to go away due to (1) or (2) above:

  • Not caring about animal welfare

  • Not caring /​ actively wanting harm to certain groups of people

  • Most moral reasoning based on religious belief

  • Repulsion around a lot of transhumanist ideas

  • Thinking badly /​ demanding punishment of people who have caused harm/​suffering (This last one is a bit more controversial, I’m sure, but I stand by it. Arguments against free will + a lack of instrumental reasons for justice in a well-designed Utopia seem enough to undo it for me.)

  Now I don’t expect reasoning on everything to converge. I agree that different weights on different innate drives probably cause people to value things a bit differently, and these values may diverge further upon being able to self-modify. But maybe we just disagree on how much this ends up mattering for a Utopia? It seems to me that this would just cause people to seek slightly different flavors of fun.

  One caveat: if people find ways to primarily use their upgraded intelligence to further rationalize their existing beliefs, then I agree that convergence is unlikely. I hope this doesn’t happen, and I think a Long Reflection period could be valuable especially with respect to getting people to not do this. In fact, I think a Long Reflection or something similar is probably extremely important for getting the above list to happen, such that a future with a Long Reflection period would be in expectation much “more valuable” than one without such a period.

  If you’re interested, I’d be willing to formalize the above prediction a bit more and make a (probably very silly and moot) bet on it. Maybe 1/​1000th of my wealth (if there is such a thing) post-singularity (if we’re alive)? Could be your chance to win many star systems ;).

en­terthe­woods’s Shortform

• enterthewoods 24 Jan 2026 22:22 UTC

  4 points

  2

  on: en­terthe­woods’s Shortform

  When you improve the capabilities of an LLM (or any AI agent with enough generality really) you also probably improve its ability to cooperate with copies of itself, where cooperation ability is some loose collection of abilities such as checking other copies work, planning work for other copies to carry out, course correcting other copies when they get sidetracked, integrating a copy’s work into the main project, etc.

  Now that LLMs are good enough to create useful multi-agent systems such as Claude Code, Cursor, Gas Town, etc., I expect these systems to get better at a faster rate than the underlying models (and not because the people are putting work into improving the structure of these systems, although I suspect that will also help).

  I’m not sure if these systems are currently more or less than the sum of their parts (if Claude is able to do a task with X difficulty, are 10 Claude agents able to do a task of >10X difficulty, 10X difficulty, or <10X difficulty?). But I expect that as cooperation gains accumulate, they will become less and less like “less than the sum of their parts” (as they stop bungling communication with each other) and more and more like “greater than the sum of their parts” (as they learn how to get gains from specialization or something?).

  This has updated me significantly towards shorter timelines, especially for time to automated coders. I really have no idea what happens after that though.

  Prediction: 80% chance of fully automated coding “teams” by the end of 2026.

• enterthewoods 3 Jan 2026 1:19 UTC

  5 points

  1

  in reply to: elifland’s comment on: AI Fu­tures Timelines and Take­off Model: Dec 2025 Update

  Thanks for the reply!

  I’m not sure I exactly understand your point.

  It is a very confusing point and I didn’t explain it well, sorry. I also might just be fundamentally confused and wrong. Hopefully this comment can explain it well enough so you can either shoot it down as incorrect or accept it.

  First of all, it might be easier to understand if we replace “effective compute” with “general capabilities” in my original comment. Effective compute causally affects capabilities which causally affects both the METR time horizon measurement and the AI R&D speedup variable. So we can screen off the effective compute node and replace it with a general capabilities node.

  If a model has achieved very high time horizons, then we think that this is direct evidence for them being an AC.

  I mostly agree with this. However, I think a model having very high time horizons is direct evidence for capabilities being high, which is then direct evidence for AI R&D speedup being high (and thus more likely to be past the AC threshold).

  This leads to an important distinction. If you want to argue that AI R&D speedup will be higher (or certain thresholds of AI R&D speedup like AC being reached faster) using the METR graph, your argument fundamentally has to be an argument about capabilities being higher (arriving sooner) or an argument about the relationship between capabilities and AI R&D speedup.

  I don’t think that your first argument about the superexponential nature of the METR graph (that infinite time horizons in finite time implies it must be superexponential) is either of these. It seems to be an argument purely about the relationship between capabilities and the METR graph.

  I also think your second argument for superexponential growth (that subsequent doublings should get easier because they require less new capabilities than earlier doublings) is mostly an argument about the relationship between capabilities and the METR graph. Although I could maybe see it being an argument about the relationship between capabilities and AI R&D speedup (the last few capabilities to be unlocked provide huge boosts to AI R&D speedup?).

  Basically, my core concern is if neither of these arguments are truly arguments about capabilities arriving faster, or about the relationship between capabilities and AI R&D speedup, then why are they being used to update the estimation of time to ACs?

  In contrast, arguments about AI R&D feeding back into itself, or about compute investment slowing down, are arguments directly about how fast capabilities will advance. So these validly affect your estimation of time to ACs.

  If you disagree with me and think that the arguments for superexponential growth are not just arguments about the relationship between capabilities and the METR graph, then we can focus our discussion there. To the extent that these are also arguments about the things we truly care about, you should adjust time to ACs based on them; this is what I was trying to capture with the correlation stuff in my first comment.

  If you do end up agreeing, I also don’t really know how to incorporate this into the model. It seems hard and messy. I think that it is true that using the METR graph is the best thing we can do right now. I just don’t think that these arguments about the superexponential nature of the METR graph should affect the estimation of time to ACs. But I do think that it is superexponential...and I think that it is the best way to estimate time to ACs...so again, messy :(. Hope this makes more sense!

• enterthewoods 2 Jan 2026 4:01 UTC

  13 points

  1

  on: AI Fu­tures Timelines and Take­off Model: Dec 2025 Update

  (Apologies for the long comment).

  I want to make a point about your arguments about the growth of time horizons being superexponential. I think they are generally correct, but I think they need to be downweighted somewhat in the timeline model.

  This is how I understand your model:

  Our starting point is to take the METR graph and extrapolate it exponentially, as they do, making a guess about what agentic coding time horizon would correspond to the AC milestone.

  And then you include adjustments to this extrapolation, some of which are arguments about superexponential growth that don’t have anything to do with AI R&D speedups feeding back into themselves. Because you are using a threshold on the METR graph to determine when ACs happen, these arguments about superexponential growth meaningfully affects your prediction of time to ACs.

  I consider the casual network to look something like this:
  

  

  
  (The METR time horizon level and the level of AI R&D speedup are both driven by the level of effective compute.)

  Since we only truly care about AI R&D speedup, we must differentiate between arguments about how fast effective compute will advance or how these advances affect the R&D speedup (which both affect AI R&D speedup and the time to ACs), and arguments about how much effective compute will affect the METR time horizon (which is not what we ultimately care about).

  The argument that superexponential growth is implied by infinite time horizons is purely an argument about the relationship between effective compute and the METR time horizon. Whether or not it is true does not change the level of effective compute you need to get ACs. This also applies to your second argument for superexponential growth (that doublings get easier to achieve naturally because less effective compute is needed to jump from 1 month to 4 months than from 1 week to 4 weeks, for example). Again this is only an argument about how increases in effective compute affect the METR time horizon graph, not how fast effective compute is increasing or how increases in effective compute increase AI R&D speedup.

  Now this doesn’t mean you have to throw out this entire section of the model. Importantly, it seems like there should be at least some correlation between the relationship between effective compute and the METR time horizon and the relationship between effective compute and the AI R&D speedup. But unless this correlation is 1-1, arguments about superexponential growth that come from the relationship between effective compute and the METR time horizon should be downweighted.

  Here’s a toy model to illustrate this better:

  Imagine there are four effective compute levels: X₁, X₂, X₃, and X₄. X₁ is where we are at right now. Let’s say that if METR is exponential in relation to effective compute, we hit Y horizon length at effective compute level X₄. On the other hand, if METR is superexponential in relation to effective compute, we hit Y horizon length at capability level X₂. Let’s imagine that we thought we would get ACs at effective compute level X₄, around where METR was supposed to hit horizon length Y if it were exponential. Suppose we now know that the METR graph is superexponential and will hit Y at X₂. How should that affect our expectation of when we will hit ACs? If the correlation between the relationship of effective compute to the METR time horizon and the relationship of effective compute to AI R&D speedup is 1-1, we should update to X₂. If there is no correlation, we should keep our estimation at X₄. If there is some correlation, maybe we say X₃?

  The consequences of this are, I think, slightly longer timelines from the model.

• enterthewoods 1 Jan 2026 17:18 UTC

  2 points

  1

  on: You will be OK

  2. A working hypothesis: I propose that even though there are multiple possible outcomes, including ones where you, I, and everyone, will very much not be OK, people should live their day to day under the hypothesis they will be OK. Not just because I think that is the most likely outcome, but also because, as I said, it is best not to dwell on the parts of the probability space that are outside your control. This was true for most people during the cold war regarding the possibility of a total nuclear war, and is true now.

  I think I disagree slightly with this idea. It feels like a local optimum to just ignore the parts of the probability space where you won’t be ok. It feels like a local optimum in the sense that its easier to attain but is inferior to the global optimum. For me, the global optimum (in the sense that this point is harder to attain but better for you and the world), which I think the post you are responding to captures quite well, is to stare The Truth in the face: map the true probability of doom the best you can (whether its high or low), and accept it fully and act and feel appropriately.
  
  If I, my friends, my family, my country, my species, and my planet are going to die, I want to know. I want to know not only so I can do my part to make that not happen, but I also want to know so that I can behave the way I want to on my deathbed. So I can prepare myself to comfort others if one day the doom starts to seem inevitable. So I can be maximally grateful for every second I still have on this planet. So I can live without regrets. So I can do good while I still can.

  This is hard. I have spent a lot of time struggling with accepting all of this. However, I think I’m getting there. And I think it has brought me to a much better place, both for myself and my planet, than where I would have ended up if I had chosen to act as if I was going to be ok.

  I don’t think this global optimum is for everyone. At least not right now. I don’t tell most of my friends and family about my perspective on doom. Especially not unprompted. Some people can’t help, and some people will suffer significantly if they knew.

  But for those of us who can, let’s try.

An in­tu­itive ex­pla­na­tion of back­door paths us­ing DAGs

• enterthewoods 14 Nov 2025 2:05 UTC

  3 points

  0

  on: Para­noia: A Begin­ner’s Guide

  Great post—enjoyable read and connected some concepts I hadn’t considered together before.
  
  The first thing that immediately comes to mind when I think about how to act in such an environment is reputation: trying to determine which actors are adversarial based on my (or other’s) previous interactions with them. I think I would try this before resorting to the other three tactics.
  
  For example, before online ratings became a thing, chain restaurants had one significant advantage over local restaurants: if you were driving through and needed a place to stop and eat, the chain may have an expected quality that is worse than the local restaurant, but at least you knew you weren’t going to be sick because of their reputation.
  And now that we have Yelp, I can just look there and see which restaurants consistently get 4-5 stars and no complaints of food poisoning, chain or not.
  
  Of course if a restaurant was really trying to deceive you, it could bury negative reviews in thousands of bot ratings (I actually don’t know if this is truly possible given moderation of many rating platforms). And when the environments get more adversarial and the benefits of deceiving you become higher (or the costs of taking the honest route become higher), I imagine a reputation-based filter like this could be easily thwarted. So this feels like an intuitive and lower-cost first line of defense, but not a reason to fully retire the big guns that you talked about in this post.

Pause from Be­hind /​ Los­ing Heroically