jchan

• jchan 21 Feb 2026 1:15 UTC

  12 points

  4

  in reply to: omegastick’s comment on: How To Es­cape Su­per Mario Bros

  I think analyzing the “Signal” is the key. It’s an output, and so it is not causally upstream of anything else, but it is evidently not necessary to play the game, so it stands in need of an explanation. And its modality (analog waveforms) is clearly unlike anything else in the machine. The narrator would be able to deduce the concept of “music” by noticing that the frequencies relate to each other by factors of 2^(1/​12) that often seem to approximate small whole-number ratios like 3:2 and 5:4. From this it can conclude that the intended users of the machine have some sensory modality that directly perceives waves and does some kind of Fourier analysis of their frequencies, and which is likely distinct from the modality by which the “screen” is perceived.

  I don’t know where this gets you exactly, since periodic waveforms can have any number of physical explanations, but it seems a lot closer to the physical world than anything else here.

• jchan 12 Feb 2026 20:59 UTC

  1 point

  0

  on: Cop­ing with Deconversion

  

  I think some of the comments suggesting you cobble together an “ersatz religion” ^[1] are misguided—it will always feel like a fake, inferior substitute for the real thing, because that’s what it is. Instead I suggest: read up on a wide variety of perspectives from different cultures and historical periods, and try to imagine how they feel from the inside. Then you might see that the very concept of “religion” is rather more fuzzy and contingent than you may have thought, and you’ll feel less of a sense that you’re missing something important.

  1. ↩︎

     This motivated me to start writing an explainer about “ersatzness” which I think is a useful concept in general. It’s not ready yet, but the above crudely-drawn visual aid conveys the essence of it.

• jchan 28 Jan 2026 1:38 UTC

  4 points

  0

  in reply to: Warty’s comment on: Disagree­ment Comes From the Dark World

  I like to call this Cassandra/​Mule discourse.

• jchan 15 Jan 2026 14:26 UTC

  5 points

  0

  in reply to: Derek DeHart’s comment on: Back­yard cat fight shows Schel­ling points pre­ex­ist language

  Assuming the claim is true, where do you go from here?

  Potential answers:

  • This can motivate us to look more closely at animal behavior and evolutionary history to help us understand how humans behave in territorial conflicts (as alluded to in that book I mentioned) - because if human territorial behavior is not fundamentally linguistic in nature, then there’s no reason to suppose it’s unique to humans.

    • By contrast, in an area of human behavior that clearly is linguistic, e.g. storytelling, studying animals won’t tell us much.

  • The problems of philosophy are iatrogenic ^[1] . So maybe if you haven’t spent a lot of time getting yourself confused by debating metaethics, this conclusion seems trite to you; but for some of us it helps dissolve the confusions we acquired earlier. (For example, I used to favor the position that moral statements are purely logical, analytic truths like those of mathematics, and so any attempt to justify them by empirical observation or “human nature” is immediately suspect.)

  • More speculatively: People have been talking recently about LLMs engaging in negotiations. But LLMs are not animals, but purely linguistic entities. So what they’re doing may differ from what humans do in some important way.

  1. ↩︎

     See also Aaronson: “the world needs good philosophers, if for no other reason than to refute bad philosophers!”

• jchan 15 Jan 2026 13:44 UTC

  24 points

  9

  in reply to: mako yass’s comment on: Back­yard cat fight shows Schel­ling points pre­ex­ist language

  If that’s the case, then isn’t it an astonishing coincidence that the negotiated boundary just so happens to coincide with the fence?

  Some experiments that would more clearly test the use of Schelling points:

  1. If instead of a fence we have an even more trivial boundary marker, like a line of 4-inch-wide curb stones, do the animals still use that as their boundary? (Prediction: Yes)

  2. If the boundary marker is surreptitiously moved slightly to one side, do the animals respect the new location, or do they continue using the old location? (Prediction: New location, until the deviation from the original line becomes significant, at which point the disfavored party will breach it.)

  3. If the boundary marker is removed entirely, do the animals start fighting again? (Prediction: Yes)

• jchan 14 Jan 2026 18:13 UTC

  1 point

  0

  in reply to: Derek DeHart’s comment on: Back­yard cat fight shows Schel­ling points pre­ex­ist language

  “non-verbal bargaining is the default from which explicit bargaining emerges when it’s insufficient”

  The point I’m gesturing at is more abstract than this—it’s not that tacit bargaining is literally prior to explicit bargaining, in the sense that everyone always starts tacit and then resorts to explicit if that fails. I’m saying that tacit bargaining is “conceptually” prior to explicit bargaining; i.e.:

  • We can observe a world where tacit bargaining takes place without even the possibility of explicit bargaining (i.e. the world of cats).

  • However, (I claim) we cannot imagine the opposite. Explicit bargaining cannot take place without the background possibility of tacit bargaining, because otherwise the words used in the attempted explicit bargaining would have no meaning.

  Is there an example you can share between humans who otherwise could engage in explicit bargaining that reinforces the non-verbal bargaining you observed in the cats?

  I haven’t actually read this, but there’s a book on this subject, The Territorial Imperative, whose Wikipedia summary is interesting and provocative. The author argues that human territorial behavior is essentially the same as that of animals, and so the verbiage we pile on top of it is less relevant to understanding what goes on in human conflicts than observations of animals.

Back­yard cat fight shows Schel­ling points pre­ex­ist language

Cross­word Hal­loween 2025: Man­made Horrors

• jchan 13 Oct 2025 17:01 UTC

  3 points

  −1

  in reply to: MichaelDickens’s comment on: Sublin­ear Utility in Pop­u­la­tion and other Un­com­mon Utilitarianism

  so your qualia weren’t that valuable

  The OP isn’t making any claim like this; the question isn’t whether any particular experience has value in-and-of-itself, but is only making a claim about the correct way to evaluate the total utility in a world with multiple experience.

  By analogy, consider special relativity: If a train is moving at 0.75c relative to the ground, and a passenger on the train throws a ball forward at 0.5c, then that means the ball is moving at 0.91c relative to the ground. But there is no reference frame in which the ball is “really” moving at 0.16c.

  Or, more pertinently, suppose we have two identical simulations playing out at the same time. Each one contributes zero marginal utility to a world in which the other one exists (and might be told this by Omega), but that doesn’t mean that the two of them together have zero utility.

• jchan 13 Oct 2025 16:46 UTC

  3 points

  0

  on: Sublin­ear Utility in Pop­u­la­tion and other Un­com­mon Utilitarianism

  I’ve had exactly the same thought before but never got around to writing it up. Thanks for doing it so I don’t have to :-)

  There are only so many possible human shaped computations that are valuable to me

  I would surmise that value-space is not so much “finite in size” but rather that it fades off into the distance in such a way that it has a finite sum over the infinite space. This is because other minds are valuable to me insofar as they can do superrationality/​FDT/​etc. with me. In fact, this is the same fading-out function as in the “perturb the simulation” scenario; i.e.:

  • $V_A\left(AB\right)$ := The value that $A$ places on a world where $A$ and $B$ both exist

  • $V_A\left(A\overline{B}\right)$ := The value that $A$ places on a world where $A$ exists but $B$ doesn’t

  • $V_A\left(\overline{A}B\right)$ := The value that $A$ places on a world where $A$ doesn’t exist but $B$ does

  • Claim: $V_A\left(AB\right)-V_A\left(A\overline{B}\right)=V_A\left(A\overline{B}\right)-V_A\left(\overline{A}B\right)$

  ---

  However, the main problem with this perspective is what to do with quantum many-worlds. Does this imply that “quantum suicide” is rational, e.g. that you should buy a lottery ticket and set up a machine that kills you if you don’t win? This is bullet I don’t want to bite (so to speak...)

Austin LW Cafe Meetup Oct. 4, 2025: “If Any­one Builds It, Every­one Dies” Discussion

• jchan 29 Sep 2025 22:09 UTC

  1 point

  −2

  in reply to: AnthonyC’s comment on: How does the cur­rent AI paradigm give rise to the “su­per­a­gency” that IABIED is con­cerned with?

  Re 1a: Intuitively what I mean by “lots of data” is something comparable in size to what ChatGPT was trained on (e.g. the Common Crawl, in the roughly 1 petabyte range); or rather, not just comparable in disk-space-usage, but in the number of distinct events to which reinforcement learning is applied. So when ChatGPT is being trained, each token (of which there are a ~quadrillion) is a chance to test the model’s predictions and adjust the model accordingly. (Incidentally, the fact that humans are able to learn language with far less data input than this suggests that there’s something fundamentally different in the way LLMs vs. humans work.)

  Therefore, for a similarly-architected AI that generates action plans (rather than text tokens), we’d expect it to require a training set with a ~quadrillion different historical cases. Now I’m pretty sure this already exceeds the amount of “stuff happening” that has ever been documented in all of history.

  I would change my opinion on this if it turns out that AI advancement is making it possible to achieve the same predictive accuracy /​ generative quality with ever less training data, in a way that doesn’t seem to be levelling off soon. (Has work been done on this?)

  Re 2a: Accordingly, the reference class for the “experiments” that need to be performed here is not like “growing cells in a petri dish overnight”, but rather more like “run a company according to this business plan for 3 months and see how much money you make.” And at the end you’ll get one data point—just 999,999,999,999,999 to go...

  Re 2b:

  What do you see as the upper bound for what can, in principle, be done with a plan that an army of IQ-180 humans (aka no better qualitative thinking than what the smartest humans can do, so that this is a strict lower bound on ASI capabilities) came up with over subjective millennia with access to all recorded information that currently exists in the world?

  I’ll grant that such an army could do some pretty impressive stuff. But this is already presupposing the existence of the superintelligence whose feasibility we are trying to explain.

  Re 3c/​d:

  I haven’t looked into this or tried it myself. (I mostly use LLMs for informational purpose, not for planning actions). Do you have any examples handy of AI being successful at real-world goals?

  (I may add some thoughts on your other points later, but I didn’t want to hold up my reply on that account.)

  ---

  Stepping back, I should reiterate that I’m talking about “the current AI paradigm”, i.e. “deep neural networks + big data + gradient descent”, and not the capabilities of any hypothetical superintelligent AI that may exist in the future. Maybe this is conceding too much, inasmuch as addressing just one specific kind of architecture doesn’t do much to alleviate fear of doom by other means. But IABIED leans heavily on this paradigm in making its case for concern:

  • the claim that AIs are “grown, not crafted”

  • the claim that AIs will develop desires (or desire-like behavior) via gradient descent

  • the claim that the risk is imminent because superintelligence is the inevitable endpoint of the work that AI researchers are currently doing, and because no new fundamental breakthroughs stand in the way of that outcome.

[Question] How does the cur­rent AI paradigm give rise to the “su­per­a­gency” that IABIED is con­cerned with?

• jchan 28 Jul 2025 21:47 UTC

  1 point

  1

  in reply to: AnnaJo’s comment on: What are non-ob­vi­ous class mark­ers?

  skiing

  It seems like skiing is a “hereditary” class marker because it’s hard to learn how to do it as an adult, and you’re probably not going to take your kids skiing unless you yourself were taught as a kid, etc.

An­thropic rea­son­ing in­tro (notes on Bostrom)

• jchan 21 Jun 2025 15:57 UTC

  4 points

  2

  in reply to: Knight Lee’s comment on: The Boat Theft The­ory of Consciousness

  trying to imagine being something with half as much consciousness

  Isn’t this what we experience every day when we go to sleep or wake up? We know it must be a gradual transition, not a sudden on/​off switch, because sleep is not experienced as a mere time-skip—when you wake up, you are aware that you were recently asleep, and not confused how it’s suddenly the next day. (Or at least, I don’t get the time-skip experience unless I’m very tired.)

  (When I had my wisdom teeth extracted under laughing gas, it really did feel like all-or-nothing, because once I reawoke I asked if they were going to get started with the surgery soon, and I had to be told “Actually it’s finished already”. This is not how I normally experience waking up every morning.)

• jchan 17 Jun 2025 13:26 UTC

  9 points

  8

  in reply to: sunwillrise’s comment on: sun­willrise’s Shortform

  • I think this is mostly just the macro-trend of the internet shifting away from open forums and blogs and towards the “cozy web” of private groupchats etc., not anything specific about LessWrong. If anything, LessWrong seems to be bucking the trend here, since it remains much more active than most other sites that had their heyday in the late 00s.

  • I don’t have any dog in the Achmiz/​Worley debate, but I’m having trouble getting in the headspace of someone who is driven away from posting here because of one specific commenter.

    • First of all, I don’t think anyone is ever under any obligation to reply to commenters at all—simply dropping out of a conversation thread doesn’t feel rude/​confrontational in the way it would be to say IRL “I’m done talking to you now.”

    • Second, I would find it far more demotivating to just get zero engagement on my posts—if I didn’t think anybody was reading, it’s hard to justify the time and effort of posting. But otherwise, even if some commenters disagree with me, my post is still part of the discourse, which makes it worthwhile.

• jchan 12 Jun 2025 13:42 UTC

  5 points

  1

  in reply to: Viliam’s comment on: Reli­gion for Rationalists

  I think this approach wouldn’t work for rationalists, for two reasons:

  • The rationality community is based around disputation, not canonicalization, of texts. That is, the litmus test for being a rationalist is not “Do you agree with this list of propositions?” (I have tried many times to draw up such a list, but this always just leads to even more debate), but rather “Are you familiar with this body of literature and do you know how to respond to it?” The kind of person who goes to LW meetups isn’t going to enjoy simply being “talked at” and told what to believe—they want to be down in the arena, getting their hands dirty.

  • Your “recommended template” is essentially individualistic—participants come with their hopes and desires already in-hand, and the only question is “How can I use this community to help me achieve my goals?” Just as a gut feeling I don’t think this is going to work well in building a community or meaningful relationships (seeing others not merely as means, but as ends in themselves—or something like that). Instead, there needs to be some shared purpose for which involvement in the community is essential and not just an afterthought. Now, this isn’t easy. “Solving AI alignment” might be a tall order. But I think the rationality community is doing a passable job at one thing at least—creating a culture of high epistemic standards that will be essential (for both ourselves and the wider world) in navigating the unprecedented challenges our civilization faces.

• jchan 5 Jun 2025 15:00 UTC

  4 points

  2

  in reply to: HoVY’s comment on: How the veil of ig­no­rance grounds sentientism

  Can’t speak for Said Achmiz, but I guess for me the main stumbling block is the unreality of the hypothetical, which you acknowledge in the section “This is not a literal description of reality” but don’t go into further. How is it possible for me to imagine what “I” would want in a world where by construction “I” don’t exist? Created Already in Motion and No Universally Compelling Arguments are gesturing at a similar problem, that there is no “ideal mind of perfect emptiness” whose reasoning can be separated from its contingent properties. Now, I don’t go that far—I’ll grant at least that logic and mathematics are universally true even if some particular person doesn’t accept them. But the veil-of-ignorance scenario is specifically inquiring into subjectivity (preferences and values), and so it doesn’t seem coherent to do so while at the same time imagining a world without the contingent properties that constitute that subjectivity.

• jchan 29 May 2025 16:55 UTC

  1 point

  −2

  in reply to: Lorec’s comment on: What should I read to un­der­stand an­ces­tral hu­man so­ciety?

  I think ancient DNA analysis is the space to watch here. We’ve all heard about Neanderthal intermixing by now, but it’s only recently become possible to determine e.g. that two skeletons found in the same grave were 2nd cousins on their father’s side, or whatever. It seems like this can tell us a lot about social behavior that would otherwise be obscure.