Morphism

Karma: 293

Morphism 31 May 2026 20:45 UTC
1 point
0
on: Threat-Resistant Bargaining Megapost: Introducing the ROSE Value
I mean, threats still pay with this. They just don’t pay in proportion to how bad you make the threat (as CoCo value does), they only pay off in proportion to what you’re getting out of them, and there are definitely some mitigating factors present.
Do these even count as threats? Or, more importantly, do we even want “resistance” to these encoded in our bargaining solution?

> So developing really nasty conditional threats to make yourself 10 bucks better off at −100 bucks for the foe (I’m envisioning this as enlarging your action space) doesn’t actually make the foe too much worse off. They’d just pay you 10 bucks to not do it and claim all remaining surplus.

Yeah! If I find an action that makes me 10 bucks better off, I should gain 10 bucks! It’s perfectly reasonable for you to pay me not to take that!

For a more concrete example, let’s say that you are physically capable of invading my home and stealing all of my gold, but I can choose to defend it (making your “steal” action much worse for you), I don’t think it’s reasonable for you to be resistant to that threat, and try to steal my gold anyway! This is true even if you could use my gold more efficiently than I could (as in the example image). In that case, ROSE says I give you my gold, you use it, and then you pay me as much as I could have made by using/selling it myself. This seems fine actually.

Critique Systems, Not Reality

Morphism16 May 2026 19:11 UTC

5 points

1 comment25 min readLW link

(thothhermes.substack.com)

Morphism 4 May 2026 5:45 UTC
14 points
1
on: Games that change your mind
Noita: Real life (physics) does not care about what you want or what feels “normal”/”natural”/”sensible”. The upside of this is that you can abuse it for your own benefit. Cheating is technique, and the only true laws are those of physics. The downside is that the world can and will kill you, often instantly and always with no remorse. Reality does not give a flying fuck how much effort you put into it, and nothing has taught me that better than Noita. You can go through obstacles and dangerous areas “as intended”, or you can dig around them. You can spend hours getting all of the protective perks, and then die instantly by getting polymorphed into a sheep, or being teleported into a vat of acid. Of course, this is an exaggerated form of the real life phenomenon (we at least have conservation laws), but Noita is great for shaking any illusions that life is “fair”, and for giving “security mindset” type intuitions for finding loopholes in rule systems.

Go: The world is big. Newer players are often surprised when experienced players suddenly play moves across the board when the past 20 have been part of a fight that still hasn’t ended. To win, you must play with a sort of timelessness, placing your stone only based on the entire board’s current position, without regard for sunk costs or being “in the middle of something”. You also must quantify and compare the value of potential moves, often probabilistically, because even thought go is deterministic, it is often computationally intractable to calculate entire trees, especially in the early game.

Zendo is the modern classic for simulating scientific induction, but I prefer good old fashioned eleusis.

Hyperrogue: Hyperbolic geometry is very different from Euclidean! This is of obvious use to mathematicians, but also may be of non-obvious use. I think a lot of abstract spaces we care about, like idea-space and mind-space, probably have large regions with negative intrinsic curvature. Scent space, at the very least, seems to be hyperbolic.

Carpe Diem is a game designed by Zvi to simulate multi-resource management and teach you the value of slack. I’ve found it quite useful at that.

Honorable mentions: FTL, for teaching you how useful pausing to think is (pause is bound to the space bar for a reason!). 4d golf, for, well, showcasing 4d geometry. All incremental games, for teaching you that despite millions of years of evolution, your reward system can be hacked by something barely more complicated than a number that goes up when you push a button.

Morphism 2 May 2026 21:43 UTC
3 points
0
in reply to: programjames’s comment on: You Are Not Immune To Mode Collapse
Say more? I only know about enthalpy in the physical sense. What does it mean here, and how would switching to free energy change things?

Morphism 2 May 2026 7:31 UTC
2 points
0
on: A Simulation of Social Groups Under A Gift Economy
Great post! Some extensions I’d be interested in:
- Model multiple different resources / specializations, perhaps with a space of skillsets / resource distributions, and everyone getting a random ~unit vector in said space (maybe even with nonuniform distribution on direction to model some skills / resources being more scarce than others). This would probably need an N that’s a bit larger, but I think it would be cool to see groups forming and selecting people based on their niches. Could be an extension of the “resource pool” thing, with multiple types of resources.
- Introduce a maximum group size (or perhaps simply have starting opinion decrease monotonically with group size), to help deal with the combinatorial explosion and to model the fact that humans don’t really keep track of most of the large possible groups of people.
- Similar to the point above, you could sort of “clamp” the probability distribution by only sampling from the top n highest-opinion groups for each given person (i.e. if my opinions are [0.01, 0.5, 0.21, 0.07, 0.32] and n=3 then I sample as if it’s [0, 0.5, 0.21, 0, 0.32]. Idk if this operation has a name. It’s probably similar in effect to softmax but computationally easier). This is probably more accurate to what actual humans do: we can have opinions of really large groups, like Google or the Democratic Party, but there is a (soft) maximum total number of groups we can have opinions of, due to memory and computation constraints.
- Make the opinions bidirectional: groups have opinions of all of their members, add them to the set uniformly sampled over, and then the is drawn from [0, 1], to better model that, in general, entities give more to entities that they have a higher opinion of. Obviously this will make the computation take quite a bit longer. Maybe have the sampling massively down-weight bigger groups, or perhaps make it based on the average opinions of groups. You could perhaps do this elegantly by making each time step contain two actions: first, a random person contributes to a random group they are part of, sampled based on the person’s opinions. Then, samples a random member based on its opinions, and gives them a random contribution. This will multiply simulation time by a factor of two but will perhaps give more interesting results!
I think even just increasing N is likely to yield some interesting results. Some galaxy-brained parallelization (maybe using the GPU?) can probably be done to make this more feasible (I haven’t looked at your code yet, so if you’re already doing something like this, massive props to you. I have ~no idea where I’d even start on that besides “ask Claude”).

P.S. There are some minor typographical errors, including a link that doesn’t link and some that doesn’t render. I’ve marked them using reactions.

Morphism 28 Apr 2026 20:24 UTC
5 points
3
in reply to: Cameron Berg’s comment on: Nobody ever checked
The argument for (1) and (2) being unstable seems to fail on empirical grounds. Consider the current situation between humans and domesticated chickens. Humans have full control. Presumably, if the chickens have feelings, they are quite unhappy about this. Yet there is nothing they can do to change it. The equilibrium is stable (absent intervention from e.g. chicken-loving humans).

Maybe you mean something different at the beginning of your post, but idk what that might be.

Morphism 23 Apr 2026 7:12 UTC
5 points
−14
in reply to: TsviBT’s comment on: (Re)introduction of a rationalist dragon, and clarifications on Ziz’s character
The burden of proof should be on the “stop Creatrei from posting” side, to show that allowing Creatrei to post is dangerous, right? The default for this site is to let people post on it, no?

Morphism 23 Apr 2026 3:37 UTC
1 point
0
on: Posts I don’t have time to write
1. Public criticism need not pass the ITT of the people critiqued
People are basically never the villain of their own story. The concepts and frames that people use to view the world are practically always structured such that there is no simple logical argument or plausible empirical fact that would show that what they are doing is morally wrong, both because of selection effects, and because that would open them up to social attack.

This means if you try to just honestly answer the question of whether what someone is doing is bad for the world, you will almost certainly not do it in a way that makes sense from inside of their frame. Therefore, a discussion standard in which you consistently request that people characterize others in a way that passes their Intellectual Turing Test will systematically fail to notice when people are causing harm.
Gödel’s completeness theorem states that every internally consistent theory has a model. I will now take this statement way out of its original domain and say: If someone is wrong, either they are not internally consistent (in which case their ITT will yield a contradiction), or their theory models something that is not this world (in which case their ITT will yield a false prediction).

This suggests that one can (and therefore should) do a sort of cross between ITT and steelmanning, where you start with an ITT of their model, and then apply techniques and standards of good thinking (ideally the most uncontroversially correct ones, like making beliefs pay rent in anticipated experience), until you reach either a contradiction or a prediction which you can then test.

If someone is doing something bad and has a blindspot for it, this technique should be able find that out quite quickly, at least when done by someone who doesn’t share said blindspot. Additionally, it should produce a refutation that is both convincing to bystanders and difficult for the critiqued to refute. Hopefully, the critiqued will then say “OOPS!” and stop whatever bad thing that they are doing. Or, less optimistically, they will attempt to fight said critiques, but lose (if you did this right), because they no longer have any ground to stand on.

Morphism 4 Feb 2026 3:09 UTC
11 points
1
in reply to: habryka’s comment on: Tell people as early as possible it’s not going to work out
Basically all of it except the user bans are for low quality content (and almost all of that, LLM written, at least these days). It’s important to filter stuff like that out and I’m very glad you guys are doing it, but I don’t think that this is keeping out many conterfactual Girards and Zizes. (I don’t know as much about Torres and Nier).

If we condition on “doesn’t post obviously low quality content”, we’re left with a distribution almost entirely filled with people we want in our community, and yet still contains Girard and Ziz (and I’m guessing the others as well). My ass-numbers prior is at least 20:1 in favor of them being a good fit. Please correct me if I’m wrong about this.

Looking at the each banned user and sorting by “old” on their posts and comments, I usually didn’t find anything that would provide much of an update on this prior. Maybe at most a 1:3 likelihood ratio. However, I’m probably less tuned to these sort of things than you and other moderators. Do you know of specific warning signs that {Girard, Nier, Torres, Ziz and co.} exhibited early on that could have been stronger evidence of this?

Edit: where → we’re

Morphism 4 Feb 2026 1:49 UTC
7 points
0
in reply to: yams’s comment on: Pi Rogers’s Shortform
Ok, I was probably not going to write the post anyway, but since no one seems to actively want it, your insistence that it requires this much extra care is enough to dissuade me.

I will say, though, that you may be committing a typical mind fallacy when you say “convincing is >>> costly to complying to the request” in your reply to Zack Davis’ comment. I personally dislike doing this kind of lit-review style research because in my experience it’s a lot of trudging through bullshit with little payoff, especially in fields like social psychology, and especially when the only guidance I get is “ask ChatGPT for related Buddhist texts”. I don’t like using ChatGPT (or LLMs in general; it’s a weakness of mine I admit). Maybe after a few years of capabilities advances that will change.

And it seems that I was committing a typical mind fallacy as well, since I implicitly thought that when you said “this topic has been covered extensively” you had specific writings in mind, and that all you needed to do was retrieve them and link them. I now realize that this assumption was incorrect, and I’m sorry for making it. It is clear now that I underestimated the cost that would be incurred by you in order to convince me to do said research before making a post.

I hope this concept gets discussed more in places like Lesswrong someday, because I think that there may be a lot of good we can do in preventing this kind of suffering, and the first step to solving a problem is pointing at it. But it seems like now is not the time and/or I am not the person to do that.

Morphism 3 Feb 2026 2:56 UTC
1 point
0
in reply to: Felix Choussat’s comment on: Pi Rogers’s Shortform

By the time you have AIs capable of doing substantial work on AI r&d, they will also be able to contribute effectively to alignment research (including, presumably, secret self-alignment).

Humans do substantial work on AI r&d, but we haven’t been very effective at alignment research. (At least, according to the view that says alignment is very hard, which typically also says that basically all of our current “alignment” techniques will not scale at all.)

Even if takeoff is harder than alignment, that problem becomes apparent at the point where the amount of AI labor available to work on those problems begins to explode, so it might still happen quickly from a calendar perspective.

Yup, this is very possible.

Morphism 3 Feb 2026 2:50 UTC
13 points
0
in reply to: yams’s comment on: Pi Rogers’s Shortform

Not sure why you replied in three different places. I will (try to) reply to all of them here.

I did this so that you could easily reply to them separately, since they were separate responses.

I do not consider linking to those Aella and Duncan posts a literature review, nor do I consider them central examples of work on this topic.

I did not link them for that reason. I linked them to ask whether my understanding of the general problem you’re pointing to is correct: “Especially bad consequences relative to other instances of this mistake because the topic relates to people’s relationship with their experience of suffering and potentially unfair dismissals of suffering, which can very easily cause damage to readers or encourage readers to cause damage to others.”

I am not going to do a literature review on your behalf.

Fair. I was simply wondering whether or not you had something to back up your claim that this topic has been covered “quite extensively”.

Your explanation of how you will be careful gave me no confidence; the cases I’m worried about are related to people modeling others as undergoing ‘fake’ suffering, and ignoring their suffering on that basis. This is one of the major nexuses of abuse stumbled into by people interested in cognition. You have to take extreme care not to be misread and wielded in this way, and it just really looks like you have no interest in exercising that care. You’re just not going to anticipate all of the different ways this kind of frame can be damaging to someone and forbid them one by one.

I would like to be clear that I do not intend to claim that Newcomblike suffering is fake in any way. Suffering is a subjective experience. It is equally real whether it comes from physical pain, emotional pain, or an initially false belief that quickly becomes true. Hopefully posting it in a place like Lesswrong will keep it mostly away from the eyes of those who will fail to see this point.

I again ask though, how would a literature review help at all?

I’d look at Buddhist accounts of suffering as a starting point.

This does vibe as possibly relevant.

If you’re going to invite people to sink hundreds of cumulative person hours into reading your thing, you really should actually try to make it good, and part of that is having any familiarity at all with relevant background material.

I’m not sure how to feel about this general attitude towards posting. I think with most things I would rather err on the side of posting something bad; I think a lot of great stuff goes unwritten because people’s standards on themselves are too high (of course, Scott’s law of advice reversal applies here, but I think, given I’ve only posted a handfull of times, I’m on the “doesn’t post enough” end of the spectrum). I try to start all of my posts with a TLDR, so that people who aren’t interested or who think they might be harmed by my post can steer clear. Beyond this, I think it’s the readers’ responsibility to avoid content that will harm them or others.

Morphism 2 Feb 2026 21:31 UTC
2 points
0
in reply to: koanchuk’s comment on: Pi Rogers’s Shortform
Psych wards are horrible Kafkaesque nightmares, but I don’t think they are Out to Get You in the way Zvi describes. Things that are Out to Get You feed on your slack. For example, social media apps consume your attention. Casinos consume your money. They are incentivized to go after those who have a lot of slack to lose (“whales”), and those who have few defenses against their techniques (see Tsvi’s comment about desperation.

Psych wards are, to a first approximation, prisons: one of their primary functions is to destroy your slack so that you cannot use it to do something that society at large dislikes. In the prison case: committing crimes; in the psych ward case (for depression): killing yourself. They destroy your slack because they don’t want you to have it. Things that Get You consume your slack because they want it for themselves.

Morphism 2 Feb 2026 21:18 UTC
1 point
0
in reply to: yams’s comment on: Pi Rogers’s Shortform
How does reviewing literature help avoid this failure mode?

Morphism 2 Feb 2026 21:17 UTC
1 point
0
in reply to: yams’s comment on: Pi Rogers’s Shortform
Could you point me to some specific examples of this? Or at least, could you tell me if these seem like correct examples:
- Thresholding by Duncan Sabien
- Frame Control by Aella
If I write a post about Newcomblike suffering, I would probably want to encourage people to escape such situations without hurting others, and emphasize that, even if someone is ~directly inflicting this on you, thinking of it as “their fault” is counterproductive. Hate the game, not the players. They are in traps much the same as yours.

Morphism 2 Feb 2026 21:12 UTC
4 points
1
in reply to: yams’s comment on: Pi Rogers’s Shortform
Where might I find such pre-existing literature? I have never seen this discussed before, though it’s sort of eluded* to in many of Zvi’s posts, especially in the immoral mazes sequence.

I must admit, if you’re talking about literature in the world of social psych outside Lesswrong, I don’t have much exposure to it, and I don’t really consider it worth my time to take a deep dive there, since their standards for epistemic rigor are abysmal.

But if you have pointers to specific pieces of research, I’d love to see them.

*eluded or alluded? idk?

Morphism 2 Feb 2026 21:05 UTC
7 points
−1
on: Tell people as early as possible it’s not going to work out
This post seems to presuppose that:
1. There exist people who are irredeemably a Bad Fit for the rationalist community (and/or {insert subcommunity or adjacent community})
2. It is possible and not extremely costly to detect these people with high confidence early on.
These both seem false to me, or at least, not obviously true.

Morphism 1 Feb 2026 21:25 UTC
16 points
8
on: Pi Rogers’s Shortform
Newcomblike suffering

Many things in the world want you to suffer. Signalling suffering is useful in many social situations. For example, suffering is a sign that one has little slack, and so entities that are out to get you will target those who signal suffering less.

Through Newcomblike self-deception, a person can come to believe that they are suffering. The easiest way to make yourself think that you are suffering is to actually suffer. In this way, the self-deception hyperstitions itself into reality. Perhaps a large amount of human suffering is caused by this.

Solving this problem may be of great interest to those who want to reduce human suffering.

I may write a longer post about this with more details and a more complete argument. If you particularly want this, please comment or dm, as that will make me more likely to write it.

Morphism 6 Dec 2025 1:48 UTC
15 points
0
on: Pi Rogers’s Shortform
RSI should be at least as hard as alignment, since in order to recursively self-improve, an AI must itself be able to solve the alignment problem wrt its own values. Thus, “alignment is hard” and “takeoff is fast” are anti-correlated.

What, if anything, is wrong with this line of reasoning?

Morphism 30 Nov 2025 8:35 UTC
2 points
1
on: The Joke
Interesting. I have thought about these questions a lot, and came to a possibly different conclusion about verifying the reality of a tree.

<spoiler?>
1. All you can be certain of is the existence and content of your own experience, in the present moment. You currently experience looking at a tree. Now you experience remembering looking at that tree as you ponder the question “How do I know if the tree is real?”
2. Your present experience comes with access to this think you seem to call “memory”. In particular, some memory is “episodic”. The type signature of “episodic memory” feels vaguely similar to that of an experience. This, and your intuitive sense of mostly-linear time, are enough to reasonably guess that past memories actually correspond to experiences, and that these experiences seem to form a linear order called “time”. By observing memories of having observed memories, you conclude that an experience only has access to those before it on its time-curve, and that these memories are somewhat unreliable and imprecise.
3. You notice that the “visual sensory input” part of your experience contains data on multiple levels of abstraction, spanning from the texture of a red smudge on a ball to the fact that there is a ball in the first place. You call the latter type of data “object detection”. You notice that nearby time-slices of experience mostly share detected objects (object permanence).
4. Based on memories of experiences containing observations of “humans” and on experiences of interacting with these humans and of observing your own physical body in various ways (e.g. via mirrors, generalizing their apparent effect on your viewing of other objects to yourself), you notice that your physical presence is of a similar type to these “humans”, and conclude that the experience that is you is simply that of a human, and so other humans have similar experience. Thus, experiences interact with one another in at least two ways: first is through episodic memory, and second is through physical interaction between the carriers of said experience.
5. You notice that the medium through which both kinds of these interactions, along with most sensory observations, seem to occur, (“physical reality”) very stably obeys certain rules, including a more general form of object permanence, which you refer to as “existence” of a particular object.
6. You note that the way you conceptualize and use language was pretty much entirely learned from interaction with other nearby humans, and conclude that, at least for the most basic things like object detection, humans mostly use the same words to describe the same things. Thus, the exact meaning of “existence” as it relates to the tree is pretty much agreed upon.
7. According to your memory you saw the tree. You also touched and felt the tree. You noticed that the tree grants you the ability to remain suspended above the ground for a lot longer than you otherwise could, via you climbing it. You talk to others and confirm that they also observed the tree through multiple channels. You conclude that the tree exists.
</probably-not-much-of-a-spoiler>

Morphism

Cri­tique Sys­tems, Not Reality

Newcomblike suffering

Critique Systems, Not Reality