I’ve formerly worked for MIRI and what’s now the Center on Long-Term Risk; I’m now making a living as an emotion coach and Substack writer.
Most of my content becomes free eventually, but if you’d like to get a paid subscription to my Substack, you’ll get it a week early and make it possible for me to write more.
Note that the Internet Archive and Archive Today are different services.
I think that didn’t tag/notify him but @geoffreymiller does, in case he wants to participate in the discussion.
Looking at the karma for some of my recent posts, “publish more by just writing about anything that happens to interest you and don’t try to predict whether it’ll interest the readers” sure is a high-variance strategy (as you might have expected).
Agree. My first thought was something like “belief correlation” or “belief interconnectedness”, as in “I forgot that beliefs are correlated”. Also a vague reference to everything is correlated.
It seems safe to assume that most YC companies were not using it much before the launch of ChatGPT (if only because the technology wasn’t available)
Strictly speaking the technology was available (I got a startup that I consulted for to adopt GPT-3 roughly a year before ChatGPT happened). That said, it wasn’t very widely known, so your take still seems like a reasonable approximation.
Wikipedia articles have traditionally been emphasized in LLM training. OpenAI never told us the dataset used to train GPT-4 or GPT-5, but the dataset used for training GPT-3 involved 3.4 repetitions of Wikipedia.
The Pile also has English Wikipedia repeated three times, which is a higher multiplier than any other subcomponent.
Being defensive can certainly mean behaviors that go that extreme but I’ve seen it used to cover much milder and more acceptable behaviors too, such as merely insisting on one’s innocence in a way where you are unwilling to admit having done any single thing wrong.
Claude Sonnet 4′s explanation of what “acting defensive in conversation” means
When someone is described as acting defensive, they’re typically engaging in behaviors that protect themselves from perceived criticism, blame, or attack. This usually involves:
Deflecting responsibility—They might redirect blame to others, make excuses, or refuse to acknowledge their role in a problem. Instead of saying “I made a mistake,” they might say “Well, you didn’t give me clear instructions” or “Anyone would have done the same thing.”
Counterattacking—Rather than addressing the issue raised, they turn the focus back on the person confronting them. If criticized for being late, they might respond with “You’re always nitpicking” or bring up the other person’s past mistakes.
Minimizing or denying—They downplay the significance of their actions or outright deny that something happened. “It wasn’t that big a deal” or “I never said that” are common defensive responses.
Emotional escalation—Their tone may become angry, hurt, or indignant. They might raise their voice, become sarcastic, or shut down entirely.
Justifying extensively—They provide lengthy explanations for why their behavior was reasonable or necessary, often missing the actual point being raised.
Taking things personally—They interpret feedback about specific actions as attacks on their character or competence.
Defensiveness usually stems from feeling threatened, vulnerable, or ashamed. While it’s a natural protective response, it often prevents productive communication and problem-solving because the person isn’t really listening to or engaging with the concerns being raised.
When someone is showing you a fear that looking at the evidence will leave you thinking they’re bad, this implies a belief that they are indeed bad—at least by your interpretation, which is obviously the one that matters to you. [...]
Because the only way that fear can be reflectively stable is if they are guilty.
I think this is assuming that the people looking at the evidence can be trusted to make a fair and impartial assessment of it and not jump to any unjustified conclusions?
I do agree that if someone has strong reasons to believe that, and to believe that nobody will be motivated to take any of the information out of context and paint them in a bad light later, etc., then hiding information only makes sense if you are in fact guilty.
But it’s very often the case that people don’t have reason to feel that secure, and have cause to believe that at least part of their audience will jump to conclusions, have all kinds of hostile motives, be inclined to treat one party’s word as intrinsically more trustworthy than the other’s, not have the time or interest to really think it through, etc..
In the kinds of examples that I gave in the original post where I’d gotten defensive despite not feeling guilty, it was exactly because the other party gave signs that they were not inclined to consider the evidence in a balanced way—if they wanted to listen to it at all.
Even if a person I’m talking to trusts me to fairly consider the evidence, if there are any other people witnessing the conversation, those others might still have hostile motives, making my interlocutor defensive. So it’s not even the case that they necessarily expect the evidence to make them look bad by my interpretation, they can expect it to make them look bad by someone else’s interpretation.
Yeah, I’m starting with this part of your response because I agree and think it is good to have clear messaging on the most unambiguously one-directional (“guilty or not”) pieces of evidence.
That’s a cool conversational move! Appreciate it.
What shouldn’t happen is that onlookers give someone a pass because of reasoning that goes as follows:
Agree. When I wrote the post, I was thinking more of a case where someone does respond to the object-level claims but in a defensive way or with non-object-level arguments mixed in, not of a case where they entirely fail to present object-level-arguments.
Basically, the asymmetry is that innocent people can often (though not always) disclose information voluntarily that makes their innocence more clear/likely. That’s the best strategy if it is available to you. It is never available to guilty people, but sometimes available to innocent people.
I suspect we might disagree on exactly how frequently this strategy is available to innocent people. I do agree that it is sometimes available to innocent people, but there are also lots of situations where e.g. the innocent person can’t offer any solid evidence that their version of the story is the correct one, or where they have some other reason not to share the full truth (e.g. protecting someone else’s privacy or truth-telling requiring them to reveal something unrelated that they are embarrassed by or have a legal obligation not to reveal), or where the truth is complicated or unusual enough that third parties might not believe it, etc.
Also, as long as the innocents are not fully convincing, many people might go “I can’t tell who is telling the truth here so just out of caution I’ll distrust everyone involved”, which gives even innocent people a motive to leverage whatever extra weapons they have to increase the chances of being believed (or equivalently, the accuser not being believed).
Justifiably accused “problem people” will almost always attempt counterattacks in one form or another (if not calling into question the accuser’s character, then at least their mental health and sanity) because they work so well as deflection.
Agree. But a relevant question is, do innocent people attempt counterattacks at a significantly lower rate? If both innocent and guilty people are roughly equally likely to attempt counterattacks, then just the presence of a counterattack isn’t strong evidence. And as long as a counterattack is not less effective for an innocent person, you’d expect both innocent and guilty people to have a similar incentive to launch them.
WRT your last paragraph, I agree with your examples and think the difference probably comes from us thinking about different kinds of examples.
After all, if you weren’t guilty, you wouldn’t need to defend yourself
This seems wrong to me? If someone says “X abused me” and X says nothing to defend themselves or refute it, people are likely to take the lack of a defense as an admission of guilt.
When you say defensiveness, does that include something like “act as though you’ve been attacked viciously by a person who is biased against you because they’re bad”?
Yeah.
The problem with the “immediately focus on maximally discrediting the accusers” is that is that it is awfully close to the tactic that actually guilty people might want to use to discredit or intimidate their accusers
Agree. But it’s also a strategy that innocent people might want to use to show that the people accusing them don’t have clean motives, or just something that they do automatically to defend themselves because they’re under stress and it does work as a general-purpose defense strategy. So it doesn’t seem like clear Bayesian evidence one way or the other?
I haven’t thought this through in detail but my first thought would be to suspect that this is a strategy that weakly favors people who are actually innocent, assuming that the audience is reasonably discerning and it doesn’t just degenerate into a popularity contest. In that while you can of course dig up dirt on anyone, being able to find accusation-relevant dirt (“this police accusing me has been known to take bribes and accuse innocent people before”) seems more likely to happen in cases where you are in fact falsely accused.
Of course, if that’s the only defense they offer and they don’t bother refuting any of the actual accusations in any substantial way, that’s certainly very suspicious. But then the suspicious thing is more the lack of an object-level response rather than the presence of a defensive response.
Maybe? My mental model of crackpots involves them writing very long manifestos.
If that makes a user unrealistic, then I’m unrealistic!
I was amused when Claude Opus abruptly stopped generating a reply to me and shut down the chat when I had asked it how a fictional galactic empire might control its frontier planets. Given that it stopped generating in the middle of a sentence that was talking about “biological monitoring” and “enhanced”, I surmised that the reference to the genetically engineered catboys/catgirls in the setting had triggered its bioengineering filters.
You seem to be missing the point here. Presumably we have the capacity to suffer because it facilitated our survival somehow. How are you so sure you don’t need to hear the message suffering was sending you?
FWIW, my current model is that “meditation removes your capacity to suffer” is not quite right. Rather, suffering is an error signal indicating that something like a prediction error is happening. Meditation reduces suffering by causing those prediction errors to get fixed.
You know how, when trying to understand a complicated phenomenon, it’s often too hard to start out with a full model of it? So you start with one that has lots of simplifying assumptions, and then see if you could gradually drop those?
I think that it’s similar with the brain. Evolution has hardwired it with some priors about the nature of reality that help bootstrap its reality- and self-modeling and motivation. Those assumptions are reasonably close to correct, but somewhat off. Among other things, what lsusr calls “desire” seems to involve creating temporary false beliefs. E.g., if you think “ugh, I really want this boring day to be over”, I think that is on some level implemented by the brain trying to overwrite the “the day isn’t over yet” data with an experience of the day finally ending.
Since these modeling assumptions cause the brain to make flawed predictions, they generate errors as the prediction is compared against the data and mismatches are detected. Whenever this happens, either on a low level (in the case of assumptions like “objects are discrete and have clear boundaries rather than being continuous in the raw data”) or on a higher level (with conscious desires like “I want this boring day to be over”), it creates a tension or error signal that is subjectively experienced as suffering.
One obvious way by which brains try to minimize that error signal is by avoiding situations that would trigger it. If something frequently causes you to suffer a lot, you likely don’t want to keep repeating it (unless this is saving you from an even greater suffering).
Alternatively, if one manages to attend to the error signal without trying to change it, it will somehow (I don’t have a good model of what exactly happens here, maybe it’s some more biologically realistic analogue of backpropagation) cause the brain to shift its priors to better match the data. This is what happens in meditation. When the priors get adjusted or moved from a category like “absolute constraints” to “intermediary modeling assumptions”, this removes a source of error and therefore also reduces suffering.
So meditation doesn’t break the capacity to suffer. The capacity for suffering, as in the ability for the brain to notice when its predictions are off, is retained or even enhanced. It’s just that as the brain refactors itself to make more accurate predictions, the number of things that would create that error signal drops drastically.
I think it’d be more accurate to say that “there’s this other factor too” rather than “this analysis is not valid”?
There are a number of comments expressing disagreement that have gotten a fair number of upvotes, so it doesn’t look to me like expressing disagreement would be unwelcome.
Edited to add: I should also mention that I don’t think this comment came out of “abstract analysis”. It came from the fact that back when I banned Eugine Nier, I then reached out to a user who had left the site because of him to let them know their harasser was banned. The user’s response was basically, “glad to hear, but I still don’t feel like coming back”. So at least in one previous case, users who had left because of a now-banned user were actually permanently out of the resulting discussion.
Thank you for your hard work! Neither the decision itself nor the work of justifying it and discussing it is particularly easy, as I can say from experience. I appreciate you putting so much effort into trying to keep the site healthy.
This post has comments from some people who agree and from some people who disagree with the decision. It seems worth making explicit that this discussion may underrepresent the amount of people who agree, because some of the people with the strongest agreement would be the ones who’ve already left the site because of Said.
Reminds me of the classic “Rakkaus matemaattisena relaationa” (Love as a mathematical relation).
Partial English translation by GPT-5
Definition 1. Let be the set of all people and Define the relation “” by
loves .
Remark 2. There are many mutually equivalent definitions for the notion “ loves ”. Invent and prove equivalent definitions.[1]
Remark 3. “” is not an equivalence relation.
Proof. It would suffice to observe the lack of reflexivity, symmetry, or transitivity, but we’ll do them all to highlight how difficult a relation we’re dealing with.
Not reflexive: Clearly “” for some self-destructive .
Not symmetric: From it does not necessarily follow that . Think about it.
Not transitive: From and does not follow that , since if it did, nearly all of us would be bisexual.
Remark 4. The lack of transitivity also implies that the relation “” is not an order.
For it to work well in practice, the relation “” would need at least symmetry and transitivity. Reflexivity is not necessary, but a rather desirable property. However, as is common in mathematics, we can forget practice and study partitions of sets via the relation. We soon notice that the lack of symmetry allows very different kinds of sets.
We begin our investigation by looking at inward-warming sets. Then we can restrict ourselves to sets whose size is greater than two. To simplify definitions we assume that no one loves themselves. In other words, we silently assume that everyone loves themselves, so it can be left unmarked separately.
Convention 5. From now on, assume and , unless otherwise stated. Also assume that for all we have .
If the notation in this section feels complicated, read Section 3 and then take another look. Drawing pictures often helps. In mathematics it’s really just about representing complicated things with simple notations—whose understanding nonetheless requires several years of study.
Definition 6. We say that a set is
1. inward-warming if, for
,
we have .
2. strongly inward-warming if the set
is a nonempty subset of .
3. together if for every there exists with and , and in addition the set
is a subset of .
4. a hippie commune if for all we have .
Remark 7.
i) A set that is together is strongly inward-warming. Conversely, a strongly inward-warming set is inward-warming.
ii) In a hippie commune, the relation “” is an equivalence relation.
The definitions in the previous section admittedly look a bit complicated. For that we develop a bit of terminology:
Convention 8. From now on, also assume with , unless otherwise stated.
Definition 9. We say that is, in the set ,
cold if there is no with .
lonely if there is no with .
popular if there exists with .
warm if there exists with .
a recluse if it is cold and lonely in .
a player if it is popular and warm in .
a slut if there exist with and as well as .
a hippie if for all .
loyal in if there exists exactly one with . Then we say that is loyal to .
’s partner in if and . Then is called a couple, and contains a couple. The couple is a strong couple in if is loyal to in and is loyal to in .
Remark 10. Every hippie is a slut.
Using these definitions, we can rewrite the earlier concepts:
Theorem 11. A set is
inward-warming if and only if there exists at least one that is warm in .
strongly inward-warming if every is cold in and there exists at least one that is warm in .
together if every is warm in and cold in .
a hippie commune if is a couple in for all .
Proof. Exercise. □
[...]
It becomes easier to visualize different sets when the relationships between elements are represented as graphs. We call the elements of the set the vertices of the graph and say that there is an edge between vertices and b if . Since “” is not symmetric, the graphs become directed. Thus we draw the set’s elements as points and edges as arrows. In the situation , we draw an arrow from vertex to vertex . Figure 1 shows a simple situation where loves , and Figure 2 shows the definitions from Section 3 and a few more complicated situations.
Convention 28. We will henceforth call the graphs formed via simply graphs.
[...]
In many cases it is helpful for visualization to color the vertices of graphs so that “neighbors” always differ in color. Since blue is the boys’ color and red the girls’ color, we want to color graphs with only two colors.
Definition 33. A vertex is a neighbor of a vertex if .
Definition 34. A vertex is hetero if all its neighbors are of a different gender than . A pair is a hetero couple if and are of different genders.
Definition 35. A graph is a rainbow if it contains a vertex that is not hetero.
Remark 36. The name “rainbow” comes from the fact that a rainbow graph cannot be colored with two colors so that neighbors always differ in color.
Definition 37. A path (from to ) in a graph is an ordered set {} whose elements satisfy or for all . The path is a cycle if . The length of a path/cycle is .
Theorem 38. If a graph contains a cycle of odd length, it contains a vertex that is not hetero.
Proof. Exercise.
Hint: Help can be sought from rock lyrics or lager beer.
As far as I understand, Archive Today circumvents paywalls without permission.