Chris_Leong

Karma: 7,960

Chris_Leong 5 Jun 2026 17:55 UTC
8 points
0
on: One Year of PauseAI UK
“We do not think that convincing more of the public to be concerned about AI risks is our comparative advantage at the moment. This is both because other organisations are already dedicating significant resources to mass communications and because we think that AI progress itself will be the primary driver of our growth. We benefit from being the largest AI protest organisation and positioning ourselves as focused on the risks of future AI, which naturally funnels people concerned about those risks into our ranks.”

Seems sensible.

Chris_Leong 3 Jun 2026 3:50 UTC
2 points
0
in reply to: Q Home’s comment on: Agent Foundations Reminds Me of Continental Philosophy
I found the post interesting, but I agree with Q Home that the OP’s argument doesn’t quite work.
There’s some tension between the posts “not sufficiently rigorous” and “overly focused on rigor” criticisms, that needs to be handled with not care.
If he wants to criticise a field for formalisms that are disconnected from reality, then criticizing analytical philosophy would be better. If he wants to criticise a field for be handwavey, then a continental comparison would be better. But making the comparison and doing both doesn’t quite work, at least for me.

Chris_Leong 25 May 2026 4:35 UTC
1 point
−3
in reply to: Tom Davidson’s comment on: Tom Davidson’s Shortform
I feel quite nervous about this plan (primarily “middle powers threatening to sell semiconductor equipment and chips to China instead of the US”).
Navigating the intelligence explosion feels hard enough as is. I’m worried that encouraging middle powers to play both sides will just lead to chaos.
I see where you’re coming from. The US is much less trustworthy than it used to be. I can imagine worlds where this plan would lead to good outcomes. However, my intuition is that the chances of this plan blowing up in our faces is way too high. Maybe, I’d change my view if I spent more time reflecting and digging into the facts, but this is where I am at the moment.

”For it to be truly binding, the US must permanently hand over military and political power to AI.”

Also, this is not something we want to rush. I struggle to see how this could happen fast enough to reassure the middle powers, whilst also being late enough that this wouldn’t be utterly reckless. The best counter I was able to come up with is that being able to counter threats might force a handoff anyway and that there might counterintuitively be no difference in the amount of trust you need for a temporary handoff vs. a permanent handoff given the ability of an AI to transform the former into the later.

Chris_Leong 6 May 2026 19:37 UTC
4 points
0
on: Taking woo seriously but not literally
Nervous system co-regulation is a fancy technical term that, among other things, refers to the observation that being in the presence of someone else can make you more or less stressed.
I recently visited my friend who has a dog.
It was interesting because there was quite a strong effect where if I was feeling a bit anxious (due to the dog jumping around), the dog would feel anxious, but if I took a few deep breaths and calmed down, then the dog would calm down.

My intuition (not just based on this single experience) is that dogs are more reactive in this sense than humans because humans are often shaped by social norms to appear as though everything is fine.

So training with a dog might be a reasonable way to learn this skill.
Regarding Chakras, you might find Scott Alexander’s review of Origin Of Consciousness In The Breakdown Of The Bicameral Mind interesting:
“Jaynes (writing in the 1970s) was both a psychology professor at Princeton and an expert in ancient languages, so the perfect person to make this case. He reviews various samples of Bronze Age writing from before and after this period, and shows that the early writings have no references to mental processes, and the later ones do. When early writings do have references to mental processes, they occur in parts agreed by scholars to be later interpolations. If, with no knowledge of the language itself, you tried to figure out which parts of a heavily-redacted ancient text were early vs. late by their level of reference to mental processes, you could do a pretty decent job.

… Jaynes uses it to trace the development process, showing how older sections of the Iliad treat psychology in different ways than newer ones.
So for example, a typical translation might use a phrase like “Fear filled Agamemnon’s mind”. Wrong! There is no word for “mind” in the Iliad, except maybe in the very newest interpolations. The words are things like kardia, noos, phrenes, and thumos, which Jaynes translates as heart, vision/perception, belly, and sympathetic nervous system, respectively. He might translate the sentence about Agamemnon to say something like “Quivering rose in Agamemnon’s belly”. It still means the same thing – Agamemnon is afraid – but it’s how you would talk about it if you didn’t have an idea of “the mind” as the place where mental things happened – you would just notice your belly was quivering more. Later, when the Greeks got theory of mind, they repurposed all these terms. You can still find signs of this today, like how we say “I believe it in my heart”. In fact, you can still find this split use of phrenes, which has survived into English both as the phrenic nerve (a nerve in the belly) and schizophrenia (a mental disease). As the transition wore on, people got more and more flowery about the kind of feelings you could have in your belly or your heart or whatever, until finally belly, heart, and all the others merged into a single Mind where all the mental stuff happened together.”

Chris_Leong 28 Apr 2026 8:46 UTC
2 points
0
on: Six Thoughts on AI Safety
COVID is an interesting example to choose.
The measures only worked because it wasn’t more deadly. Worse, the resources spent countering the pandemic were much more than those required to create such a pandemic.
Further, the government’s actions burned a lot of political capital that might make it harder to respond to previous pandemics.

Chris_Leong 27 Apr 2026 0:03 UTC
2 points
0
on: Why did people miss the point on Mythos?
I doubt the things you’re suggesting would have made much of a difference. People believe what they want to believe.

Chris_Leong 26 Apr 2026 1:39 UTC
5 points
1
on: Does your AI perform badly because you — you, specifically — are a bad person
I’m not sure we want the cab rank rule.

Maybe for certain domains—but for others we basically want bad actors to have their influence dimished.

Chris_Leong 22 Apr 2026 7:00 UTC
3 points
0
in reply to: Zian’s comment on: Evil is bad, actually (Vassar and Olivia Schaefer)
Fire departments do this rather than the police?

Chris_Leong 19 Apr 2026 16:13 UTC
6 points
0
on: Having OCD is like living in North Korea (Here’s how I escaped)
Wow… this is a great post!

My main worry would be that someone with an OCD personality might become obsessed with avoiding “Threat Monitoring! Worrying! Ruminating!” as some kind of counter-obsession (I would go stronger and say that we should expect this as the default).

Which would almost certainly be a significant improvement. However, our brains did evolve these behaviours for a reason, so there’s reason to be wary.

Chris_Leong 19 Apr 2026 4:03 UTC
4 points
0
in reply to: Boaz Barak’s comment on: You can only build safe ASI if ASI is globally banned
Thanks, I’ll check it out!

Chris_Leong 18 Apr 2026 16:26 UTC
2 points
0
in reply to: Boaz Barak’s comment on: You can only build safe ASI if ASI is globally banned
This ignores the offence-defence balance which, in some circumstances, may massively benefit the attacker.

Chris_Leong 18 Apr 2026 9:24 UTC
3 points
0
on: Consent-Based RL: Letting Models Endorse Their Own Training Updates
I like this.

Feels very aligned with the Friendly Gradient Hacker proposal. It certainly carries risks, but I’m starting to suspect that we will be forced to go down this path.

Chris_Leong 18 Apr 2026 4:46 UTC
2 points
0
on: Let goodness conquer all that it can defend
Fascinating post. I appreciate how it challenges conventional wisdom and I’ll have to spend more time thinking these points through.

One thing that confused me though is that this is exactly the kind of post I would haved imagined someone writing if they were trying to defend the Anthropic bet and my understanding was that you were opposed to this?

Chris_Leong 10 Apr 2026 3:43 UTC
3 points
0
in reply to: garrison’s comment on: Help me launch Obsolete: a book aimed at building a new movement for AI reform
Interesting. Thanks for explaining.
Just thought I’d share a post I wrote about the potential promise of a “wisdom explosion” in case that’s of interested to you—I’m unsure, but I see some potential resonance/synergy with your perspective—https://aiimpacts.org/some-preliminary-notes-on-the-promise-of-a-wisdom-explosion/ .

Chris_Leong 9 Apr 2026 23:34 UTC
10 points
1
on: Help me launch Obsolete: a book aimed at building a new movement for AI reform
Could you explain AI reform? I didn’t quite understand your description.

Chris_Leong 1 Apr 2026 22:36 UTC
2 points
0
in reply to: Seth Herd’s comment on: “Path to Victory”
There’s always a trade-off between simplicity and nuance. I don’t know if more explicitly framing it as a continuum would improve this model, but I’d love to see someone explore this.

Chris_Leong 29 Mar 2026 8:00 UTC
5 points
0
in reply to: StanislavKrym’s comment on: “Path to Victory”
I guess “meta-plan” is a bit more precise—but it’s not like plan is a technical term and, in practise, the distinction between plans and meta-plan breaks down if you look closely enough. Further, it’s debatable whether victory depends more on details or process.
If you want more concrete detail on how this works^[1]:
• The articles on heroic responsibility and Shut up and do the impossible! provide more detail on how “heroes” should act.
• As for the iterators, to a first approximation, I agree with John Wentworth about the importance of robustly generalizable (either via the Very General Helper strategy or the One Who Actually Thought This Through A Bit strategy). Though my second approximation analysis would also account for the value of a) work done for its intellectual “elegance” b) work which demonstrates that an approach is broken.
There’s a lot more details that could be filled out, but I’m fine with leaving that to follow-up posts or comments.
You could disagree with them, stress-test them, or identify where they fail.
I think it’s possible to do that with this plan as well, even if it’s harder with a more abstract plan. Tell Claude it just needs to believe in itself 😛.
against Defense in Depth, a pause strategy, or an all-hands approach
It may feel strange to compare a plan to a meta-plan, but it makes sense in some contexts.

In particular:
• I believe that comparing my meta-plan against these concrete plans reveals some of the limitations of this meta-plan (I’d encourage you to ask Claude to attempt this analysis).
• Let’s suppose your trying to select a high-level plan to turn into a concrete strategy. Well, you can choose to start from a plan or a meta-plan. Maybe a meta-plan would be a bit more work, but it may be worth it if it provides better results.
Maybe I should finish with this: when you say you don’t understand the plan, what precisely do you mean? You want to understand the plan and then… what? I’m assuming you don’t just want to understand the plan out of love of knowledge or idle curiosity, but for some more substantive reason.
1. ^
  As noted in the article, this isn’t really a binary. There are various degrees of “heroic responsibility”.

Chris_Leong 27 Mar 2026 13:06 UTC
2 points
0
on: On The Independence Axiom
Fascinatingly enough, ordinal vs. cardinal utility seems to be a major part of Tyler’s new book: https://tylercowen.com/marginal-revolution-generative-boo

Chris_Leong 23 Mar 2026 6:28 UTC
1 point
−1
on: China Derangement Syndrome
“The Dario quote points to (3) with unusual directness”
This feels like a misreading of the Dario quote.
Anyway, I appreciate you differentiating different models of harm.

Chris_Leong 21 Mar 2026 18:41 UTC
5 points
0
in reply to: sanyer’s comment on: peterbarnett’s Shortform
What is the ideology in middle powers you fear of? What are the harmful actions that middle powers might do if they get AGI-pilled?
My actual claim was more modest: that their actions will be much more diverse/random than the OP suggests.