MichaelDickens

Karma: 2,471

MichaelDickens 29 Apr 2026 17:36 UTC
2 points
0
in reply to: DirectedEvolution’s comment on: Simon Lermen’s Shortform

These “put your money where your mouth is” arguments are turning into an unbelievably stupid form of rhetoric.

Seems harsh. At worst they are a misguided in a way that’s subtle enough that intelligent people still get the wrong answer.

MichaelDickens 29 Apr 2026 17:35 UTC
6 points
2
in reply to: Simon Lermen’s comment on: Simon Lermen’s Shortform
This requires having more views about worlds where money is most valuable. There’s a U-curve where
- excellent policies pass: we’re in the clear. low value of money
- decent policies pass, or maybe they fail by a thin margin: more work required. high value of money
- nothing is even close to passing: we’re doomed. low value of money
It’s complicated to determine where the peak of the U-curve is, or even which side of it you’re on. I think I’d have a hard time figuring out which side of a bet to take to maximize expected value.

For example, it’s not obvious to me that the doomer should prefer to bet YES on a datacenter moratorium getting passed.

MichaelDickens 27 Apr 2026 18:45 UTC
15 points
3
on: MichaelDickens’s Shortform
There are some people in AI safety who I feel the most aligned with, and who I think are doing some of the most important work to advocate for AI x-risk being a big deal, and doing the best job of resisting the siren call of AI capabilities work. I’ve noticed a pattern where these people tend to have a hair trigger for who’s behaving unethically and will aggressively call people out for doing things that I don’t see as that big a deal. In fact some of these people have called each other out.

My best guess for what’s going on here is that the sort of person who advocates strongly for what they believe in will sometimes have false positives, and they will advocate strongly for those false positives, too.

There are also people who are very careful and analytical, and who don’t start fights, but they aren’t leading advocacy efforts either. I can think of a couple examples of people who do both, but they’re exceedingly rare.

MichaelDickens 27 Apr 2026 18:25 UTC
6 points
0
on: Update on the Alex Bores campaign
- I appreciate the practice of transparently updating previous recommendations with new information!
- On the object level, without having though deeply about it, I like your reasoning. The points about LTF’s theory of change and how this affects the value of donations is both non-obvious in prospect and easy to follow in retrospect.

MichaelDickens 27 Apr 2026 18:21 UTC
2 points
0
in reply to: PeterMcCluskey’s comment on: Claude’s Constitution
Tell me if this is an accurate description of your reasoning:
1. I thought it was not feasible to mix corrigibility with value alignment—we should aim for CAST instead.
2. I saw how Claude’s Constitution tries to mix corrigibility with values.
3. I don’t necessarily think the constitution is doing a good job at that, but it made me realize that I was too hasty to rule out the feasibility of mixing corrigibility with values.

MichaelDickens 26 Apr 2026 19:32 UTC
3 points
1
on: Did anyone here used to hate exercise and learned to not hate it?
Other commenters have some ideas for how to make exercise enjoyable. Another possibility is to optimize for making exercise short:
- Do high-intensity intervals once a week, for 15–30 minutes. Example: 5 intervals of 30 seconds sprinting + 1 minute walking. Long-duration exercise is better, but interval exercise is more time-efficient.
- Do a full-body weight routine once a week. You can see strength gains with only one heavy set per muscle group per week. Should take less than 30 minutes to do three exercises (e.g. squats + bench press + pull-ups / machine pull-downs).
That’s less than an hour of exercise per week, but it’s enough to give you a solid base.

MichaelDickens 26 Apr 2026 18:37 UTC
3 points
0
on: A problem with the most recently published version of CEV
This issue with PCEV runs into a general problem with alignment targets: should you aim for what’s objectively good, or for what agents can agree on?

(I’m going to go out on a limb and say that the fanatics’ preferences in your thought experiment are objectively bad.)

You can make claims like “PCEV could result in an objectively bad(ish) outcome due to fanatics’ preferences”, in which case why not say fanatics are excluded from PCEV? Why not just lean into doing the thing that’s objectively good?

Pragmatically, the problem with excluding people from CEV (or baking in a morality/axiology) is that it makes it harder for people to agree on an alignment target, and you might end up with people warring over an alignment target—and the war could be catastrophically bad. But fanatics are sufficiently unpopular that it seems fine in this case. In fact I would guess that zero humans’ CEV[1] is fanatical in this way—people who advocate for eternal torture are confused, and would not endorse this upon reflection.

But this introduces the more complicated question of “what pragmatics/special cases should be considered?” That question makes theoretical work hard, although e.g. Claude’s Constitution is basically a giant list of (often internally contradictory) special cases. I don’t think the way Claude’s Constitution specifies an alignment target will scale to ASI because the contradictions become untenable.

Separately, I think the fanatics problem is unlikely to matter in practice because (1) true fanatics are rare and therefore their vote counts for little; (2) Hedonistic Imperative-style welfare-optimized minds probably have symmetric-ish happiness and suffering, unlike in evolved beings where max suffering >>> max happiness; and this dampens how bad it is to introduce suffering as the result of a negotiation. Although I’m not overwhelmingly confident about either of those points.

[1] insofar as individual humans have a CEV, which actually I don’t think most people do, or at least you need some method of resolving internal contradictions. resolving contradictions is impossible in theory, but in practice it still happens somehow (sometimes). but that’s a whole other issue

MichaelDickens 26 Apr 2026 18:20 UTC
4 points
0
on: Coherent extrapolated volition (alignment target)
The arguments about which entities to include or exclude seem to contradict each other, or don’t really justify their positions. Examples:
- Says we should include powerless humans so as not to be a jerk. Isn’t it similarly jerkish to exclude powerless non-humans?
- Says to exclude non-human sapients because “they aren’t here to protest”. Well neither are powerless humans!
- Says we should exclude mammals because they might not be moral patients. How do I know other humans are moral patients?
- Says we should exclude mammals because they might have strange preferences. Okay, so then we should also exclude fundamentalist Christians and Muslims who want heretics to burn in hell forever.
- “Including mammals into the extrapolation base for CEV potentially sets in stone what could well be an error, the sort of thing we’d predictably change our minds about later.” – I could similarly argue that we should exclude any humans who don’t care about animal welfare. Including those humans could potentially set in stone bad outcomes for animals, and later I’d predictably have preferred to exclude those people.
The only argument that seems to me to have force is “avoid a slap-fight over who gets to rule the world”. The argument for excluding particular (plausibly-)moral patients is that if you try to include them, you might be conquered by someone else who doesn’t include them, and get a worse ultimate outcome.

MichaelDickens 26 Apr 2026 1:07 UTC
4 points
2
on: Claude’s Constitution
By my read, you’re updating your beliefs (somewhat) away from “corrigibility should be the alignment target” and toward “constitutional AI will work”. What is the reason for that update? As far as I can tell, the evidence we have is basically (1) Anthropic is trying to align AI via the Constitution; (2) constitutionally-aligned Claude scores pretty well on superficial “alignment” benchmarks. I take this as basically epsilon evidence that Anthropic’s strategy will work for superintelligence, so I want to hear more about what evidence you’re updating on.

MichaelDickens 24 Apr 2026 23:07 UTC
2 points
0
in reply to: MichaelDickens’s comment on: Manifund’s Falcon Fund
This is something I’ve written about before (e.g. Which types of AI alignment research are most likely to be good for all sentient beings?) but there are LessWrong regulars who could provide much better insight than I can.

MichaelDickens 24 Apr 2026 23:00 UTC
4 points
2
on: Manifund’s Falcon Fund
Some things I believe:
- It’s good that this fund exists.
- More people should be thinking about the impact of AI on sentient beings.
- Model constitutions are unlikely to be relevant to ASI.
- More generally, pretty much everything AI x Animals people are currently working on (AFAIK) is unlikely to be relevant to ASI.
There are a lot of people on LessWrong who have a better picture of what might be relevant to ASI, and I’d like to see comments from them on what sort of direction they’d want to see for Falcon Fund or for orgs in the space.

MichaelDickens 19 Apr 2026 23:43 UTC
2 points
0
in reply to: Elliott Thornley (EJT)’s comment on: Can AI make advancements in moral philosophy by writing proofs?

Which Claude model did you use? Did you use extended thinking?

Opus 4.6. I’ve never touched my extended thinking settings but it looks like it’s off by default.

MichaelDickens 19 Apr 2026 0:39 UTC
5 points
0
in reply to: habryka’s comment on: Vladimir Putin’s CEV is probably not that bad
Thinking out loud here but I can see a classification of preferences into two types:
1. if it didn’t make me feel good, I wouldn’t want it anymore
2. if it didn’t make me feel good, I’d still want it
Watching movies is a type 1 preference (with some exceptions): people want to watch movies that they enjoy watching. If a movie is bad, I’ll stop watching it.

For me, donating money to EA causes is a type 2 preference. It doesn’t make me feel good, but it’s still important to me. Parents taking care of their children is a type 2 preference: a lot of times it sucks, but they do it anyway.

I think “lording power over others” is more of a type 1 preference.

That said, I would rather not bet the fate of the world on me being right about this.

MichaelDickens 18 Apr 2026 23:48 UTC
1 point
−5
in reply to: Thane Ruthenis’s comment on: Vladimir Putin’s CEV is probably not that bad

I think plenty of people intrinsically enjoy having power over others, and the ability to lord that power over them.

What they enjoy is the feeling of enjoyment, right? What if someone can get the feeling of lording power over others—or an even more intense version of that feeling—without actually lording power over others?

The concept of “values” isn’t clearly defined for humans, but it seems to me that it’s more accurate to say that a sadist’s “terminal value” is the feeling of enjoyment they get from power, not the power itself. The power is a means to achieve the good feelings.

For example, if a sadist started feeling awful every time they lorded power over someone, they’d probably stop doing it.

MichaelDickens 18 Apr 2026 23:39 UTC
2 points
0
on: Claude knows who you are
I tested this using two different outlines of posts I’m working on. Claude 4.7 successfully identified me based on the first outline, and declined to guess based on the second outline due to high uncertainty.

MichaelDickens 17 Apr 2026 17:25 UTC
5 points
0
in reply to: williawa’s comment on: williawa’s Shortform

lesswrong people upvoting stuff because they [...] wanna create “common knowledge” about how doomy we ought to be.

IMO a big failure of the ~2012–2020-era AI safety field was a failure to create common knowledge that AI risk is really bad actually. I think this is a common view of LWers, and upvoting doomy posts is a way to rectify this.

But I do think the problem is basically solved on LW at this point. LWers are mostly on the same page. The problem still exits elsewhere, and I think there’s value in writing about evidence of misalignment / evidence that alignment is hard / etc. as a useful thing to point to from outside LessWrong, but upvoting those posts to the moon is less useful.

MichaelDickens 16 Apr 2026 2:28 UTC
4 points
0
in reply to: ChristianKl’s comment on: Only Law Can Prevent Extinction
Fair question. Not recently but last I checked I was well calibrated on the sorts of questions that are in calibration quizzes.

MichaelDickens 14 Apr 2026 23:55 UTC
7 points
3
in reply to: Raemon’s comment on: Only Law Can Prevent Extinction
80%/90% confidence is enough to be action-guiding (IMO) but I wouldn’t call it “high”. On a scientific question where there’s good data, it shouldn’t be hard to get to 99% or even 99.9% confidence.

If you have a single study with p = 0.049, and God descends from heaven and tells you that the study had perfect methodology, then you should update your beliefs by about 5:1. That alone gets you from a 50% prior to an ~80% posterior.

The lab experiment meta-analysis (Orazani et al. 2021) found a very strong p-value, I’m just not sure how well lab experiments generalize to real life.

I will say that I don’t know that I have a good sense of how to convert a within-experiment odds update to a subjective odds update (accounting for methodology flaws, publication bias, etc.). So maybe my subjective credences aren’t good. I just have a sense that like, if violent protests worked, I would expect these studies to have had different outcomes. But I wouldn’t be extraordinarily surprised if it turned out that violent protests work after all.

MichaelDickens 14 Apr 2026 23:27 UTC
8 points
0
in reply to: Warty’s comment on: Only Law Can Prevent Extinction
I did a lit review a few months ago. My conclusions were:
- Violent protests probably don’t work (80% credence), and they plausibly backfire but it’s unclear (40% credence).
- Peaceful protests probably do work (90% credence).
However, the literature is too coarse to give good evidence on questions like, “What happens if you have a civil movement with a mixture of violent and nonviolent protests, compared to a counterfactual where all protests are nonviolent?” If I extrapolate from the narrower results that are supported by the literature, I’d guess that a pure-nonviolent movement would be most effective, but there’s no decent-quality direct evidence to my knowledge.

MichaelDickens 14 Apr 2026 23:19 UTC
2 points
0
in reply to: Linch’s comment on: Linch’s Shortform

So it’s possible we can punt this question to our future selves with AI assistance.

I suspect that it can’t be punted. e.g. see Clarifying “wisdom”: Foundational topics for aligned AIs to prioritize before irreversible decisions and Problems I’ve Tried to Legibilize