Joseph Miller

Karma: 3,002

Joseph Miller 16 Jun 2026 6:56 UTC
5 points
3
in reply to: habryka’s comment on: Joseph Miller’s Shortform
Maybe you could make an argument that they don’t care enough, but I’m pretty confident that MATS does care about this. When I did it, they had a whole reading group program whose primary aim, as I understand it, was to get people to understand and care more about the fundamental safety issues. And I also believe they try to select for people who care about safety, at least to some extent.

Joseph Miller 14 Jun 2026 23:52 UTC
207 points
93
on: Joseph Miller’s Shortform
The AI safety ecosystem is so well resourced that it has been correctly identified by many as one of the best paths into high prestige AI research jobs.
This person on twitter has written a popular article about getting into frontier ai labs and a Field Guide to AI Fellowships. The “AI Fellowships” are mostly AI safety programs funded by CG/OpenPhil. I have also noticed that people in ML research are quite likely to have heard of MATS and be interested in participating, even when they have very little interest in AI safety.
Idk how good or bad this is. It definitely causes a lot of ML researchers to engage with the AI safety literature, where they otherwise would not have. But it’s worth noting that while the primary driver of applications to programs like MATS used be concern about AI safety, now it is increasingly a desire to work at a frontier lab.
I used to select participants for LASR labs, one of programs listed, and we actively tried to choose people who cared about AI safety, but I think we often did not succeed and indeed some people on the program now work at frontier labs in roles that I think have little to do with safety.
Safety-washing in practice mostly looks like people rationalizing research or jobs that they would have done anyway as necessary for safety. It’s easy to do because it is in fact genuinely unclear in most cases what is helpful and harmful for safety. Only by looking at the larger pattern can we notice the suspicious abundance of conveniently overlapping opportunities that further both safety and a person’s short-term interests. However, I think it will be less common in people whose primary motivation is to prevent AI catastrophe.

Joseph Miller 10 Jun 2026 10:26 UTC
3 points
−9
in reply to: leogao’s comment on: leogao’s Shortform
I think the massive point in Christianity’s favor is that it possibly invented morality as we conceive of it.

Joseph Miller 10 Jun 2026 10:17 UTC
7 points
4
in reply to: fluxxrider’s comment on: Claude Fable 5 and Mythos 5 [Linkpost]
I think the implication from the pricing is that the main difference is that Mythos 5 is significantly smaller than Mythos Preview.

Joseph Miller 8 Jun 2026 0:31 UTC
13 points
0
on: Mental causation is not load-bearing
Here’s my one line distillation:
To explain why we know about consciousness, we do not need to explain how qualia cause physical effects. We only need to explain (intelligibly) how conscious states are determined by (supervene on) certain physical states.
Is that a fair summary?

Joseph Miller 6 Jun 2026 0:24 UTC
4 points
1
in reply to: Michaël Trazzi’s comment on: One Year of PauseAI UK
Thanks, Michael. These are good questions.
We wrote a memo which was sent to all MPs prior to the debate and drafted some of the speeches, putting us in a strong position to work with those MPs when proposing amendments to the Cyber Security and Resilience Bill.
How do you know it put you in a stronger position? Like how many MPs do you know used some of your speech notes, and how many strong relationships did you develop?
I believe that Ben Lake, Iqbal Mohamed and Shockat Adam used our notes in their speeches.
This was most clearly useful for our relationship with Iqbal Mohamed, who has tabled one of our proposed amendments for the Cyber Security and Resilience Bill and who solicited our input when he was selected to ask oral questions in Parliament recently. Ben Lake didn’t table our proposed amendment, but continues to be responsive in our communication and indicates a willingness to work with us in future. We have just reached out to several MPs to propose co-signing Iqbal’s amendment, so we shall see how many of them are supportive.
In October we held a screening in the UK Parliament of filmmaker Michaël Trazzi’s documentary about SB-1047, the proposed California AI legislation. This helped to inform MPs and Peers about the kinds of AI legislation that could be in a UK AI bill
Thanks again for hosting this! Can you clarify for readers how many MPs actually came to the movie screening & talked to you guys? I think it makes a difference to know if it was like 1-2 MPs or dozens. (Note: I remember you telling me it was more like the former, happy for you to rectify).
Yes, you’re right. It was quite small. I believe it was 2 MPs and 1 Lord. Also about 4 or 5 parliamentary staffers.
PauseAI UK has 10,000 highly dedicated volunteers who act as a dominant lobbying force on AI policy matters.
How realistic is this scenario? Like how many dedicated volunteers does PauseAI currently have? Without mentioning the timeline and the likelihood of this scenario, I’m wondering if you’re thinking of a 0.1% ideal scenario over 5 years, or if you’re actually 50% confident this could happen in 2-3 years. Like if you currently have 5-10 dedicated volunteers spending several hours (> 5 hours) a week working on PauseAI UK, then 10,000 would be more than 10 doublings, so ~6 years if we believe the number of dedicated volunteers follow the same exponential growth as you claim for protests?
We have something about 20 volunteers who will spend >3 hours per month helping with PauseAI stuff behind the scenes (not including time spent participating in events or activities). And maybe 100 volunteers who will fairly reliably participate in events or activities when prompted.
Certainly, 10,000 dedicated volunteers is an ambitious goal and would be an big achievement. I think it’s most useful to talk about timelines in terms of AI progress, rather than years, because our growth will likely be a function of AI capabilities. I would estimate something like 20% probability would could achieve 10,000 dedicated volunteers before TAI with our current level of funding and staff. This would rise to around 50% if we had 10 staff. But that is with the important caveat that I would redefine a dedicated volunteer as someone who spends 5 hours per month doing PauseAI stuff (including participation in events and activities), as I think that would be the level of involvement we would require of an average volunteer to achieve the outcome described.
So very roughly, that scenario would be about like P_dou x P_war x P_gov likely. In my view, if I was to give very rough numbers that’d be about 0.5 x 0.1 x 0.1 = 0.5% likely.
I think this estimate is misleading because P_warn and P_gov are very much not independent. P_gov | P_warn is fairly high I think. I’m very uncertain about P_warn, so 0.1 seems a bit overconfident to me. I’d estimate 0.3.
And then you’d need to also factor in how likely would it be for a protest of this size to happen without PauseAI, and how much leverage the UK would have to lead us to an actual treaty anyway.
We have some evidence for the counterfactual impact of PauseAI UK by comparing to other countries. As mentioned, London has consistently had the largest AI protests of any city since 2023, but obviously there are a million confounders.
But I don’t think this is quite the right counterfactual to consider, at least from a funding perspective. PauseAI will continue to exist in some form, whether or not it gets funding. The question is whether how much bigger the protests will be with and without PauseAI UK getting funding. Or perhaps whether some other group could organise bigger protests given the funding.
Regarding the Exponentials and 7 months doubling
Yes, your comments are basically valid. I wasn’t trying to imply that people should take the 7 month doubling time seriously as a specific number. The chart speaks for itself in showing the main point, which is that protests are growing in size and this growth is accelerating.
On your specific points:
Feb 2025 was a genuine anomaly because I was busy around that time and put far less work into organising it than other protests. The number of attendees is always strongly correlated with the work spent organising, so it’s not surprising this one was smaller.
On the Feb 2026 protest, it could definitely be fair to say that 300 is an over-representation of the trend given that it was a joint protest, although the sign-ups also show a similar trend and are only for PauseAI.
All these numbers should be taken as rough approximations. It’s very hard to count how many people are in a place when they are all moving. People arrive and leave at different times, but these numbers tend to reflect the largest number present at any one time, rather than the total who joined at any time.
And the Feb 2026 is particularly hard to count: PauseAI had more sign-ups than Pull the Plug and many people were there for both PauseAI and Pull the Plug, so I think 150 is a bit of an underestimate of the number of people who would count as PauseAI protestors. And that protest crossed the size threshold where it was basically impossible to ever have a view of every person at once, in the location we were at. So the 300 number is itself highly uncertain. Pull the Plug estimated turnout of 500, so I think we’re being reasonably conservative.

Joseph Miller 5 Jun 2026 19:01 UTC
4 points
0
in reply to: JennaS’s comment on: One Year of PauseAI UK
Thank you! Fixed.

Joseph Miller 5 Jun 2026 17:59 UTC
12 points
0
in reply to: anaguma’s comment on: One Year of PauseAI UK
lol I’ve already had a crisis over this.
I ran an informal survey and found that this was a rare enough thought that it wasn’t a big deal. But we will come up with a different design at some point.

One Year of PauseAI UK

Joseph Miller and PauseAI UK

5 Jun 2026 16:41 UTC

94 points

7 comments11 min readLW link

(pauseai.uk)

Joseph Miller 3 Jun 2026 12:18 UTC
3 points
0
in reply to: cubefox’s comment on: Tech I’m skeptical of and why
It wasn’t about rules for datacenters. He was saying that the regulatory barriers for building new electric power are so high that it’s easier to build the power capacity in space.

Joseph Miller 2 Jun 2026 1:40 UTC
3 points
0
on: Tech I’m skeptical of and why
My understanding from Elon Musk’s Dwarkesh podcast is that the case for space datacenters was not that they’d be more cost effectively exactly. But that you literally wouldn’t be able to build enough power in Western countries given current rules for terrestrial data centers to be switched on.

Joseph Miller 21 Apr 2026 19:50 UTC
5 points
−3
in reply to: Zack_M_Davis’s comment on: Joseph Miller’s Shortform
If you make people dislike you then you will push them away from your viewpoint whether or not your viewpoint is true.
If you make people like you then there’s a greater chance they will chance they will judge your view on the merits.

Joseph Miller 21 Apr 2026 12:22 UTC
9 points
−14
on: Joseph Miller’s Shortform
In response to Annoyingly Principled People, and what befalls them.
I think some people should internalize that often Principles Don’t Justify Drama (Among Humans). Drama destroys future discussion and activates primal emotions that will override rational deliberation. If you want to bring people around to your viewpoint, make them like you.
If everyone died on their hill every time someone violated what they thought was an important principle, society would not function. Cooperate with people you disagree with and when you violate someone else’s principle, be thankful that they are going to cooperate with you.
The best historical example of this is probably religious toleration. Thank God for those who tolerated infidels.
Of course the art is in knowing which hills to actually die on and who you should really shun as your enemy. To get it right, you need to fully appreciate the costs.

Joseph Miller 4 Feb 2026 23:04 UTC
4 points
−3
in reply to: Oliver Daniels’s comment on: Joseph Miller’s Shortform
I only have anecdata but I’ve talked to quite a few people and most people say it’s is a good idea to use the myriad of other concerns about AI as a force multiplier on shared policy goals.

Joseph Miller 4 Feb 2026 22:15 UTC
23 points
−7
on: Joseph Miller’s Shortform
The next PauseAI UK protest will be (AFAIK) the first coalition protest between different AI activist groups, the main other group being Pull the Plug, a new organisation focused primarily on current AI harms. It will almost certainly be the largest protest focused exclusively on AI to date.
In my experience, the vast majority of people in AI safety are in favor of big-tent coalition protests on AI in theory. But when faced with the reality of working with other groups who don’t emphasize existential risk, they have misgivings. So I’m curious what people here will think of this.
Personally I’m excited about the protest and I’ve found the organizers of Pull the Plug to be very sincere and good to work with, but I’ve also set things up so that the brands of PauseAI UK and Pull the Plug are clearly distinct, so that our messaging remains clearly focused on the risks of future AI. For example, we have a separate signup page and we have our own demands focused on decelerating frontier development.

Joseph Miller 15 Jan 2026 16:47 UTC
4 points
2
in reply to: Joseph Miller’s comment on: Joseph Miller’s Shortform
Or even better, at the the time when a post is frontpaged, check if will actually appear on the frontpage. If it is too old and has too little karma to be seen, then use the time-since-frontpaged.

Joseph Miller 15 Jan 2026 14:20 UTC
4 points
0
in reply to: Lucius Bushnaq’s comment on: Simple summary of AI Safety laws
US laws:
Well firstly note that the RAISE act has $1M - $3M escalating fines explicitly built in.
But yes, my understanding is that if you pay up, you can carry on. However, they can be fined “per violation” and the law does not precisely define what this means. So the Attorney General can argue that each missed requirement / each non-complying model is a separate violation. Or if they’re feeling very aggressive, the AG could try to say that each day of non-compliance is a separate violation, but that may not hold up in court.
This is where injunctive relief could come in, to directly order a company to retract a model. As noted, this was only explicitly included in the RAISE act^[1], and that looks set to be removed. However, AGs can often get injunctions even without explicit statutory language.
EU AI Act:
No, the consequences will escalate. Fines can be periodic, the maximum fine is huge and the length of the infraction is a factor in determining the penalty.
And on top of that, companies have a duty to take “immediate corrective actions”. The Commission (with advice from the AI Office) can issue administrative orders requiring the withdrawal of a model. And ignoring that can get you into very heavy fines or criminal penalties.
1. ^
  Except on whistleblower protections, where SB 53 explicitly includes injunctive relief.

Joseph Miller 15 Jan 2026 3:15 UTC
6 points
2
in reply to: habryka’s comment on: Joseph Miller’s Shortform
Perhaps it could use time-since-frontpaged only if the karma is below some threshold.

Joseph Miller 14 Jan 2026 23:42 UTC
29 points
18
on: Joseph Miller’s Shortform
On LessWrong, the frontpage algorithm down-weights older posts based on the time-since-posted, not the time-since-frontpaged. So, if a post doesn’t get frontpaged until a few days after posting, then it’s unlikely to get many views.
LessWrong has an autofrontpager that works a reasonable amount of the time. Otherwise, posts have to be manually frontpaged by a person. In my experience, this was always quite quick, but my most recent post was not frontpaged until 3 days after it was posted, so AFAICT it never actually appeared on the frontpage (unless you clicked “Load More”).
I think the solution is to downweight posts based on the time-since-frontpaged.

Joseph Miller 7 Jan 2026 22:03 UTC
2 points
0
in reply to: DanielFilan’s comment on: Simple summary of AI Safety laws
Strictly speaking, the version that passed the senate has now been put into law.
But to answer your question, no, this post describes what the bill will be like after the agreed upon “chapter amendments” have been implemented.
Footnote 4 mentions one of the changes that is expected to be made.