Ryan Kidd

Karma: 2,960

Co-Executive Director at MATS Research (2022-present)
Co-Founder & Board Member at London Initiative for Safe AI (2023-present)
Manifund Regrantor (2023-present) | RFPs here
Advisor, Catalyze Impact (2023-present) | ToC here
Advisor, AI Safety ANZ (2024-present) | ToC here
Advisor, Pivotal Research (2024-present)
Advisor, Halcyon Futures (2025-present)
Advisor, Black in AI Safety and Ethics (2025-present)
Advisor, Alignment Foundation (2026-present)
Ph.D. in Physics at the University of Queensland (2017-2023)
Group organizer at Effective Altruism UQ (2018-2021)

Personal website: ryankidd.ai

Give me feedback! :)

Ryan Kidd 27 Mar 2026 23:46 UTC
4 points
2
on: Where do AI Safety Fellows go? Analyzing a dataset of 600+ alumni
I found over 10% of fellows did another fellowship after their fellowship. This doesn’t feel enormously efficient.
This matches my finding that 10% of each MATS cohort, on average, did MATS before. I don’t think this is particularly bad; it’s somewhat analogous to undergrads doing several internships in different fields or with different companies before graduating.

Ryan Kidd 12 Mar 2026 22:52 UTC
4 points
1
in reply to: Cleo Nardo’s comment on: The case for AI safety capacity-building work
I now think the net-neutral ratio of new median safety to capabilities researchers is probably closer to 1:100 (though I think 1:10 isn’t crazy). However, the longitudinal MATS career data indicates our alumni are employed in safety to capabilities roles 8:1.

Ryan Kidd 24 Feb 2026 23:20 UTC
9 points
3
in reply to: Ryan Kidd’s comment on: Ryan Kidd’s Shortform
My sense of priorities for an Australian AISI.
Priorities
1. Enable safe and secure frontier AI training and deployment in Australia
  1. Build pathways around copyright restriction that prevents frontier AI companies from building training datacenters in Australia; the OpenAI datacenter being built in Sydney is for inference only
  2. Frontier AI will change the nature of work, shape international power, and potentially change what it means to be human and Australia doesn’t have a seat at the table; fix this
  3. Create jobs and opportunities for Australians to build and contribute to safe and secure AI in Australia, rather than all going overseas
2. Set clear national guidelines for AI safety, security, and transparency
  1. Set national AI standards similar to SB 53, RAISE Act, EU AI Act
  2. Create a “regulatory market” for AI and empower insurers, standard-setters, and auditors
  3. Allow beneficial AI development by removing regulatory uncertainty
3. Advance Australian interests in AI safety and security internationally
  1. Contribute to frontier AI evaluations and safety research like UK AISI, US CAISI
  2. Leverage Australia’s role as a “middle power” to create beneficial AI outcomes
  3. Share resources with the international network of AI safety institutes
Not priorities
1. Overly focus on algorithmic bias, privacy violations, or misinformation/deepfakes to the exclusion of systemic or catastrophic risk from AI
2. Overly focus on non-frontier AI technologies like recommender systems or AI for drug discovery instead of frontier AI systems like ChatGPT, Claude, Gemini, Grok, Llama, DeepSeek, Kimi
3. Focus on enforcing or extending copyright laws that prevent AI investment in Australia

Ryan Kidd 23 Feb 2026 16:13 UTC
8 points
0
in reply to: Ryan Kidd’s comment on: Ryan Kidd’s Shortform
Credit to Daniel Eth, who inspired this.

Ryan Kidd 22 Feb 2026 3:52 UTC
4 points
0
in reply to: 142857’s comment on: Ryan Kidd’s Shortform
Thanks, fixed!

Ryan Kidd 22 Feb 2026 2:40 UTC
115 points
41
on: Ryan Kidd’s Shortform
Someone recently shared an interesting minimal model of electoral politics with me. Elected officials are influenced by two primary groups: voters and donors. Sometimes voter and donor interests align, but often they don’t. Voters are the ultimate deciders, but they are only really swayed by the top-3 most salient issues. On those issues, elected officials side with voters, but on the rest they tend to side with donors. It’s simply too costly for elected officials to go against donors on low-public salience issues; they can spend the donated money on campaigning for the top-salience issues, after all!
One interesting low-salience issue is AI policy. Voters are generally anti-AI (they don’t like datacenters, automated surveillance, or recommender systems) but donors (often tech billionaires) are pro-AI. As AI is currently a low-salience issue for US voters (at least relative to the broader economy, healthcare, and immigration), politicians are siding with donors. For meaningful political action on AI, one would need either: AI to become a top-3 salience issue to voters (e.g., via job loss, weaponization, or warning shots); or more anti-AI donors to mobilize.

Ryan Kidd 9 Feb 2026 20:14 UTC
5 points
0
in reply to: Chris_Leong’s comment on: Ryan Kidd’s Shortform
Yes, I would generally support picking the latter as they have a “faster time to mentorship/research leadership/impact” and the field seems currently bottlenecked on mentorship and research leads, not marginal engineers (though individual research leads might feel bottlenecked on marginal engineers).
We should prioritize people who already have research or engineering experience or a very high iteration speed as we are operating under time constraints; AGI is coming soon. Additionally, I think “research taste” will be more important than engineering ability given AI automation and this takes a long time to build; better to select people with existing research experience they can adapt from another field (also promotes interdisciplinary knowledge transfer).
I talk more about it here.

Ryan Kidd 9 Feb 2026 2:52 UTC
4 points
2
in reply to: BryceStansfield’s comment on: Ryan Kidd’s Shortform
Yoshua Bengio, Paul Christiano, Geoffrey Irving seem more like technical AI safety experts than AI policy experts, but they arguably have strong influence on governments.

Ryan Kidd 8 Feb 2026 23:27 UTC
6 points
3
in reply to: Ryan Kidd’s comment on: Ryan Kidd’s Shortform
What should be done? I think:
- Office hubs: Expand SASS, which is in close proximity to the new OpenAI and Anthropic offices in Sydney. Start an AI safety hub in Canberra to support the new AISI. Successful AI safety hubs have benefited from prominent founding member organizations like ARC, CG, Redwood, and MIRI (for Constellation) and Apollo, BlueDot, and MATS (for LISA). Similarly, SASS should bring together orgs like the Gradient Institute and Harmony Intelligence in a shared space, and the new Canberra hub should be built around Good Ancestors. Office hubs can benefit member orgs by providing cheaper returns to scale, hosting shared networking events, facilitating collaboration, and providing a pipeline of strong new recruits in the form of office guests and members.
- Training programs: Expand TARA and the Sydney AI Safety Fellowship program, focusing on accelerating top talent and building local mentorship capacity for future programs. Don’t focus on maximizing impact on participants; this is less important than reducing the mentorship bottleneck, which is best served by boosting the most advanced participants.
- Academic labs: Build relationships with AI/CS academics at UniMelb, Monash, USyd, ANU, UTS, UNSW, UA, UQ, etc. Help launch AI safety courses like Roy Rinberg and Boaz Barak did at Harvard. Other course inspiration is provided by Stanford and CAIS. Start AI safety academic labs like UC Berkeley CHAI, MIT AAG, NYU ARG, Bau Lab, Stanford HAI, CMU FOCAL, etc.
- Conferences: Run an annual AI safety conference like the Australian AI Safety Forum 2024, bringing together academia, industry, government, and nonprofit field-builders. EAGx is probably not enough, as many people from academia, industry, and government likely won’t attend.

Ryan Kidd 8 Feb 2026 23:04 UTC
2 points
0
in reply to: aog’s comment on: Ryan Kidd’s Shortform
I suspect that some LWers would interpret this as a (bad) argument for countries to build datacenters so they can exercise political control over AGI. I don’t think this works.

Ryan Kidd 8 Feb 2026 20:08 UTC
6 points
4
in reply to: aog’s comment on: Ryan Kidd’s Shortform
Exactly! Also:
- AI safety researchers tend to have reasonable views on AI safety and can serve as local advisors to governments, which probably trust foreign experts less.
- Securing datacenters for SL5 to prevent proliferation of AGI is really hard and likely requires significant AI security expertise in government and local defence contractors.
- A regulatory market approach to AI safety (possibly only useful pre-superintelligence) requires competent local auditors, standard-setters, and insurers.

Ryan Kidd 7 Feb 2026 23:58 UTC
79 points
43
on: Ryan Kidd’s Shortform
AI safety field-building in Australia should accelerate. My rationale:
- OpenAI opened a Sydney office in Dec 2025 and Anthropic is planning to open a Sydney office in 2026. These offices may hire safety staff from local talent, or partner with local auditing, evaluation, and security companies, including Harmony Intelligence, Good Ancestors, and Gradient Institute.
- An Australian AISI was announced for early 2026 and is currently hiring. The UK AISI has benefited from close partnerships with Apollo Research, METR, and the LISA office community. There is a community space in Sydney, the Sydney AI Safety Space, and two field-building organizations, AI Safety ANZ and TARA, but these could expand substantially.
- Australia seems like a prime location for datacenter build-out. OpenAI published an “AI blueprint” for Australia, calling for datacenter build-out, and started building a $4.6B datacenter in Sydney in Dec 2025. Australia is a NATO partner, Five Eyes member, and member of the AUKUS security partnership with the US and UK; it’s much more secure and aligned with US/UK interests than Saudi Arabia. Australia is the second-largest exporter of thermal coal, has vast solar and wind resources, and the highest uranium reserves on earth. Australia is currently quite anti-nuclear at the moment, but it has no earthquakes or tsunamis to disrupt power plants. Janet Egan (CNAS) recently called for the development of US military AI projects in Australia, similar to the Pine Gap facility in the Northern Territory. AI safety & security research and political pressure for safety standards should focus on countries with frontier AI companies and datacenters.
- Several prominent AI safety researchers have come from Australia, including Marcus Hutter, Buck Shlegeris, Jan Leike (PhD only), Ramana Kumar, Dan Murfet, and Daniel Filan. A decent number of MATS fellows come from Australia. Australian citizens can easily emigrate to the US (E-3 visa) or UK (YMS visa) for work. Seven of the top-100 computer science universities are in Australia.
- Australia has a history as a middle power between the US and China, two of its largest trading partners. The country may play a significant role in international diplomacy efforts, such as a superintelligence ban or international AGI project.
- Australia has a strong history of environmental activism, which might (or might not) be a useful asset for AI safety political mobilization.

Ryan Kidd 27 Jan 2026 22:17 UTC
3 points
−2
in reply to: Ryan Kidd’s comment on: Ryan Kidd’s Shortform
I think that the distribution of mentors we are drawing from is slowing growing to include more highly respected academics and industry professionals by percentage. I think this increases the average quality of our mentor applicant pool, but I understand that this is might be controversial. Note that I still think our most impactful mentors are well-known within the AI safety field and most of the top-50 most impactful researchers in AI safety apply to mentor at MATS.

Ryan Kidd 27 Jan 2026 20:50 UTC
5 points
0
in reply to: Neel Nanda’s comment on: Ryan Kidd’s Shortform
This is a hard question to answer precisely, as we have changed the metrics by which we have evaluated potential mentors several times. The average quality of mentors we accept has grown each program, by my lights. I weakly think that the average quality of mentors applying has also grown, though much slower.

Ryan Kidd 27 Jan 2026 18:09 UTC
12 points
0
on: Ryan Kidd’s Shortform
Update on MATS applications trends:
- Mentor applications seem to be growing 2.2x/year and we accepted 20% as primary mentors for 10.0 (Summer 2026).
- Fellow applications seem to be growing 1.8x/year (counting only applicants who applied to at least one mentor) and we plan to accept 5% for 10.0.

Ryan Kidd 2 Jan 2026 18:29 UTC
2 points
0
in reply to: Ryan Kidd’s comment on: Ryan Kidd’s Shortform
In contrast to the apparent exponential growth in AI conference attendees, the number of AI publications since 2013 has grown quadratically (data from the Standford HAI 2025 AI Index Report). Quadratic growth in publications suggests that a linearly (constantly) increasing number of researchers are producing publications at a linear (constant) rate. Extrapolating a little, this growth rate suggests there will be 3.7M cumulative publications by 2030 (since 2013).
If the AI researcher growth rate is linear, the exponential growth of AI conference attendees might be due to increased industry presence. Also, it’s possible that the attendee growth rate is also quadratic.

Ryan Kidd 30 Dec 2025 3:26 UTC
3 points
0
in reply to: Ryan Kidd’s comment on: Ryan Kidd’s Shortform
Open question: how fast did the field of cybersecurity grow since the launch of the internet?

Ryan Kidd 30 Dec 2025 3:15 UTC
4 points
0
on: Ryan Kidd’s Shortform
Attendees at the top-four AI conferences have been growing at 1.26x/year on average. Data is from Our World in Data. I skipped 2020-2021 for all conferences and 2022 for ICLR, as these conferences were virtual due to the COVID pandemic and had increased virtual attendance.
One could infer from these growth rates that the academic field of AI is growing 1.26x/year. Interestingly, the AI safety field (including technical and governance) seems to be growing at 1.25x/year.

Ryan Kidd 23 Dec 2025 18:36 UTC
4 points
0
in reply to: Ryan Kidd’s comment on: Ryan Kidd’s Shortform
The top-10 most-cited papers that MATS contributed to are (all with at least 290 citations)
Compare this to the top-10 highest-karma LessWrong posts that MATS contributed to (all with over 200 karma):

Ryan Kidd 23 Dec 2025 18:15 UTC
5 points
0
in reply to: Ryan Kidd’s comment on: Ryan Kidd’s Shortform
Here is a plot of the annual citations received by MATS, EleutherAI, and Apart research, adjusted so they start on the same year. The three organizations are somewhat comparable, as they leverage large networks of external collaborators: MATS mentors/fellows, EleutherAI Discord, Apart sprint participants.
The EleutherAI data fits a logistic curve perfectly, asymptoting to ~18.5k citations/year. I can’t fit the others as at least 4 data points are needed to fit a logistic curve.

Ryan Kidd

Priorities

Not priorities