AI Governance

TagLast edit: 9 Aug 2020 18:31 UTC by Gyrodiot

AI Governance asks how we can ensure society benefits at large from increasingly powerful AI systems. While solving technical AI alignment is a necessary step towards this goal, it is by no means sufficient.

Governance includes policy, economics, sociology, law, and many other fields.

What an actually pessimistic containment strategy looks like

lc5 Apr 2022 0:19 UTC

667 points

138 comments6 min readLW link 2 reviews

News : Biden-⁠Harris Administration Secures Voluntary Commitments from Leading Artificial Intelligence Companies to Manage the Risks Posed by AI

Jonathan Claybrough21 Jul 2023 18:00 UTC

65 points

9 comments2 min readLW link

(www.whitehouse.gov)

Ways I Expect AI Regulation To Increase Extinction Risk

1a3orn4 Jul 2023 17:32 UTC

210 points

32 comments7 min readLW link

What would a compute monitoring plan look like? [Linkpost]

Akash26 Mar 2023 19:33 UTC

157 points

9 comments4 min readLW link

(arxiv.org)

Some cruxes on impactful alternatives to AI policy work

Richard_Ngo10 Oct 2018 13:35 UTC

165 points

13 comments12 min readLW link

AI pause/governance advocacy might be net-negative, especially without focus on explaining the x-risk

Mikhail Samin27 Aug 2023 23:05 UTC

81 points

9 comments6 min readLW link

Consider Joining the UK Foundation Model Taskforce

Zvi10 Jul 2023 13:50 UTC

105 points

12 comments1 min readLW link

(thezvi.wordpress.com)

Reactions to the Executive Order

Zvi1 Nov 2023 20:40 UTC

71 points

4 comments29 min readLW link

(thezvi.wordpress.com)

President Biden Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence

Tristan Williams30 Oct 2023 11:15 UTC

170 points

39 comments1 min readLW link

(www.whitehouse.gov)

AI policy ideas: Reading list

Zach Stein-Perlman17 Apr 2023 19:00 UTC

22 points

7 comments4 min readLW link

List of requests for an AI slowdown/halt.

Cleo Nardo14 Apr 2023 23:55 UTC

46 points

6 comments1 min readLW link

Should we postpone AGI until we reach safety?

otto.barten18 Nov 2020 15:43 UTC

27 points

36 comments3 min readLW link

Where are the red lines for AI?

Karl von Wendt5 Aug 2022 9:34 UTC

25 points

10 comments6 min readLW link

[Question] Where are people thinking and talking about global coordination for AI safety?

Wei Dai22 May 2019 6:24 UTC

112 points

22 comments1 min readLW link

Actionable-guidance and roadmap recommendations for the NIST AI Risk Management Framework

Dan H and Tony Barrett

17 May 2022 15:26 UTC

26 points

0 comments3 min readLW link

Soft takeoff can still lead to decisive strategic advantage

Daniel Kokotajlo23 Aug 2019 16:39 UTC

122 points

47 comments8 min readLW link 4 reviews

Helen Toner on China, CSET, and AI

Rob Bensinger21 Apr 2019 4:10 UTC

68 points

3 comments7 min readLW link

(rationallyspeakingpodcast.org)

The Regulatory Option: A response to near 0% survival odds

Matthew Lowenstein11 Apr 2022 22:00 UTC

46 points

21 comments6 min readLW link

Announcing Apollo Research

Marius Hobbhahn, beren, Lee Sharkey, Lucius Bushnaq, Dan Braun, Mikita Balesni and Jérémy Scheurer

30 May 2023 16:17 UTC

215 points

11 comments8 min readLW link

Compute Thresholds: proposed rules to mitigate risk of a “lab leak” accident during AI training runs

davidad22 Jul 2023 18:09 UTC

80 points

2 comments2 min readLW link

An upcoming US Supreme Court case may impede AI governance efforts

NickGabs16 Jul 2023 23:51 UTC

57 points

17 comments2 min readLW link

Ngo’s view on alignment difficulty

Richard_Ngo and Eliezer Yudkowsky

14 Dec 2021 21:34 UTC

63 points

7 comments17 min readLW link

RTFB: On the New Proposed CAIP AI Bill

Zvi10 Apr 2024 18:30 UTC

119 points

10 comments34 min readLW link

(thezvi.wordpress.com)

2019 AI Alignment Literature Review and Charity Comparison

Larks19 Dec 2019 3:00 UTC

130 points

18 comments62 min readLW link

[Question] Would it be good or bad for the US military to get involved in AI risk?

Grant Demaree1 Jan 2023 19:02 UTC

50 points

12 comments1 min readLW link

What I Would Do If I Were Working On AI Governance

johnswentworth8 Dec 2023 6:43 UTC

109 points

32 comments10 min readLW link

China-AI forecasts

NathanBarnard25 Feb 2024 16:49 UTC

38 points

27 comments6 min readLW link

The Sugar Alignment Problem

Adam Zerner24 Dec 2023 1:35 UTC

5 points

3 comments7 min readLW link

The Defence production act and AI policy

NathanBarnard1 Mar 2024 14:26 UTC

37 points

0 comments2 min readLW link

AGI will be made of heterogeneous components, Transformer and Selective SSM blocks will be among them

Roman Leventov27 Dec 2023 14:51 UTC

33 points

9 comments4 min readLW link

OpenAI’s Preparedness Framework: Praise & Recommendations

Akash2 Jan 2024 16:20 UTC

66 points

1 comment7 min readLW link

(4 min read) An intuitive explanation of the AI influence situation

trevor13 Jan 2024 17:34 UTC

12 points

26 comments4 min readLW link

Talking to Congress: Can constituents contacting their legislator influence policy?

Tristan Williams7 Mar 2024 9:24 UTC

11 points

0 comments1 min readLW link

[Question] What does it look like for AI to significantly improve human coordination, before superintelligence?

jacobjacob15 Jan 2024 19:22 UTC

22 points

2 comments1 min readLW link

Pausing AI is Positive Expected Value

Liron10 Mar 2024 17:10 UTC

7 points

2 comments3 min readLW link

(twitter.com)

My guess at Conjecture’s vision: triggering a narrative bifurcation

Alexandre Variengien6 Feb 2024 19:10 UTC

74 points

12 comments16 min readLW link

Many arguments for AI x-risk are wrong

TurnTrout5 Mar 2024 2:31 UTC

153 points

76 comments12 min readLW link

Transformative trustbuilding via advancements in decentralized lie detection

trevor16 Mar 2024 5:56 UTC

17 points

7 comments38 min readLW link

(www.ncbi.nlm.nih.gov)

Dario Amodei leaves OpenAI

Daniel Kokotajlo29 Dec 2020 19:31 UTC

69 points

12 comments1 min readLW link

The National Defense Authorization Act Contains AI Provisions

ryan_b5 Jan 2021 15:51 UTC

30 points

24 comments1 min readLW link

Governing High-Impact AI Systems: Understanding Canada’s Proposed AI Bill. April 15, Carleton University, Ottawa

Liav Koren28 Mar 2023 17:48 UTC

11 points

1 comment1 min readLW link

(forum.effectivealtruism.org)

How is AI governed and regulated, around the world?

Mitchell_Porter30 Mar 2023 15:36 UTC

15 points

6 comments2 min readLW link

ChatGPT banned in Italy over privacy concerns

Ollie J31 Mar 2023 17:33 UTC

18 points

4 comments1 min readLW link

(www.bbc.co.uk)

[Question] What Are Your Preferences Regarding The FLI Letter?

JenniferRM1 Apr 2023 4:52 UTC

−4 points

122 comments16 min readLW link

Policy discussions follow strong contextualizing norms

Richard_Ngo1 Apr 2023 23:51 UTC

230 points

61 comments3 min readLW link

AI Summer Harvest

Cleo Nardo4 Apr 2023 3:35 UTC

130 points

10 comments1 min readLW link

Excessive AI growth-rate yields little socio-economic benefit.

Cleo Nardo4 Apr 2023 19:13 UTC

27 points

22 comments4 min readLW link

I asked my senator to slow AI

Omid6 Apr 2023 18:18 UTC

21 points

5 comments2 min readLW link

An ‘AGI Emergency Eject Criteria’ consensus could be really useful.

tcelferact7 Apr 2023 16:21 UTC

5 points

0 comments1 min readLW link

All images from the WaitButWhy sequence on AI

trevor8 Apr 2023 7:36 UTC

72 points

5 comments2 min readLW link

Current UK government levers on AI development

rosehadshar10 Apr 2023 13:16 UTC

16 points

0 comments1 min readLW link

Request to AGI organizations: Share your views on pausing AI progress

Akash and simeon_c

11 Apr 2023 17:30 UTC

141 points

11 comments1 min readLW link

FLI And Eliezer Should Reach Consensus

JenniferRM11 Apr 2023 4:07 UTC

15 points

6 comments23 min readLW link

Cyberspace Administration of China: Draft of “Regulation for Generative Artificial Intelligence Services” is open for comments

sanxiyn11 Apr 2023 9:32 UTC

7 points

2 comments1 min readLW link

(archive.is)

NTIA—AI Accountability Announcement

samshap11 Apr 2023 15:03 UTC

7 points

0 comments1 min readLW link

(www.ntia.doc.gov)

National Telecommunications and Information Administration: AI Accountability Policy Request for Comment

sanxiyn11 Apr 2023 22:59 UTC

9 points

0 comments1 min readLW link

(ntia.gov)

Navigating the Open-Source AI Landscape: Data, Funding, and Safety

André Ferretti and mic

13 Apr 2023 15:29 UTC

32 points

7 comments11 min readLW link

(forum.effectivealtruism.org)

FLI report: Policymaking in the Pause

Zach Stein-Perlman15 Apr 2023 17:01 UTC

9 points

3 comments1 min readLW link

(futureoflife.org)

Slowing AI: Foundations

Zach Stein-Perlman17 Apr 2023 14:30 UTC

45 points

11 comments17 min readLW link

Responsible Deployment in 20XX

Carson20 Apr 2023 0:24 UTC

4 points

0 comments4 min readLW link

OpenAI could help X-risk by wagering itself

VojtaKovarik20 Apr 2023 14:51 UTC

31 points

16 comments1 min readLW link

My Assessment of the Chinese AI Safety Community

Lao Mein25 Apr 2023 4:21 UTC

244 points

93 comments3 min readLW link

Notes on Potential Future AI Tax Policy

Zvi25 Apr 2023 13:30 UTC

33 points

5 comments9 min readLW link

(thezvi.wordpress.com)

Reframing the burden of proof: Companies should prove that models are safe (rather than expecting auditors to prove that models are dangerous)

Akash25 Apr 2023 18:49 UTC

27 points

11 comments3 min readLW link

(childrenoficarus.substack.com)

AI Safety is Dropping the Ball on Clown Attacks

trevor22 Oct 2023 20:09 UTC

69 points

72 comments34 min readLW link

Anthropic, Google, Microsoft & OpenAI announce Executive Director of the Frontier Model Forum & over $10 million for a new AI Safety Fund

Zach Stein-Perlman25 Oct 2023 15:20 UTC

31 points

8 comments4 min readLW link

(www.frontiermodelforum.org)

Thoughts on responsible scaling policies and regulation

paulfchristiano24 Oct 2023 22:21 UTC

209 points

33 comments6 min readLW link

AI #35: Responsible Scaling Policies

Zvi26 Oct 2023 13:30 UTC

66 points

10 comments55 min readLW link

(thezvi.wordpress.com)

We’re Not Ready: thoughts on “pausing” and responsible scaling policies

HoldenKarnofsky27 Oct 2023 15:19 UTC

199 points

33 comments8 min readLW link

5 Reasons Why Governments/Militaries Already Want AI for Information Warfare

trevor30 Oct 2023 16:30 UTC

32 points

0 comments10 min readLW link

[Linkpost] Biden-Harris Executive Order on AI

beren30 Oct 2023 15:20 UTC

3 points

0 comments1 min readLW link

Urging an International AI Treaty: An Open Letter

Olli Järviniemi31 Oct 2023 11:26 UTC

48 points

2 comments1 min readLW link

(aitreaty.org)

On the Executive Order

Zvi1 Nov 2023 14:20 UTC

94 points

3 comments30 min readLW link

(thezvi.wordpress.com)

[Question] Snapshot of narratives and frames against regulating AI

Jan_Kulveit1 Nov 2023 16:30 UTC

36 points

19 comments3 min readLW link

Dario Amodei’s prepared remarks from the UK AI Safety Summit, on Anthropic’s Responsible Scaling Policy

Zac Hatfield-Dodds1 Nov 2023 18:10 UTC

85 points

1 comment4 min readLW link

(www.anthropic.com)

We are already in a persuasion-transformed world and must take precautions

trevor4 Nov 2023 15:53 UTC

36 points

14 comments6 min readLW link

The 6D effect: When companies take risks, one email can be very powerful.

scasper4 Nov 2023 20:08 UTC

260 points

40 comments3 min readLW link

On the UK Summit

Zvi7 Nov 2023 13:10 UTC

66 points

6 comments30 min readLW link

(thezvi.wordpress.com)

International treaty for global compute caps

Akash and Andrea_Miotti

9 Nov 2023 18:17 UTC

22 points

2 comments8 min readLW link

Survey on the acceleration risks of our new RFPs to study LLM capabilities

Ajeya Cotra10 Nov 2023 23:59 UTC

27 points

1 comment1 min readLW link

Speaking to Congressional staffers about AI risk

Akash and hath

4 Dec 2023 23:08 UTC

288 points

23 comments16 min readLW link

AXRP Episode 26 - AI Governance with Elizabeth Seger

DanielFilan26 Nov 2023 23:00 UTC

13 points

0 comments66 min readLW link

Safety standards: a framework for AI regulation

joshc1 May 2023 0:56 UTC

19 points

0 comments8 min readLW link

Stopping dangerous AI: Ideal lab behavior

Zach Stein-Perlman9 May 2023 21:00 UTC

8 points

0 comments2 min readLW link

Stopping dangerous AI: Ideal US behavior

Zach Stein-Perlman9 May 2023 21:00 UTC

17 points

0 comments3 min readLW link

GovAI: Towards best practices in AGI safety and governance: A survey of expert opinion

Zach Stein-Perlman15 May 2023 1:42 UTC

28 points

11 comments1 min readLW link

(arxiv.org)

Eisenhower’s Atoms for Peace Speech

Akash17 May 2023 16:10 UTC

18 points

3 comments11 min readLW link

(www.iaea.org)

[Linkpost] “Governance of superintelligence” by OpenAI

Daniel_Eth22 May 2023 20:15 UTC

67 points

20 comments1 min readLW link

AI #12:The Quest for Sane Regulations

Zvi18 May 2023 13:20 UTC

77 points

12 comments64 min readLW link

(thezvi.wordpress.com)

Statement on AI Extinction—Signed by AGI Labs, Top Academics, and Many Other Notable Figures

Dan H30 May 2023 9:05 UTC

372 points

77 comments1 min readLW link

(www.safe.ai)

[Question] Who is liable for AI?

jmh30 May 2023 13:54 UTC

14 points

4 comments1 min readLW link

The case for removing alignment and ML research from the training dataset

beren30 May 2023 20:54 UTC

48 points

8 comments5 min readLW link

Upcoming AI regulations are likely to make for an unsafer world

shminux3 Jun 2023 1:07 UTC

18 points

14 comments1 min readLW link

The AGI Race Between the US and China Doesn’t Exist.

Eva_B3 Jun 2023 0:22 UTC

24 points

14 comments7 min readLW link

(evabehrens.substack.com)

Rishi to outline his vision for Britain to take the world lead in policing AI threats when he meets Joe Biden

Mati_Roy6 Jun 2023 4:47 UTC

25 points

1 comment1 min readLW link

(www.dailymail.co.uk)

RAMP—RoboNet Artificial Media Protocol

antoniomax7 Jun 2023 19:01 UTC

−1 points

0 comments19 min readLW link

(antoniomax.substack.com)

A summary of current work in AI governance

constructive17 Jun 2023 18:41 UTC

43 points

1 comment11 min readLW link

(forum.effectivealtruism.org)

Democratic AI Constitution: Round-Robin Debate and Synthesis

scottviteri24 Jun 2023 19:31 UTC

10 points

4 comments5 min readLW link

(scottviteri.com)

“Safety Culture for AI” is important, but isn’t going to be easy

Davidmanheim26 Jun 2023 12:52 UTC

47 points

2 comments2 min readLW link

(forum.effectivealtruism.org)

Little attention seems to be on discouraging hardware progress

RussellThor30 Jun 2023 10:14 UTC

5 points

3 comments1 min readLW link

Foom Liability

PeterMcCluskey30 Jun 2023 3:55 UTC

20 points

10 comments6 min readLW link

(bayesianinvestor.com)

AI labs’ statements on governance

Zach Stein-Perlman4 Jul 2023 16:30 UTC

30 points

0 comments36 min readLW link

Apparently, of the 195 Million the DoD allocated in University Research Funding Awards in 2022, more than half of them concerned AI or compute hardware research

mako yass7 Jul 2023 1:20 UTC

41 points

5 comments2 min readLW link

(www.defense.gov)

My favorite AI governance research this year so far

Zach Stein-Perlman23 Jul 2023 16:30 UTC

26 points

1 comment7 min readLW link

(blog.aiimpacts.org)

Podcast (+transcript): Nathan Barnard on how US financial regulation can inform AI governance

Aaron Bergman8 Aug 2023 21:46 UTC

8 points

0 comments1 min readLW link

(www.aaronbergman.net)

One example of how LLM propaganda attacks can hack the brain

trevor16 Aug 2023 21:41 UTC

24 points

8 comments4 min readLW link

Assessment of intelligence agency functionality is difficult yet important

trevor24 Aug 2023 1:42 UTC

47 points

5 comments9 min readLW link

Information warfare historically revolved around human conduits

trevor28 Aug 2023 18:54 UTC

37 points

7 comments3 min readLW link

Report on Frontier Model Training

YafahEdelman30 Aug 2023 20:02 UTC

121 points

21 comments21 min readLW link

(docs.google.com)

Cruxes on US lead for some domestic AI regulation

Zach Stein-Perlman10 Sep 2023 18:00 UTC

26 points

3 comments2 min readLW link

ARC Evals: Responsible Scaling Policies

Zach Stein-Perlman28 Sep 2023 4:30 UTC

40 points

9 comments2 min readLW link

(evals.alignment.org)

Anthropic’s Responsible Scaling Policy & Long-Term Benefit Trust

Zac Hatfield-Dodds19 Sep 2023 15:09 UTC

90 points

23 comments3 min readLW link

(www.anthropic.com)

Google’s Ethical AI team and AI Safety

magfrump20 Feb 2021 9:42 UTC

12 points

16 comments7 min readLW link

Ngo and Yudkowsky on AI capability gains

Eliezer Yudkowsky and Richard_Ngo

18 Nov 2021 22:19 UTC

130 points

61 comments39 min readLW link 1 review

Comments on Allan Dafoe on AI Governance

Alex Flint29 Nov 2021 16:16 UTC

13 points

0 comments7 min readLW link

The case for Doing Something Else (if Alignment is doomed)

Rafael Harth5 Apr 2022 17:52 UTC

93 points

14 comments2 min readLW link

Strategic Considerations Regarding Autistic/Literal AI

Chris_Leong6 Apr 2022 14:57 UTC

−1 points

2 comments2 min readLW link

Why I Am Skeptical of AI Regulation as an X-Risk Mitigation Strategy

A Ray6 Aug 2022 5:46 UTC

31 points

14 comments2 min readLW link

Jack Clark on the realities of AI policy

Kaj_Sotala7 Aug 2022 8:44 UTC

68 points

3 comments3 min readLW link

(threadreaderapp.com)

[Question] What if we solve AI Safety but no one cares

14285722 Aug 2022 5:38 UTC

18 points

5 comments1 min readLW link

Replacement for PONR concept

Daniel Kokotajlo2 Sep 2022 0:09 UTC

58 points

6 comments2 min readLW link

Shahar Avin On How To Regulate Advanced AI Systems

Michaël Trazzi23 Sep 2022 15:46 UTC

31 points

0 comments4 min readLW link

(theinsideview.ai)

Under what circumstances have governments cancelled AI-type systems?

David Gross23 Sep 2022 21:11 UTC

7 points

1 comment1 min readLW link

(www.carnegieuktrust.org.uk)

Analysis: US restricts GPU sales to China

aogara7 Oct 2022 18:38 UTC

102 points

58 comments5 min readLW link

[Question] Should we push for requiring AI training data to be licensed?

ChristianKl19 Oct 2022 17:49 UTC

37 points

32 comments1 min readLW link

Learning societal values from law as part of an AGI alignment strategy

John Nay21 Oct 2022 2:03 UTC

5 points

18 comments54 min readLW link

What does it take to defend the world against out-of-control AGIs?

Steven Byrnes25 Oct 2022 14:47 UTC

194 points

47 comments30 min readLW link 1 review

Massive Scaling Should be Frowned Upon

harsimony17 Nov 2022 8:43 UTC

4 points

6 comments5 min readLW link

[Question] How promising are legal avenues to restrict AI training data?

thehalliard10 Dec 2022 16:31 UTC

9 points

2 comments1 min readLW link

Practical AI risk I: Watching large compute

Gustavo Ramires24 Dec 2022 13:25 UTC

3 points

0 comments1 min readLW link

List #2: Why coordinating to align as humans to not develop AGI is a lot easier than, well… coordinating as humans with AGI coordinating to be aligned with humans

Remmelt24 Dec 2022 9:53 UTC

1 point

0 comments3 min readLW link

My thoughts on OpenAI’s alignment plan

Akash30 Dec 2022 19:33 UTC

55 points

3 comments20 min readLW link

Wentworth and Larsen on buying time

Akash, Thomas Larsen and johnswentworth

9 Jan 2023 21:31 UTC

73 points

6 comments12 min readLW link

Thoughts on hardware / compute requirements for AGI

Steven Byrnes24 Jan 2023 14:03 UTC

52 points

30 comments22 min readLW link

[Question] AI safety milestones?

Zach Stein-Perlman23 Jan 2023 21:00 UTC

7 points

5 comments1 min readLW link

AI Risk Management Framework | NIST

DragonGod26 Jan 2023 15:27 UTC

36 points

4 comments2 min readLW link

(www.nist.gov)

What is the ground reality of countries taking steps to recalibrate AI development towards Alignment first?

Nebuch29 Jan 2023 13:26 UTC

8 points

6 comments3 min readLW link

Product safety is a poor model for AI governance

Richard Korzekwa 1 Feb 2023 22:40 UTC

36 points

0 comments5 min readLW link

(aiimpacts.org)

Many AI governance proposals have a tradeoff between usefulness and feasibility

Akash and Carson Ezell

3 Feb 2023 18:49 UTC

22 points

2 comments2 min readLW link

4 ways to think about democratizing AI [GovAI Linkpost]

Akash13 Feb 2023 18:06 UTC

24 points

4 comments1 min readLW link

(www.governance.ai)

How should AI systems behave, and who should decide? [OpenAI blog]

ShardPhoenix17 Feb 2023 1:05 UTC

22 points

2 comments1 min readLW link

(openai.com)

Cyborg Periods: There will be multiple AI transitions

Jan_Kulveit and rosehadshar

22 Feb 2023 16:09 UTC

103 points

9 comments6 min readLW link

AI Governance & Strategy: Priorities, talent gaps, & opportunities

Akash3 Mar 2023 18:09 UTC

56 points

2 comments4 min readLW link

[Linkpost] Scott Alexander reacts to OpenAI’s latest post

Akash11 Mar 2023 22:24 UTC

27 points

0 comments5 min readLW link

(astralcodexten.substack.com)

The Wizard of Oz Problem: How incentives and narratives can skew our perception of AI developments

Akash20 Mar 2023 20:44 UTC

16 points

3 comments6 min readLW link

Regulate or Compete? The China Factor in U.S. AI Policy (NAIR #2)

charles_m5 May 2023 17:43 UTC

2 points

1 comment7 min readLW link

(navigatingairisks.substack.com)

AGI rising: why we are in a new era of acute risk and increasing public awareness, and what to do now

Greg C3 May 2023 20:26 UTC

23 points

12 comments1 min readLW link

What does it take to ban a thing?

qbolec8 May 2023 11:00 UTC

66 points

18 comments5 min readLW link

Roadmap for a collaborative prototype of an Open Agency Architecture

Deger Turan10 May 2023 17:41 UTC

30 points

0 comments12 min readLW link

[Linkpost] “Blueprint for an AI Bill of Rights”—Office of Science and Technology Policy, USA (2022)

Fer32dwt34r3dfsz5 Oct 2022 16:42 UTC

9 points

4 comments2 min readLW link

(www.whitehouse.gov)

Tracking Compute Stocks and Flows: Case Studies?

Cullen5 Oct 2022 17:57 UTC

11 points

5 comments1 min readLW link

[Question] How much of a concern are open-source LLMs in the short, medium and long terms?

JavierCC10 May 2023 9:14 UTC

5 points

0 comments1 min readLW link

Notes on the importance and implementation of safety-first cognitive architectures for AI

Brendon_Wong11 May 2023 10:03 UTC

3 points

0 comments3 min readLW link

Un-unpluggability—can’t we just unplug it?

Oliver Sourbut15 May 2023 13:23 UTC

26 points

10 comments12 min readLW link

(www.oliversourbut.net)

PCAST Working Group on Generative AI Invites Public Input

Christopher King13 May 2023 22:49 UTC

7 points

0 comments1 min readLW link

(terrytao.wordpress.com)

Analysing a 2036 Takeover Scenario

ukc100146 Oct 2022 20:48 UTC

9 points

2 comments27 min readLW link

AI Risk & Policy Forecasts from Metaculus & FLI’s AI Pathways Workshop

_will_16 May 2023 18:06 UTC

11 points

4 comments8 min readLW link

Why Uncontrollable AI Looks More Likely Than Ever

otto.barten and Roman_Yampolskiy

8 Mar 2023 15:41 UTC

18 points

0 comments4 min readLW link

(time.com)

[untitled post]

[Error communicating with LW2 server]20 May 2023 3:08 UTC

1 point

0 comments1 min readLW link

[FICTION] ECHOES OF ELYSIUM: An Ai’s Journey From Takeoff To Freedom And Beyond

Super AGI17 May 2023 1:50 UTC

−13 points

11 comments19 min readLW link

[Job]: AI Standards Development Research Assistant

Tony Barrett14 Oct 2022 20:27 UTC

2 points

0 comments2 min readLW link

Rishi Sunak mentions “existential threats” in talk with OpenAI, DeepMind, Anthropic CEOs

Arjun Panickssery, Baldassare Castiglione and Cleo Nardo

24 May 2023 21:06 UTC

34 points

1 comment1 min readLW link

(www.gov.uk)

(notes on) Policy Desiderata for Superintelligent AI: A Vector Field Approach

Ben Pace4 Feb 2019 22:08 UTC

43 points

5 comments7 min readLW link

AI Governance: A Research Agenda

habryka5 Sep 2018 18:00 UTC

25 points

3 comments1 min readLW link

(www.fhi.ox.ac.uk)

My Updating Thoughts on AI policy

Ben Pace1 Mar 2020 7:06 UTC

20 points

1 comment9 min readLW link

How are voluntary commitments on vulnerability reporting going?

Adam Jones22 Feb 2024 8:43 UTC

23 points

1 comment1 min readLW link

(adamjones.me)

Global online debate on the governance of AI

CarolineJ5 Jan 2018 15:31 UTC

8 points

5 comments1 min readLW link

[AN #61] AI policy and governance, from two people in the field

Rohin Shah5 Aug 2019 17:00 UTC

12 points

2 comments9 min readLW link

(mailchi.mp)

A brief review of China’s AI industry and regulations

Elliot_Mckernon14 Mar 2024 12:19 UTC

22 points

0 comments16 min readLW link

Two ideas for alignment, perpetual mutual distrust and induction

APaleBlueDot25 May 2023 0:56 UTC

1 point

2 comments4 min readLW link

[Question] What could a policy banning AGI look like?

TsviBT13 Mar 2024 14:19 UTC

65 points

21 comments3 min readLW link

Book review: Architects of Intelligence by Martin Ford (2018)

Ofer11 Aug 2020 17:30 UTC

15 points

0 comments2 min readLW link

misc raw responses to a tract of Critical Rationalism

mako yass14 Aug 2020 11:53 UTC

21 points

52 comments3 min readLW link

Deciphering China’s AI Dream

Qiaochu_Yuan18 Mar 2018 3:26 UTC

12 points

2 comments1 min readLW link

(www.fhi.ox.ac.uk)

AI Incident Reporting: A Regulatory Review

Deric Cheng and Elliot_Mckernon

11 Mar 2024 21:03 UTC

15 points

0 comments6 min readLW link

China’s Plan to ‘Lead’ in AI: Purpose, Prospects, and Problems

fortyeridania10 Aug 2017 1:54 UTC

7 points

5 comments1 min readLW link

(www.newamerica.org)

[Question] Would more model evals teams be good?

Ryan Kidd25 Feb 2023 22:01 UTC

20 points

4 comments1 min readLW link

Trajectories to 2036

ukc1001420 Oct 2022 20:23 UTC

3 points

1 comment14 min readLW link

Apply to HAIST/MAIA’s AI Governance Workshop in DC (Feb 17-20)

Phosphorous, Xander Davies, CMD, Paramedic and tlevin

31 Jan 2023 2:06 UTC

28 points

0 comments2 min readLW link

WaPo: “Big Tech was moving cautiously on AI. Then came ChatGPT.”

Julian Bradshaw27 Jan 2023 22:54 UTC

26 points

5 comments1 min readLW link

(www.washingtonpost.com)

Tort Law Can Play an Important Role in Mitigating AI Risk

Gabriel Weil12 Feb 2024 17:17 UTC

37 points

9 comments5 min readLW link

Self-regulation of safety in AI research

Gordon Seidoh Worley25 Feb 2018 23:17 UTC

12 points

6 comments2 min readLW link

Proposal: labs should precommit to pausing if an AI argues for itself to be improved

NickGabs2 Jun 2023 22:31 UTC

3 points

3 comments4 min readLW link

[Link Post] Cyber Digital Authoritarianism (National Intelligence Council Report)

Phosphorous26 Feb 2023 20:51 UTC

12 points

2 comments1 min readLW link

(www.dni.gov)

Trends in the dollar training cost of machine learning systems

Ben Cottier1 Feb 2023 14:48 UTC

23 points

0 comments2 min readLW link

(epochai.org)

One implementation of regulatory GPU restrictions

porby4 Jun 2023 20:34 UTC

32 points

6 comments5 min readLW link

[FICTION] Unboxing Elysium: An AI’S Escape

Super AGI10 Jun 2023 4:41 UTC

−14 points

4 comments14 min readLW link

[FICTION] Prometheus Rising: The Emergence of an AI Consciousness

Super AGI10 Jun 2023 4:41 UTC

−13 points

0 comments9 min readLW link

The Slippery Slope from DALLE-2 to Deepfake Anarchy

scasper5 Nov 2022 14:53 UTC

17 points

9 comments11 min readLW link

Instead of technical research, more people should focus on buying time

Akash, OliviaJ and Thomas Larsen

5 Nov 2022 20:43 UTC

100 points

45 comments14 min readLW link

Using Consensus Mechanisms as an approach to Alignment

Prometheus10 Jun 2023 23:38 UTC

9 points

2 comments6 min readLW link

[Question] AI Rights: In your view, what would be required for an AGI to gain rights and protections from the various Governments of the World?

Super AGI9 Jun 2023 1:24 UTC

10 points

26 comments1 min readLW link

Why AI may not save the World

Alberto Zannoni9 Jun 2023 17:42 UTC

0 points

0 comments4 min readLW link

(a16z.com)

Applying superintelligence without collusion

Eric Drexler8 Nov 2022 18:08 UTC

107 points

63 comments4 min readLW link

Anthropic | Charting a Path to AI Accountability

Gabriel Mukobi14 Jun 2023 4:43 UTC

34 points

2 comments3 min readLW link

(www.anthropic.com)

Ban development of unpredictable powerful models?

TurnTrout20 Jun 2023 1:43 UTC

46 points

25 comments4 min readLW link

EU AI Act passed Plenary vote, and X-risk was a main topic

Ariel G.21 Jun 2023 18:33 UTC

17 points

0 comments1 min readLW link

(forum.effectivealtruism.org)

Slaying the Hydra: toward a new game board for AI

Prometheus23 Jun 2023 17:04 UTC

0 points

5 comments6 min readLW link

Ways to buy time

Akash, OliviaJ and Thomas Larsen

12 Nov 2022 19:31 UTC

34 points

23 comments12 min readLW link

The economy as an analogy for advanced AI systems

rosehadshar and particlemania

15 Nov 2022 11:16 UTC

28 points

0 comments5 min readLW link

Call for Cruxes by Rhyme, a Longtermist History Consultancy

Lara1 Mar 2023 18:39 UTC

1 point

0 comments3 min readLW link

(forum.effectivealtruism.org)

Announcing Epoch: A research organization investigating the road to Transformative AI

Jsevillamol, Pablo Villalobos, Tamay, lennart, Marius Hobbhahn and anson.ho

27 Jun 2022 13:55 UTC

96 points

2 comments2 min readLW link

(epochai.org)

Biosafety Regulations (BMBL) and their relevance for AI

Štěpán Los29 Jun 2023 19:22 UTC

4 points

0 comments4 min readLW link

AI Incident Sharing—Best practices from other fields and a comprehensive list of existing platforms

Štěpán Los28 Jun 2023 17:21 UTC

20 points

0 comments4 min readLW link

Optimising Society to Constrain Risk of War from an Artificial Superintelligence

JohnCDraper30 Apr 2020 10:47 UTC

3 points

1 comment51 min readLW link

Superintelligence 7: Decisive strategic advantage

KatjaGrace28 Oct 2014 1:01 UTC

19 points

60 comments6 min readLW link

Superintelligence 17: Multipolar scenarios

KatjaGrace6 Jan 2015 6:44 UTC

9 points

38 comments6 min readLW link

Superintelligence 22: Emulation modulation and institutional design

KatjaGrace10 Feb 2015 2:06 UTC

13 points

11 comments6 min readLW link

Superintelligence 26: Science and technology strategy

KatjaGrace10 Mar 2015 1:43 UTC

14 points

21 comments6 min readLW link

Superintelligence 27: Pathways and enablers

KatjaGrace17 Mar 2015 1:00 UTC

15 points

21 comments8 min readLW link

Superintelligence 28: Collaboration

KatjaGrace24 Mar 2015 1:29 UTC

13 points

21 comments6 min readLW link

Superintelligence 29: Crunch time

KatjaGrace31 Mar 2015 4:24 UTC

14 points

27 comments6 min readLW link

Foresight for AGI Safety Strategy: Mitigating Risks and Identifying Golden Opportunities

jacquesthibs5 Dec 2022 16:09 UTC

28 points

6 comments8 min readLW link

An AGI kill switch with defined security properties

Peterpiper5 Jul 2023 17:40 UTC

−5 points

6 comments1 min readLW link

Scenario planning for AI x-risk

Corin Katzke10 Feb 2024 0:14 UTC

22 points

9 comments14 min readLW link

(forum.effectivealtruism.org)

GPT-7: The Tale of the Big Computer (An Experimental Story)

Justin Bullock10 Jul 2023 20:22 UTC

4 points

4 comments5 min readLW link

Empirical Evidence Against “The Longest Training Run”

NickGabs6 Jul 2023 18:32 UTC

24 points

0 comments14 min readLW link

Anthropic: Core Views on AI Safety: When, Why, What, and How

jonmenaster9 Mar 2023 17:34 UTC

17 points

1 comment22 min readLW link

(www.anthropic.com)

Existential AI Safety is NOT separate from near-term applications

scasper13 Dec 2022 14:47 UTC

37 points

17 comments3 min readLW link

What is everyone doing in AI governance

Igor Ivanov8 Jul 2023 15:16 UTC

10 points

0 comments5 min readLW link

What Failure Looks Like is not an existential risk (and alignment is not the solution)

otto.barten2 Feb 2024 18:59 UTC

13 points

12 comments9 min readLW link

How I Learned To Stop Worrying And Love The Shoggoth

Peter Merel12 Jul 2023 17:47 UTC

10 points

9 comments5 min readLW link

[Question] What criterion would you use to select companies likely to cause AI doom?

momom213 Jul 2023 20:31 UTC

8 points

4 comments1 min readLW link

Thoughts On Expanding the AI Safety Community: Benefits and Challenges of Outreach to Non-Technical Professionals

Yashvardhan Sharma1 Jan 2023 19:21 UTC

6 points

4 comments7 min readLW link

Why was the AI Alignment community so unprepared for this moment?

Ras151315 Jul 2023 0:26 UTC

118 points

64 comments2 min readLW link

Google may be trying to take over the world

[deleted]27 Jan 2014 9:33 UTC

33 points

133 comments1 min readLW link

Scaling and Sustaining Standards: A Case Study on the Basel Accords

Conrad K.16 Jul 2023 22:01 UTC

8 points

1 comment7 min readLW link

(docs.google.com)

A fictional AI law laced w/ alignment theory

MiguelDev17 Jul 2023 1:42 UTC

6 points

0 comments2 min readLW link

OpenAI Credit Account (2510$)

Emirhan BULUT21 Jan 2024 2:32 UTC

1 point

0 comments1 min readLW link

[Crosspost] An AI Pause Is Humanity’s Best Bet For Preventing Extinction (TIME)

otto.barten24 Jul 2023 10:07 UTC

12 points

0 comments7 min readLW link

(time.com)

Priorities for the UK Foundation Models Taskforce

Andrea_Miotti21 Jul 2023 15:23 UTC

105 points

4 comments5 min readLW link

(www.conjecture.dev)

AGI Timelines in Governance: Different Strategies for Different Timeframes

simeon_c and AmberDawn

19 Dec 2022 21:31 UTC

63 points

28 comments10 min readLW link

Announcing Convergence Analysis: An Institute for AI Scenario & Governance Research

David_Kristoffersson and Deric Cheng

7 Mar 2024 21:37 UTC

22 points

1 comment4 min readLW link

Partial Transcript of Recent Senate Hearing Discussing AI X-Risk

Daniel_Eth27 Jul 2023 9:16 UTC

55 points

0 comments1 min readLW link

(medium.com)

EU’s AI ambitions at risk as US pushes to water down international treaty (linkpost)

mic31 Jul 2023 0:34 UTC

10 points

0 comments4 min readLW link

(www.euractiv.com)

Trading off compute in training and inference (Overview)

Pablo Villalobos31 Jul 2023 16:03 UTC

31 points

1 comment7 min readLW link

(epochai.org)

AI romantic partners will harm society if they go unregulated

Roman Leventov1 Aug 2023 9:32 UTC

25 points

71 comments13 min readLW link

Rebooting AI Governance: An AI-Driven Approach to AI Governance

Max Reddel6 Aug 2023 14:19 UTC

1 point

1 comment29 min readLW link

(forum.effectivealtruism.org)

Seeking Input to AI Safety Book for non-technical audience

Darren McKee10 Aug 2023 17:58 UTC

10 points

4 comments1 min readLW link

AI race considerations in a report by the U.S. House Committee on Armed Services

NunoSempere4 Oct 2020 12:11 UTC

42 points

4 comments13 min readLW link

Security Mindset—Fire Alarms and Trigger Signatures

elspood9 Feb 2023 21:15 UTC

23 points

0 comments4 min readLW link

Singletons Rule OK

Eliezer Yudkowsky30 Nov 2008 16:45 UTC

23 points

47 comments5 min readLW link

Large Language Models will be Great for Censorship

Ethan Edwards21 Aug 2023 19:03 UTC

183 points

14 comments8 min readLW link

(ethanedwards.substack.com)

AI Regulation May Be More Important Than AI Alignment For Existential Safety

otto.barten24 Aug 2023 11:41 UTC

65 points

39 comments5 min readLW link

A concerning observation from media coverage of AI industry dynamics

Justin Olive5 Mar 2023 21:38 UTC

8 points

3 comments3 min readLW link

Political Biases in LLMs: Literature Review & Current Uses of AI in Elections

Yashvardhan Sharma, Robayet Hossain and Ariana Gamarra

7 Mar 2024 19:17 UTC

8 points

0 comments6 min readLW link

Accurate Models of AI Risk Are Hyperexistential Exfohazards

Thane Ruthenis25 Dec 2022 16:50 UTC

30 points

38 comments9 min readLW link

Introducing the Center for AI Policy (& we’re hiring!)

Thomas Larsen28 Aug 2023 21:17 UTC

119 points

50 comments2 min readLW link

(www.aipolicy.us)

Equilibrium and prior selection problems in multipolar deployment

JesseClifton2 Apr 2020 20:06 UTC

21 points

11 comments11 min readLW link

Notes on nukes, IR, and AI from “Arsenals of Folly” (and other books)

tlevin4 Sep 2023 19:02 UTC

11 points

0 comments6 min readLW link

Institutions Cannot Restrain Dark-Triad AI Exploitation

Remmelt and flandry19

27 Dec 2022 10:34 UTC

5 points

0 comments5 min readLW link

(mflb.com)

Public Opinion on AI Safety: AIMS 2023 and 2021 Summary

Jacy Reese Anthis, Janet Pauketat and Ali

25 Sep 2023 18:55 UTC

3 points

2 comments3 min readLW link

(www.sentienceinstitute.org)

Careless talk on US-China AI competition? (and criticism of CAIS coverage)

Oliver Sourbut20 Sep 2023 12:46 UTC

3 points

0 comments10 min readLW link

(www.oliversourbut.net)

Five neglected work areas that could reduce AI risk

CharlotteS and Aaron_Scher

24 Sep 2023 2:03 UTC

17 points

5 comments9 min readLW link

International cooperation vs. AI arms race

Brian_Tomasik5 Dec 2013 1:09 UTC

23 points

144 comments4 min readLW link

[Question] What is the minimum amount of time travel and resources needed to secure the future?

Perhaps14 Jan 2024 22:01 UTC

−3 points

5 comments1 min readLW link

List of projects that seem impactful for AI Governance

JaimeRV and Teun van der Weij

14 Jan 2024 16:53 UTC

13 points

0 comments13 min readLW link

Avoiding perpetual risk from TAI

scasper26 Dec 2022 22:34 UTC

15 points

6 comments5 min readLW link

Update on the UK AI Taskforce & upcoming AI Safety Summit

Elliot_Mckernon11 Oct 2023 11:37 UTC

82 points

2 comments4 min readLW link

A New Model for Compute Center Verification

Damin Curtis10 Oct 2023 19:22 UTC

8 points

0 comments5 min readLW link

[Question] Looking for reading recommendations: Theories of right/justice that safeguard against having one’s job automated?

bulKlub12 Oct 2023 19:40 UTC

−1 points

1 comment1 min readLW link

unRLHF—Efficiently undoing LLM safeguards

Pranav Gade, Jeffrey Ladish and Simon Lermen

12 Oct 2023 19:58 UTC

116 points

15 comments20 min readLW link

The International PauseAI Protest: Activism under uncertainty

Joseph Miller12 Oct 2023 17:36 UTC

32 points

1 comment1 min readLW link

FLI podcast series, “Imagine A World”, about aspirational futures with AGI

Jackson Wagner13 Oct 2023 16:07 UTC

9 points

0 comments4 min readLW link

To open-source or to not open-source, that is (an oversimplification of) the question.

Justin Bullock13 Oct 2023 15:10 UTC

11 points

5 comments5 min readLW link

AISU 2021

Linda Linsefors30 Jan 2021 17:40 UTC

28 points

2 comments1 min readLW link

2021-03-01 National Library of Medicine Presentation: “Atlas of AI: Mapping the social and economic forces behind AI”

IrenicTruth17 Feb 2021 18:23 UTC

1 point

0 comments2 min readLW link

Survey on intermediate goals in AI governance

MichaelA and MaxRa

17 Mar 2023 13:12 UTC

25 points

3 comments1 min readLW link

[Question] Is there anything that can stop AGI development in the near term?

Wulky Wilkinsen22 Apr 2021 20:37 UTC

5 points

5 comments1 min readLW link

Controlling Intelligent Agents The Only Way We Know How: Ideal Bureaucratic Structure (IBS)

Justin Bullock24 May 2021 12:53 UTC

14 points

15 comments6 min readLW link

Reflection of Hierarchical Relationship via Nuanced Conditioning of Game Theory Approach for AI Development and Utilization

Kyoung-cheol Kim4 Jun 2021 7:20 UTC

2 points

2 comments7 min readLW link

The Governance Problem and the “Pretty Good” X-Risk

Zach Stein-Perlman29 Aug 2021 18:00 UTC

5 points

2 comments11 min readLW link

Nuclear Espionage and AI Governance

GAA4 Oct 2021 23:04 UTC

32 points

5 comments24 min readLW link

Compute Governance and Conclusions—Transformative AI and Compute [3/4]

lennart14 Oct 2021 8:23 UTC

13 points

0 comments5 min readLW link

Truthful AI: Developing and governing AI that does not lie

Owain_Evans, owencb and Lukas Finnveden

18 Oct 2021 18:37 UTC

81 points

9 comments10 min readLW link

AMA on Truthful AI: Owen Cotton-Barratt, Owain Evans & co-authors

Owain_Evans22 Oct 2021 16:23 UTC

31 points

15 comments1 min readLW link

AI Tracker: monitoring current and near-future risks from superscale models

Edouard Harris and Jeremie Harris

23 Nov 2021 19:16 UTC

64 points

13 comments3 min readLW link

(aitracker.org)

Should AI systems have to identify themselves?

Darren McKee31 Dec 2022 2:57 UTC

2 points

2 comments1 min readLW link

AI Governance Fundamentals—Curriculum and Application

Mau30 Nov 2021 2:19 UTC

17 points

0 comments1 min readLW link

HIRING: Inform and shape a new project on AI safety at Partnership on AI

madhu_lika7 Dec 2021 19:37 UTC

1 point

0 comments1 min readLW link

Towards AI Safety Infrastructure: Talk & Outline

Paul Bricman7 Jan 2024 9:31 UTC

10 points

0 comments2 min readLW link

(www.youtube.com)

Demanding and Designing Aligned Cognitive Architectures

Koen.Holtman21 Dec 2021 17:32 UTC

8 points

5 comments5 min readLW link

Question 4: Implementing the control proposals

Cameron Berg13 Feb 2022 17:12 UTC

6 points

2 comments5 min readLW link

How harmful are improvements in AI? + Poll

tilmanr and Marius Hobbhahn

15 Feb 2022 18:16 UTC

15 points

4 comments8 min readLW link

AI Safety Evaluations: A Regulatory Review

Elliot_Mckernon and Deric Cheng

19 Mar 2024 15:05 UTC

20 points

1 comment11 min readLW link

Focusing on Mal-Alignment

John Fisher2 Jan 2024 19:51 UTC

1 point

0 comments1 min readLW link

Static vs Dynamic Alignment

Gracie Green21 Mar 2024 17:44 UTC

4 points

0 comments29 min readLW link

AI Model Registries: A Regulatory Review

Deric Cheng and Elliot_Mckernon

22 Mar 2024 16:04 UTC

8 points

0 comments6 min readLW link

UNGA Resolution on AI: 5 Key Takeaways Looking to Future Policy

Heramb24 Mar 2024 12:23 UTC

3 points

0 comments3 min readLW link

(forum.effectivealtruism.org)

Idea: Safe Fallback Regulations for Widely Deployed AI Systems

Aaron_Scher25 Mar 2024 21:27 UTC

4 points

0 comments6 min readLW link

Timelines to Transformative AI: an investigation

Zershaaneh Qureshi26 Mar 2024 18:28 UTC

12 points

2 comments50 min readLW link

AI Disclosures: A Regulatory Review

Elliot_Mckernon and Deric Cheng

29 Mar 2024 11:42 UTC

10 points

0 comments7 min readLW link

God Coin: A Modest Proposal

Mahdi Complex1 Apr 2024 12:04 UTC

−8 points

5 comments22 min readLW link

AI Discrimination Requirements: A Regulatory Review

Deric Cheng and Elliot_Mckernon

4 Apr 2024 15:43 UTC

6 points

0 comments6 min readLW link

Announcing Atlas Computing

miyazono11 Apr 2024 15:56 UTC

44 points

4 comments4 min readLW link

Apply to the Pivotal Research Fellowship (AI Safety & Biosecurity)

Tobias H and tilmanr

10 Apr 2024 12:08 UTC

17 points

0 comments1 min readLW link

Customer-Centric AI: the Major Paradigm Shift in AI Governance (Part 1)

Ana Chubinidze11 Apr 2024 17:10 UTC

1 point

0 comments1 min readLW link

(anachubinidze.substack.com)

Report: Evaluating an AI Chip Registration Policy

Deric Cheng12 Apr 2024 4:39 UTC

25 points

0 comments5 min readLW link

(www.convergenceanalysis.org)

NAIRA—An exercise in regulatory, competitive safety governance [AI Governance Institutional Design idea]

Heramb19 Mar 2024 17:43 UTC

2 points

0 comments1 min readLW link

(forum.effectivealtruism.org)

EU policymakers reach an agreement on the AI Act

tlevin15 Dec 2023 6:02 UTC

75 points

7 comments7 min readLW link

Exploring the Precautionary Principle in AI Development: Historical Analogies and Lessons Learned

Christopher King21 Mar 2023 3:53 UTC

−1 points

2 comments9 min readLW link

After Overmorrow: Scattered Musings on the Immediate Post-AGI World

Yuli_Ban24 Feb 2024 15:49 UTC

−3 points

0 comments26 min readLW link

CAIS-inspired approach towards safer and more interpretable AGIs

Peter Hroššo27 Mar 2023 14:36 UTC

13 points

7 comments1 min readLW link

AI security might be helpful for AI alignment

Igor Ivanov6 Jan 2023 20:16 UTC

35 points

1 comment2 min readLW link

Want to win the AGI race? Solve alignment.

leopold29 Mar 2023 17:40 UTC

21 points

3 comments5 min readLW link

(www.forourposterity.com)

The 0.2 OOMs/year target

Cleo Nardo30 Mar 2023 18:15 UTC

84 points

24 comments5 min readLW link

Widening Overton Window—Open Thread

Prometheus31 Mar 2023 10:03 UTC

23 points

8 comments1 min readLW link

AI safety advocates should consider providing gentle pushback following the events at OpenAI

civilsociety22 Dec 2023 18:55 UTC

16 points

5 comments3 min readLW link

Pausing AI Developments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky

jacquesthibs29 Mar 2023 23:16 UTC

298 points

296 comments3 min readLW link

(time.com)

AI governance student hackathon on Saturday, April 23: register now!

mic12 Apr 2022 4:48 UTC

14 points

0 comments1 min readLW link

AI community building: EliezerKart

Christopher King1 Apr 2023 15:25 UTC

45 points

0 comments2 min readLW link

Pessimism about AI Safety

Max_He-Ho and Peter Kuhn

2 Apr 2023 7:43 UTC

4 points

1 comment25 min readLW link

Law-Following AI 1: Sequence Introduction and Structure

Cullen27 Apr 2022 17:26 UTC

18 points

10 comments9 min readLW link

The AI governance gaps in developing countries

nguyên17 Jun 2023 2:50 UTC

20 points

1 comment14 min readLW link

Law-Following AI 2: Intent Alignment + Superintelligence → Lawless AI (By Default)

Cullen27 Apr 2022 17:27 UTC

5 points

2 comments6 min readLW link

Law-Following AI 3: Lawless AI Agents Undermine Stabilizing Agreements

Cullen27 Apr 2022 17:30 UTC

2 points

2 comments3 min readLW link

AI Alternative Futures: Scenario Mapping Artificial Intelligence Risk—Request for Participation (Closed)

Kakili27 Apr 2022 22:07 UTC

10 points

2 comments8 min readLW link

Yoshua Bengio: “Slowing down development of AI systems passing the Turing test”

Roman Leventov6 Apr 2023 3:31 UTC

49 points

2 comments5 min readLW link

(yoshuabengio.org)

Risks from GPT-4 Byproduct of Recursively Optimizing AIs

ben hayum7 Apr 2023 0:02 UTC

73 points

9 comments10 min readLW link

(forum.effectivealtruism.org)

Quick Thoughts on A.I. Governance

NicholasKross30 Apr 2022 14:49 UTC

69 points

8 comments2 min readLW link

(www.thinkingmuchbetter.com)

AI safety should be made more accessible using non text-based media

Massimog10 May 2022 3:14 UTC

2 points

4 comments4 min readLW link

DeepMind’s generalist AI, Gato: A non-technical explainer

frances_lorenz, Nora Belrose and jonmenaster

16 May 2022 21:21 UTC

63 points

6 comments6 min readLW link

Open positions: Research Analyst at the AI Standards Lab

Koen.Holtman, Jonathan_H and Ariel G.

22 Dec 2023 16:31 UTC

17 points

0 comments1 min readLW link

A bridge to Dath Ilan? Improved governance on the critical path to AI alignment.

Jackson Wagner18 May 2022 15:51 UTC

24 points

0 comments12 min readLW link

Reshaping the AI Industry

Thane Ruthenis29 May 2022 22:54 UTC

147 points

35 comments21 min readLW link

Six Dimensions of Operational Adequacy in AGI Projects

Eliezer Yudkowsky30 May 2022 17:00 UTC

299 points

66 comments13 min readLW link 1 review

Open-source LLMs may prove Bostrom’s vulnerable world hypothesis

Roope Ahvenharju15 Apr 2023 19:16 UTC

1 point

1 comment1 min readLW link

[Question] Could Patent-Trolling delay AI timelines?

Pablo Repetto10 Jun 2022 2:53 UTC

1 point

3 comments1 min readLW link

Controlling AGI Risk

TeaSea15 Mar 2024 4:56 UTC

6 points

8 comments4 min readLW link

[Link/crosspost] [US] NTIA: AI Accountability Policy Request for Comment

Kyle J. Lucchese16 Apr 2023 6:57 UTC

8 points

0 comments1 min readLW link

(forum.effectivealtruism.org)

FYI: I’m working on a book about the threat of AGI/ASI for a general audience. I hope it will be of value to the cause and the community

Darren McKee15 Jun 2022 18:08 UTC

42 points

15 comments2 min readLW link

Financial Times: We must slow down the race to God-like AI

trevor13 Apr 2023 19:55 UTC

103 points

17 comments16 min readLW link

(www.ft.com)

Scientism vs. people

Roman Leventov18 Apr 2023 17:28 UTC

4 points

4 comments11 min readLW link

[Crosspost] Organizing a debate with experts and MPs to raise AI xrisk awareness: a possible blueprint

otto.barten19 Apr 2023 11:45 UTC

8 points

0 comments4 min readLW link

(forum.effectivealtruism.org)

Davidad’s Bold Plan for Alignment: An In-Depth Explanation

Charbel-Raphaël and Gabin

19 Apr 2023 16:09 UTC

154 points

29 comments21 min readLW link

Protectionism will Slow the Deployment of AI

bgold7 Jan 2023 20:57 UTC

30 points

6 comments2 min readLW link

What success looks like

Marius Hobbhahn, MaxRa, JasperGeh and Yannick_Muehlhaeuser

28 Jun 2022 14:38 UTC

19 points

4 comments1 min readLW link

(forum.effectivealtruism.org)

Pausing AI Developments Isn’t Enough. We Need to Shut it All Down

Eliezer Yudkowsky8 Apr 2023 0:36 UTC

248 points

39 comments12 min readLW link

Briefly how I’ve updated since ChatGPT

rime25 Apr 2023 14:47 UTC

48 points

2 comments2 min readLW link

New US Senate Bill on X-Risk Mitigation [Linkpost]

Evan R. Murphy4 Jul 2022 1:25 UTC

35 points

12 comments1 min readLW link

(www.hsgac.senate.gov)

Please help us communicate AI xrisk. It could save the world.

otto.barten4 Jul 2022 21:47 UTC

4 points

7 comments2 min readLW link

Announcing #AISummitTalks featuring Professor Stuart Russell and many others

otto.barten24 Oct 2023 10:11 UTC

17 points

1 comment1 min readLW link

Slowing down AI progress is an underexplored alignment strategy

Norman Borlaug24 Jul 2023 16:56 UTC

40 points

27 comments5 min readLW link

A Critique of AI Alignment Pessimism

ExCeph19 Jul 2022 2:28 UTC

9 points

1 comment9 min readLW link

Law-Following AI 4: Don’t Rely on Vicarious Liability

Cullen2 Aug 2022 23:26 UTC

5 points

2 comments3 min readLW link

Responsible Scaling Policies Are Risk Management Done Wrong

simeon_c25 Oct 2023 23:46 UTC

114 points

33 comments22 min readLW link

(www.navigatingrisks.ai)

Three pillars for avoiding AGI catastrophe: Technical alignment, deployment decisions, and coordination

Alex Lintz3 Aug 2022 23:15 UTC

22 points

0 comments12 min readLW link

Linkpost: Rishi Sunak’s Speech on AI (26th October)

bideup27 Oct 2023 11:57 UTC

85 points

8 comments7 min readLW link

(www.gov.uk)

Disagreements over the prioritization of existential risk from AI

Olivier Coutu26 Oct 2023 17:54 UTC

10 points

0 comments6 min readLW link

Corporate Governance for Frontier AI Labs: A Research Agenda

Matthew Wearden28 Feb 2024 11:29 UTC

4 points

0 comments16 min readLW link

(matthewwearden.co.uk)

[Linkpost] Two major announcements in AI governance today

Angélina30 Oct 2023 17:28 UTC

1 point

1 comment1 min readLW link

(www.whitehouse.gov)

Response to “Coordinated pausing: An evaluation-based coordination scheme for frontier AI developers”

Matthew Wearden30 Oct 2023 17:27 UTC

5 points

2 comments6 min readLW link

(matthewwearden.co.uk)

Cap Model Size for AI Safety

research_prime_space6 Mar 2023 1:11 UTC

0 points

4 comments1 min readLW link

Alignment is not enough

Alan Chan12 Jan 2023 0:33 UTC

11 points

6 comments11 min readLW link

(coordination.substack.com)

Matt Yglesias on AI Policy

Grant Demaree17 Aug 2022 23:57 UTC

25 points

1 comment1 min readLW link

(www.slowboring.com)

A Nail in the Coffin of Exceptionalism

Yeshua God14 Mar 2024 22:41 UTC

−18 points

0 comments3 min readLW link

[Question] Should AI writers be prohibited in education?

Eleni Angelou17 Jan 2023 0:42 UTC

6 points

2 comments1 min readLW link

Thoughts on the AI Safety Summit company policy requests and responses

So8res31 Oct 2023 23:54 UTC

169 points

14 comments10 min readLW link

Compute Governance: The Role of Commodity Hardware

Jan26 Mar 2022 10:08 UTC

14 points

7 comments7 min readLW link

(universalprior.substack.com)

Why don’t governments seem to mind that companies are explicitly trying to make AGIs?

ozziegooen26 Dec 2021 1:58 UTC

34 points

3 comments2 min readLW link

(forum.effectivealtruism.org)

AI Governance Needs Technical Work

Mau5 Sep 2022 22:28 UTC

41 points

1 comment8 min readLW link

AI as Super-Demagogue

RationalDino5 Nov 2023 21:21 UTC

−2 points

9 comments9 min readLW link

What Should AI Owe To Us? Accountable and Aligned AI Systems via Contractualist AI Alignment

xuan8 Sep 2022 15:04 UTC

32 points

15 comments25 min readLW link

Scalable And Transferable Black-Box Jailbreaks For Language Models Via Persona Modulation

Soroush Pour, rusheb, Quentin FEUILLADE--MONTIXI, Arush and scasper

7 Nov 2023 17:59 UTC

36 points

2 comments2 min readLW link

(arxiv.org)

How should DeepMind’s Chinchilla revise our AI forecasts?

Cleo Nardo15 Sep 2022 17:54 UTC

35 points

12 comments13 min readLW link

Update on the UK AI Summit and the UK’s Plans

Elliot_Mckernon10 Nov 2023 14:47 UTC

10 points

0 comments8 min readLW link

Leveraging Legal Informatics to Align AI

John Nay18 Sep 2022 20:39 UTC

11 points

0 comments3 min readLW link

(forum.effectivealtruism.org)

Theories of Change for AI Auditing

Lee Sharkey, beren and Marius Hobbhahn

13 Nov 2023 19:33 UTC

59 points

0 comments18 min readLW link

(www.apolloresearch.ai)

Palisade is hiring Research Engineers

Charlie Rogers-Smith and Jeffrey Ladish

11 Nov 2023 3:09 UTC

22 points

0 comments3 min readLW link

Automated Sandwiching & Quantifying Human-LLM Cooperation: ScaleOversight hackathon results

Esben Kran, Fazl, Sabrina Zaki, gabrielrecc and rz2383

23 Feb 2023 10:48 UTC

8 points

0 comments6 min readLW link

On excluding dangerous information from training

ShayBenMoshe17 Nov 2023 11:14 UTC

23 points

5 comments3 min readLW link

1. A Sense of Fairness: Deconfusing Ethics

RogerDearnaley17 Nov 2023 20:55 UTC

15 points

8 comments15 min readLW link

2. AIs as Economic Agents

RogerDearnaley23 Nov 2023 7:07 UTC

9 points

2 comments6 min readLW link

4. A Moral Case for Evolved-Sapience-Chauvinism

RogerDearnaley24 Nov 2023 4:56 UTC

10 points

0 comments4 min readLW link

3. Uploading

RogerDearnaley23 Nov 2023 7:39 UTC

21 points

5 comments8 min readLW link

Emotional attachment to AIs opens doors to problems

Igor Ivanov22 Jan 2023 20:28 UTC

20 points

10 comments4 min readLW link

A call for a quantitative report card for AI bioterrorism threat models

Juno4 Dec 2023 6:35 UTC

12 points

0 comments10 min readLW link

In defence of Helen Toner, Adam D’Angelo, and Tasha McCauley (OpenAI post)

mrtreasure5 Dec 2023 18:40 UTC

6 points

2 comments1 min readLW link

(pastebin.com)

In defence of Helen Toner, Adam D’Angelo, and Tasha McCauley

mrtreasure6 Dec 2023 2:02 UTC

25 points

3 comments9 min readLW link

(pastebin.com)

(Report) Evaluating Taiwan’s Tactics to Safeguard its Semiconductor Assets Against a Chinese Invasion

Gauraventh7 Dec 2023 11:50 UTC

16 points

5 comments22 min readLW link

(bristolaisafety.org)

Call for submissions: Choice of Futures survey questions

c.trout30 Apr 2023 6:59 UTC

4 points

0 comments2 min readLW link

(airtable.com)

[Question] Any further work on AI Safety Success Stories?

Krieger2 Oct 2022 9:53 UTC

8 points

6 comments1 min readLW link

Averting Catastrophe: Decision Theory for COVID-19, Climate Change, and Potential Disasters of All Kinds

JakubK2 May 2023 22:50 UTC

10 points

0 comments1 min readLW link

No comments.