AI Risk

TagLast edit: 2 Nov 2022 20:48 UTC by brook

AI Risk is analysis of the risks associated with building powerful AI systems.

Superintelligence FAQ

Scott Alexander20 Sep 2016 19:00 UTC

93 points

16 comments27 min readLW link

What failure looks like

paulfchristiano17 Mar 2019 20:18 UTC

325 points

49 comments8 min readLW link 2 reviews

Specification gaming examples in AI

Vika3 Apr 2018 12:30 UTC

43 points

9 comments1 min readLW link 2 reviews

Discussion with Eliezer Yudkowsky on AGI interventions

Rob Bensinger and Eliezer Yudkowsky

11 Nov 2021 3:01 UTC

326 points

258 comments34 min readLW link 1 review

PreDCA: vanessa kosoy’s alignment protocol

carado20 Aug 2022 10:03 UTC

46 points

8 comments7 min readLW link

(carado.moe)

Intuitions about goal-directed behavior

Rohin Shah1 Dec 2018 4:25 UTC

52 points

15 comments6 min readLW link

Epistemological Framing for AI Alignment Research

adamShimi8 Mar 2021 22:05 UTC

53 points

7 comments9 min readLW link

AGI Ruin: A List of Lethalities

Eliezer Yudkowsky5 Jun 2022 22:05 UTC

737 points

653 comments30 min readLW link

What can the principal-agent literature tell us about AI risk?

Alexis Carlier8 Feb 2020 21:28 UTC

102 points

31 comments16 min readLW link

MIRI announces new “Death With Dignity” strategy

Eliezer Yudkowsky2 Apr 2022 0:43 UTC

322 points

527 comments18 min readLW link

Developmental Stages of GPTs

orthonormal26 Jul 2020 22:03 UTC

140 points

74 comments7 min readLW link 1 review

A Gym Gridworld Environment for the Treacherous Turn

Michaël Trazzi28 Jul 2018 21:27 UTC

73 points

9 comments3 min readLW link

(github.com)

[Question] Will OpenAI’s work unintentionally increase existential risks related to AI?

adamShimi11 Aug 2020 18:16 UTC

53 points

56 comments1 min readLW link

Another (outer) alignment failure story

paulfchristiano7 Apr 2021 20:12 UTC

216 points

39 comments12 min readLW link 1 review

On how various plans miss the hard bits of the alignment challenge

So8res12 Jul 2022 2:49 UTC

263 points

81 comments29 min readLW link

AI Safety bounty for practical homomorphic encryption

acylhalide19 Aug 2022 12:27 UTC

29 points

9 comments4 min readLW link

Meta AI announces Cicero: Human-Level Diplomacy play (with dialogue)

Jacy Reese Anthis22 Nov 2022 16:50 UTC

95 points

64 comments1 min readLW link

(www.science.org)

AI will change the world, but won’t take it over by playing “3-dimensional chess”.

boazbarak and benedelman

22 Nov 2022 18:57 UTC

111 points

89 comments24 min readLW link

How good is humanity at coordination?

Buck21 Jul 2020 20:01 UTC

78 points

44 comments3 min readLW link

Are minimal circuits deceptive?

evhub7 Sep 2019 18:11 UTC

66 points

11 comments8 min readLW link

Soft takeoff can still lead to decisive strategic advantage

Daniel Kokotajlo23 Aug 2019 16:39 UTC

121 points

46 comments8 min readLW link 4 reviews

Should we postpone AGI until we reach safety?

otto.barten18 Nov 2020 15:43 UTC

27 points

36 comments3 min readLW link

On Solving Problems Before They Appear: The Weird Epistemologies of Alignment

adamShimi11 Oct 2021 8:20 UTC

100 points

11 comments15 min readLW link

DL towards the unaligned Recursive Self-Optimization attractor

jacob_cannell18 Dec 2021 2:15 UTC

32 points

22 comments4 min readLW link

Truthful LMs as a warm-up for aligned AGI

Jacob_Hilton17 Jan 2022 16:49 UTC

65 points

14 comments13 min readLW link

AI Could Defeat All Of Us Combined

HoldenKarnofsky9 Jun 2022 15:50 UTC

168 points

32 comments17 min readLW link

(www.cold-takes.com)

Intent alignment should not be the goal for AGI x-risk reduction

John Nay26 Oct 2022 1:24 UTC

−6 points

10 comments3 min readLW link

A challenge for AGI organizations, and a challenge for readers

Rob Bensinger and Eliezer Yudkowsky

1 Dec 2022 23:11 UTC

276 points

30 comments2 min readLW link

“Endgame safety” for AGI

Steven Byrnes24 Jan 2023 14:15 UTC

72 points

4 comments5 min readLW link

Alexander and Yudkowsky on AGI goals

Scott Alexander and Eliezer Yudkowsky

24 Jan 2023 21:09 UTC

147 points

49 comments26 min readLW link

Critiquing “What failure looks like”

Grue_Slinky27 Dec 2019 23:59 UTC

35 points

6 comments3 min readLW link

The Main Sources of AI Risk?

Daniel Kokotajlo and Wei_Dai

21 Mar 2019 18:28 UTC

105 points

25 comments2 min readLW link

Clarifying some key hypotheses in AI alignment

Ben Cottier and Rohin Shah

15 Aug 2019 21:29 UTC

79 points

12 comments9 min readLW link

“Taking AI Risk Seriously” (thoughts by Critch)

Raemon29 Jan 2018 9:27 UTC

110 points

68 comments13 min readLW link

Some conceptual highlights from “Disjunctive Scenarios of Catastrophic AI Risk”

Kaj_Sotala12 Feb 2018 12:30 UTC

33 points

4 comments6 min readLW link

(kajsotala.fi)

Non-Adversarial Goodhart and AI Risks

Davidmanheim27 Mar 2018 1:39 UTC

22 points

11 comments6 min readLW link

Six AI Risk/Strategy Ideas

Wei_Dai27 Aug 2019 0:40 UTC

64 points

18 comments4 min readLW link 1 review

[Question] Did AI pioneers not worry much about AI risks?

lisperati9 Feb 2020 19:58 UTC

42 points

9 comments1 min readLW link

Some disjunctive reasons for urgency on AI risk

Wei_Dai15 Feb 2019 20:43 UTC

36 points

24 comments1 min readLW link

Drexler on AI Risk

PeterMcCluskey1 Feb 2019 5:11 UTC

34 points

10 comments9 min readLW link

(www.bayesianinvestor.com)

A shift in arguments for AI risk

Richard_Ngo28 May 2019 13:47 UTC

32 points

7 comments1 min readLW link

(fragile-credences.github.io)

Disentangling arguments for the importance of AI safety

Richard_Ngo21 Jan 2019 12:41 UTC

129 points

23 comments8 min readLW link

AI Safety “Success Stories”

Wei_Dai7 Sep 2019 2:54 UTC

114 points

27 comments4 min readLW link 1 review

Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More

Ben Pace4 Oct 2019 4:08 UTC

206 points

60 comments15 min readLW link 2 reviews

[AN #80]: Why AI risk might be solved without additional intervention from longtermists

Rohin Shah2 Jan 2020 18:20 UTC

34 points

94 comments10 min readLW link

(mailchi.mp)

The strategy-stealing assumption

paulfchristiano16 Sep 2019 15:23 UTC

72 points

46 comments12 min readLW link 3 reviews

Thinking soberly about the context and consequences of Friendly AI

Mitchell_Porter16 Oct 2012 4:33 UTC

21 points

39 comments1 min readLW link

Announcement: AI alignment prize winners and next round

cousin_it15 Jan 2018 14:33 UTC

80 points

68 comments2 min readLW link

What Failure Looks Like: Distilling the Discussion

Ben Pace29 Jul 2020 21:49 UTC

79 points

14 comments7 min readLW link

Uber Self-Driving Crash

jefftk7 Nov 2019 15:00 UTC

110 points

1 comment2 min readLW link

(www.jefftk.com)

Reply to Holden on ‘Tool AI’

Eliezer Yudkowsky12 Jun 2012 18:00 UTC

152 points

357 comments17 min readLW link

Stanford Encyclopedia of Philosophy on AI ethics and superintelligence

Kaj_Sotala2 May 2020 7:35 UTC

43 points

19 comments7 min readLW link

(plato.stanford.edu)

AGI Safety Literature Review (Everitt, Lea & Hutter 2018)

Kaj_Sotala4 May 2018 8:56 UTC

14 points

1 comment1 min readLW link

(arxiv.org)

Response to Oren Etzioni’s “How to know if artificial intelligence is about to destroy civilization”

Daniel Kokotajlo27 Feb 2020 18:10 UTC

27 points

5 comments8 min readLW link

Why don’t singularitarians bet on the creation of AGI by buying stocks?

John_Maxwell11 Mar 2020 16:27 UTC

43 points

20 comments4 min readLW link

The problem/solution matrix: Calculating the probability of AI safety “on the back of an envelope”

John_Maxwell20 Oct 2019 8:03 UTC

22 points

4 comments2 min readLW link

Three Stories for How AGI Comes Before FAI

John_Maxwell17 Sep 2019 23:26 UTC

27 points

8 comments6 min readLW link

Brainstorming additional AI risk reduction ideas

John_Maxwell14 Jun 2012 7:55 UTC

19 points

37 comments1 min readLW link

AI Alignment 2018-19 Review

Rohin Shah28 Jan 2020 2:19 UTC

125 points

6 comments35 min readLW link

The Fusion Power Generator Scenario

johnswentworth8 Aug 2020 18:31 UTC

136 points

29 comments3 min readLW link

A guide to Iterated Amplification & Debate

Rafael Harth15 Nov 2020 17:14 UTC

68 points

10 comments15 min readLW link

Work on Security Instead of Friendliness?

Wei_Dai21 Jul 2012 18:28 UTC

51 points

107 comments2 min readLW link

An unaligned benchmark

paulfchristiano17 Nov 2018 15:51 UTC

31 points

0 comments9 min readLW link

Artificial Intelligence: A Modern Approach (4th edition) on the Alignment Problem

Zack_M_Davis17 Sep 2020 2:23 UTC

72 points

12 comments5 min readLW link

(aima.cs.berkeley.edu)

Clarifying “What failure looks like”

Sam Clarke20 Sep 2020 20:40 UTC

95 points

14 comments17 min readLW link

Relaxed adversarial training for inner alignment

evhub10 Sep 2019 23:03 UTC

61 points

28 comments1 min readLW link

An overview of 11 proposals for building safe advanced AI

evhub29 May 2020 20:38 UTC

198 points

36 comments38 min readLW link 2 reviews

Risks from Learned Optimization: Introduction

evhub, Chris van Merwijk, vlad_m, Joar Skalse and Scott Garrabrant

31 May 2019 23:44 UTC

166 points

42 comments12 min readLW link 3 reviews

Risks from Learned Optimization: Conclusion and Related Work

evhub, Chris van Merwijk, vlad_m, Joar Skalse and Scott Garrabrant

7 Jun 2019 19:53 UTC

78 points

4 comments6 min readLW link

Deceptive Alignment

evhub, Chris van Merwijk, vlad_m, Joar Skalse and Scott Garrabrant

5 Jun 2019 20:16 UTC

97 points

11 comments17 min readLW link

The Inner Alignment Problem

evhub, Chris van Merwijk, vlad_m, Joar Skalse and Scott Garrabrant

4 Jun 2019 1:20 UTC

99 points

17 comments13 min readLW link

Conditions for Mesa-Optimization

evhub, Chris van Merwijk, vlad_m, Joar Skalse and Scott Garrabrant

1 Jun 2019 20:52 UTC

75 points

48 comments12 min readLW link

AI risk hub in Singapore?

Daniel Kokotajlo29 Oct 2020 11:45 UTC

57 points

18 comments4 min readLW link

Thoughts on Robin Hanson’s AI Impacts interview

Steven Byrnes24 Nov 2019 1:40 UTC

25 points

3 comments7 min readLW link

The AI Safety Game (UPDATED)

Daniel Kokotajlo5 Dec 2020 10:27 UTC

44 points

9 comments3 min readLW link

[Question] Suggestions of posts on the AF to review

adamShimi16 Feb 2021 12:40 UTC

56 points

20 comments1 min readLW link

Google’s Ethical AI team and AI Safety

magfrump20 Feb 2021 9:42 UTC

12 points

16 comments7 min readLW link

Behavioral Sufficient Statistics for Goal-Directedness

adamShimi11 Mar 2021 15:01 UTC

21 points

12 comments9 min readLW link

Review of “Fun with +12 OOMs of Compute”

adamShimi, Joe_Collman and Gyrodiot

28 Mar 2021 14:55 UTC

63 points

21 comments8 min readLW link 1 review

April drafts

AI Impacts1 Apr 2021 18:10 UTC

49 points

2 comments1 min readLW link

(aiimpacts.org)

25 Min Talk on MetaEthical.AI with Questions from Stuart Armstrong

June Ku29 Apr 2021 15:38 UTC

21 points

7 comments1 min readLW link

Less Realistic Tales of Doom

Mark Xu6 May 2021 23:01 UTC

110 points

13 comments4 min readLW link

Rogue AGI Embodies Valuable Intellectual Property

Mark Xu and CarlShulman

3 Jun 2021 20:37 UTC

70 points

9 comments3 min readLW link

Environmental Structure Can Cause Instrumental Convergence

TurnTrout22 Jun 2021 22:26 UTC

71 points

44 comments16 min readLW link

(arxiv.org)

Alex Turner’s Research, Comprehensive Information Gathering

adamShimi23 Jun 2021 9:44 UTC

15 points

3 comments3 min readLW link

Sam Altman and Ezra Klein on the AI Revolution

Zack_M_Davis27 Jun 2021 4:53 UTC

38 points

17 comments1 min readLW link

(www.nytimes.com)

Approaches to gradient hacking

adamShimi14 Aug 2021 15:16 UTC

16 points

8 comments8 min readLW link

[Question] What are good alignment conference papers?

adamShimi28 Aug 2021 13:35 UTC

12 points

2 comments1 min readLW link

Why the technological singularity by AGI may never happen

hippke3 Sep 2021 14:19 UTC

5 points

14 comments1 min readLW link

[Question] Conditional on the first AGI being aligned correctly, is a good outcome even still likely?

iamthouthouarti6 Sep 2021 17:30 UTC

2 points

1 comment1 min readLW link

Bayeswatch 7: Wildfire

lsusr8 Sep 2021 5:35 UTC

47 points

6 comments3 min readLW link

[Book Review] “The Alignment Problem” by Brian Christian

lsusr20 Sep 2021 6:36 UTC

70 points

16 comments6 min readLW link

Is progress in ML-assisted theorem-proving beneficial?

mako yass28 Sep 2021 1:54 UTC

11 points

3 comments1 min readLW link

Interview with Skynet

lsusr30 Sep 2021 2:20 UTC

49 points

1 comment2 min readLW link

Epistemic Strategies of Selection Theorems

adamShimi18 Oct 2021 8:57 UTC

32 points

1 comment12 min readLW link

Epistemic Strategies of Safety-Capabilities Tradeoffs

adamShimi22 Oct 2021 8:22 UTC

5 points

0 comments6 min readLW link

Drug addicts and deceptively aligned agents—a comparative analysis

Jan5 Nov 2021 21:42 UTC

41 points

2 comments12 min readLW link

(universalprior.substack.com)

Using Brain-Computer Interfaces to get more data for AI alignment

Robbo7 Nov 2021 0:00 UTC

37 points

10 comments7 min readLW link

Using blinders to help you see things for what they are

Adam Zerner11 Nov 2021 7:07 UTC

13 points

2 comments2 min readLW link

My current uncertainties regarding AI, alignment, and the end of the world

dominicq14 Nov 2021 14:08 UTC

2 points

3 comments2 min readLW link

Ngo and Yudkowsky on alignment difficulty

Eliezer Yudkowsky and Richard_Ngo

15 Nov 2021 20:31 UTC

238 points

147 comments99 min readLW link 1 review

Applications for AI Safety Camp 2022 Now Open!

adamShimi17 Nov 2021 21:42 UTC

47 points

3 comments1 min readLW link

[Question] Does the Structure of an algorithm matter for AI Risk and/or consciousness?

Logan Zoellner3 Dec 2021 18:31 UTC

7 points

5 comments1 min readLW link

Framing approaches to alignment and the hard problem of AI cognition

ryan_greenblatt15 Dec 2021 19:06 UTC

16 points

15 comments27 min readLW link

Some abstract, non-technical reasons to be non-maximally-pessimistic about AI alignment

Rob Bensinger12 Dec 2021 2:08 UTC

66 points

37 comments7 min readLW link

Summary of the Acausal Attack Issue for AIXI

Diffractor13 Dec 2021 8:16 UTC

14 points

6 comments4 min readLW link

My Overview of the AI Alignment Landscape: A Bird’s Eye View

Neel Nanda15 Dec 2021 23:44 UTC

117 points

9 comments15 min readLW link

My Overview of the AI Alignment Landscape: Threat Models

Neel Nanda25 Dec 2021 23:07 UTC

50 points

4 comments28 min readLW link

AI Fire Alarm Scenarios

PeterMcCluskey28 Dec 2021 2:20 UTC

10 points

0 comments6 min readLW link

(www.bayesianinvestor.com)

More Is Different for AI

jsteinhardt4 Jan 2022 19:30 UTC

137 points

22 comments3 min readLW link

(bounded-regret.ghost.io)

Challenges with Breaking into MIRI-Style Research

Chris_Leong17 Jan 2022 9:23 UTC

73 points

15 comments3 min readLW link

Thoughts on AGI safety from the top

jylin042 Feb 2022 20:06 UTC

36 points

3 comments32 min readLW link

Paradigm-building from first principles: Effective altruism, AGI, and alignment

Cameron Berg8 Feb 2022 16:12 UTC

24 points

5 comments14 min readLW link

Paradigm-building: Introduction

Cameron Berg8 Feb 2022 0:06 UTC

26 points

0 comments2 min readLW link

How I Formed My Own Views About AI Safety

Neel Nanda27 Feb 2022 18:50 UTC

64 points

6 comments13 min readLW link

(www.neelnanda.io)

[Question] Would (myopic) general public good producers significantly accelerate the development of AGI?

mako yass2 Mar 2022 23:47 UTC

25 points

10 comments1 min readLW link

AXRP Episode 13 - First Principles of AGI Safety with Richard Ngo

DanielFilan31 Mar 2022 5:20 UTC

24 points

1 comment48 min readLW link

[RETRACTED] It’s time for EA leadership to pull the short-timelines fire alarm.

Not Relevant8 Apr 2022 16:07 UTC

112 points

165 comments4 min readLW link

AMA Conjecture, A New Alignment Startup

adamShimi9 Apr 2022 9:43 UTC

46 points

42 comments1 min readLW link

Why I’m Worried About AI

peterbarnett23 May 2022 21:13 UTC

22 points

2 comments12 min readLW link

Complex Systems for AI Safety [Pragmatic AI Safety #3]

Dan H and ThomasW

24 May 2022 0:00 UTC

49 points

2 comments21 min readLW link

Will working here advance AGI? Help us not destroy the world!

Yonatan Cale29 May 2022 11:42 UTC

30 points

46 comments1 min readLW link

Distilled—AGI Safety from First Principles

Harrison G29 May 2022 0:57 UTC

9 points

1 comment14 min readLW link

Perform Tractable Research While Avoiding Capabilities Externalities [Pragmatic AI Safety #4]

Dan H and ThomasW

30 May 2022 20:25 UTC

44 points

3 comments25 min readLW link

Confused why a “capabilities research is good for alignment progress” position isn’t discussed more

Kaj_Sotala2 Jun 2022 21:41 UTC

129 points

27 comments4 min readLW link

I’m trying out “asteroid mindset”

Alex_Altair3 Jun 2022 13:35 UTC

88 points

5 comments4 min readLW link

Epistemological Vigilance for Alignment

adamShimi6 Jun 2022 0:27 UTC

56 points

11 comments10 min readLW link

A Quick Guide to Confronting Doom

Ruby13 Apr 2022 19:30 UTC

224 points

36 comments2 min readLW link

Why I don’t believe in doom

mukashi7 Jun 2022 23:49 UTC

6 points

30 comments4 min readLW link

Open Problems in AI X-Risk [PAIS #5]

Dan H and ThomasW

10 Jun 2022 2:08 UTC

53 points

4 comments36 min readLW link

Another plausible scenario of AI risk: AI builds military infrastructure while collaborating with humans, defects later.

avturchin10 Jun 2022 17:24 UTC

10 points

2 comments1 min readLW link

How dangerous is human-level AI?

Alex_Altair10 Jun 2022 17:38 UTC

21 points

4 comments8 min readLW link

On A List of Lethalities

Zvi13 Jun 2022 12:30 UTC

157 points

48 comments54 min readLW link

(thezvi.wordpress.com)

Continuity Assumptions

Jan_Kulveit13 Jun 2022 21:31 UTC

26 points

13 comments4 min readLW link

Slow motion videos as AI risk intuition pumps

Andrew_Critch14 Jun 2022 19:31 UTC

217 points

36 comments2 min readLW link

Alignment Risk Doesn’t Require Superintelligence

JustisMills15 Jun 2022 3:12 UTC

35 points

4 comments2 min readLW link

[Question] Has there been any work on attempting to use Pascal’s Mugging to make an AGI behave?

Chris_Leong15 Jun 2022 8:33 UTC

7 points

17 comments1 min readLW link

Where I agree and disagree with Eliezer

paulfchristiano19 Jun 2022 19:15 UTC

781 points

205 comments20 min readLW link

Are we there yet?

theflowerpot20 Jun 2022 11:19 UTC

2 points

2 comments1 min readLW link

Confusion about neuroscience/cognitive science as a danger for AI Alignment

Samuel Nellessen22 Jun 2022 17:59 UTC

2 points

1 comment3 min readLW link

(snellessen.com)

The Basics of AGI Policy (Flowchart)

Trevor126 Jun 2022 2:01 UTC

18 points

8 comments2 min readLW link

[Linkpost] Existential Risk Analysis in Empirical Research Papers

Dan H2 Jul 2022 0:09 UTC

40 points

0 comments1 min readLW link

(arxiv.org)

Confusions in My Model of AI Risk

peterbarnett7 Jul 2022 1:05 UTC

22 points

9 comments5 min readLW link

The Alignment Problem

lsusr11 Jul 2022 3:03 UTC

45 points

20 comments3 min readLW link

Robustness to Scaling Down: More Important Than I Thought

adamShimi23 Jul 2022 11:40 UTC

37 points

5 comments3 min readLW link

Comparing Four Approaches to Inner Alignment

Lucas Teixeira29 Jul 2022 21:06 UTC

33 points

1 comment9 min readLW link

Survey: What (de)motivates you about AI risk?

Daniel_Friedrich3 Aug 2022 19:17 UTC

1 point

0 comments1 min readLW link

(forms.gle)

The alignment problem from a deep learning perspective

Richard_Ngo10 Aug 2022 22:46 UTC

93 points

13 comments27 min readLW link

Oversight Misses 100% of Thoughts The AI Does Not Think

johnswentworth12 Aug 2022 16:30 UTC

87 points

49 comments1 min readLW link

DeepMind alignment team opinions on AGI ruin arguments

Vika12 Aug 2022 21:06 UTC

367 points

36 comments14 min readLW link

Shapes of Mind and Pluralism in Alignment

adamShimi13 Aug 2022 10:01 UTC

30 points

1 comment2 min readLW link

Thoughts on ‘List of Lethalities’

Alex Lawsen 17 Aug 2022 18:33 UTC

27 points

0 comments10 min readLW link

What if we approach AI safety like a technical engineering safety problem

zeshen20 Aug 2022 10:29 UTC

30 points

5 comments7 min readLW link

No One-Size-Fit-All Epistemic Strategy

adamShimi20 Aug 2022 12:56 UTC

23 points

1 comment2 min readLW link

Beyond Hyperanthropomorphism

PointlessOne21 Aug 2022 17:55 UTC

3 points

17 comments1 min readLW link

(studio.ribbonfarm.com)

All AGI safety questions welcome (especially basic ones) [Sept 2022]

plex8 Sep 2022 11:56 UTC

22 points

47 comments2 min readLW link

[Question] Why are we sure that AI will “want” something?

shminux16 Sep 2022 20:35 UTC

31 points

58 comments1 min readLW link

Refine’s Third Blog Post Day/Week

adamShimi17 Sep 2022 17:03 UTC

18 points

0 comments1 min readLW link

Announcing AISIC 2022 - the AI Safety Israel Conference, October 19-20

Davidmanheim21 Sep 2022 19:32 UTC

13 points

0 comments1 min readLW link

Results from the language model hackathon

Esben Kran10 Oct 2022 8:29 UTC

21 points

1 comment4 min readLW link

Counterarguments to the basic AI x-risk case

KatjaGrace14 Oct 2022 13:00 UTC

344 points

122 comments34 min readLW link

(aiimpacts.org)

A conversation about Katja’s counterarguments to AI risk

Matthew Barnett, Ege Erdil and Brangus Brangus

18 Oct 2022 18:40 UTC

43 points

9 comments33 min readLW link

Learning societal values from law as part of an AGI alignment strategy

John Nay21 Oct 2022 2:03 UTC

3 points

18 comments54 min readLW link

Empowerment is (almost) All We Need

jacob_cannell23 Oct 2022 21:48 UTC

40 points

43 comments17 min readLW link

All AGI Safety questions welcome (especially basic ones) [~monthly thread]

mwatkins and Robert Miles

26 Jan 2023 21:01 UTC

34 points

48 comments2 min readLW link

Mechanism Design for AI Safety—Reading Group Curriculum

Rubi J. Hudson25 Oct 2022 3:54 UTC

7 points

3 comments1 min readLW link

All AGI Safety questions welcome (especially basic ones) [~monthly thread]

Robert Miles1 Nov 2022 23:23 UTC

67 points

102 comments2 min readLW link

AI Safety Microgrant Round

Chris_Leong14 Nov 2022 4:25 UTC

22 points

1 comment1 min readLW link

Poster Session on AI Safety

Neil Crawford12 Nov 2022 3:50 UTC

7 points

6 comments1 min readLW link

What I Learned Running Refine

adamShimi24 Nov 2022 14:49 UTC

103 points

5 comments4 min readLW link

Podcast: Shoshannah Tekofsky on skilling up in AI safety, visiting Berkeley, and developing novel research ideas

Akash25 Nov 2022 20:47 UTC

37 points

2 comments9 min readLW link

AI Safety Seems Hard to Measure

HoldenKarnofsky8 Dec 2022 19:50 UTC

69 points

5 comments14 min readLW link

(www.cold-takes.com)

AI Neorealism: a threat model & success criterion for existential safety

davidad15 Dec 2022 13:42 UTC

41 points

0 comments3 min readLW link

Podcast: Tamera Lanham on AI risk, threat models, alignment proposals, externalized reasoning oversight, and working at Anthropic

Akash20 Dec 2022 21:39 UTC

18 points

2 comments11 min readLW link

Racing through a minefield: the AI deployment problem

HoldenKarnofsky22 Dec 2022 16:10 UTC

33 points

1 comment13 min readLW link

(www.cold-takes.com)

My thoughts on OpenAI’s alignment plan

Akash30 Dec 2022 19:33 UTC

46 points

0 comments20 min readLW link

Wentworth and Larsen on buying time

Akash, Thomas Larsen and johnswentworth

9 Jan 2023 21:31 UTC

67 points

6 comments12 min readLW link

World-Model Interpretability Is All We Need

Thane Ruthenis14 Jan 2023 19:37 UTC

29 points

17 comments21 min readLW link

ea.domains—Domains Free to a Good Home

plex12 Jan 2023 13:32 UTC

21 points

0 comments1 min readLW link

How we could stumble into AI catastrophe

HoldenKarnofsky13 Jan 2023 16:20 UTC

63 points

6 comments18 min readLW link

(www.cold-takes.com)

Thoughts on refusing harmful requests to large language models

William_S19 Jan 2023 19:49 UTC

30 points

4 comments2 min readLW link

[Linkpost] TIME article: DeepMind’s CEO Helped Take AI Mainstream. Now He’s Urging Caution

Akash21 Jan 2023 16:51 UTC

55 points

2 comments3 min readLW link

(time.com)

Talk to me about your summer/career plans

Akash31 Jan 2023 18:29 UTC

24 points

0 comments2 min readLW link

Access to AI: a human right?

dmtea25 Jul 2020 9:38 UTC

5 points

3 comments2 min readLW link

Agentic Language Model Memes

FactorialCode1 Aug 2020 18:03 UTC

16 points

1 comment2 min readLW link

Conversation with Paul Christiano

abergal11 Sep 2019 23:20 UTC

44 points

6 comments30 min readLW link

(aiimpacts.org)

Transcription of Eliezer’s January 2010 video Q&A

curiousepic14 Nov 2011 17:02 UTC

112 points

9 comments56 min readLW link

Responses to Catastrophic AGI Risk: A Survey

lukeprog8 Jul 2013 14:33 UTC

17 points

8 comments1 min readLW link

How can I reduce existential risk from AI?

lukeprog13 Nov 2012 21:56 UTC

63 points

92 comments8 min readLW link

Thoughts on Ben Garfinkel’s “How sure are we about this AI stuff?”

David Scott Krueger (formerly: capybaralet)6 Feb 2019 19:09 UTC

25 points

17 comments1 min readLW link

Reframing misaligned AGI’s: well-intentioned non-neurotypical assistants

zhukeepa1 Apr 2018 1:22 UTC

46 points

14 comments2 min readLW link

When is unaligned AI morally valuable?

paulfchristiano25 May 2018 1:57 UTC

63 points

52 comments10 min readLW link

Introducing the AI Alignment Forum (FAQ)

habryka, Ben Pace, Raemon and jimrandomh

29 Oct 2018 21:07 UTC

86 points

8 comments6 min readLW link

Swimming Upstream: A Case Study in Instrumental Rationality

TurnTrout3 Jun 2018 3:16 UTC

74 points

7 comments8 min readLW link

Current AI Safety Roles for Software Engineers

ozziegooen9 Nov 2018 20:57 UTC

70 points

9 comments4 min readLW link

[Question] Why is so much discussion happening in private Google Docs?

Wei_Dai12 Jan 2019 2:19 UTC

100 points

22 comments1 min readLW link

Problems in AI Alignment that philosophers could potentially contribute to

Wei_Dai17 Aug 2019 17:38 UTC

76 points

14 comments2 min readLW link

Two Neglected Problems in Human-AI Safety

Wei_Dai16 Dec 2018 22:13 UTC

86 points

24 comments2 min readLW link

Announcement: AI alignment prize round 4 winners

cousin_it20 Jan 2019 14:46 UTC

74 points

41 comments1 min readLW link

Soon: a weekly AI Safety prerequisites module on LessWrong

null30 Apr 2018 13:23 UTC

35 points

10 comments1 min readLW link

And the AI would have got away with it too, if...

Stuart_Armstrong22 May 2019 21:35 UTC

75 points

7 comments1 min readLW link

2017 AI Safety Literature Review and Charity Comparison

Larks24 Dec 2017 18:52 UTC

41 points

5 comments23 min readLW link

Should ethicists be inside or outside a profession?

Eliezer Yudkowsky12 Dec 2018 1:40 UTC

87 points

6 comments9 min readLW link

I Vouch For MIRI

Zvi17 Dec 2017 17:50 UTC

36 points

9 comments5 min readLW link

(thezvi.wordpress.com)

Beware of black boxes in AI alignment research

cousin_it18 Jan 2018 15:07 UTC

39 points

10 comments1 min readLW link

AI Alignment Prize: Round 2 due March 31, 2018

Zvi12 Mar 2018 12:10 UTC

28 points

2 comments3 min readLW link

(thezvi.wordpress.com)

Three AI Safety Related Ideas

Wei_Dai13 Dec 2018 21:32 UTC

68 points

38 comments2 min readLW link

A rant against robots

Lê Nguyên Hoang14 Jan 2020 22:03 UTC

64 points

7 comments5 min readLW link

Opportunities for individual donors in AI safety

Alex Flint31 Mar 2018 18:37 UTC

30 points

3 comments11 min readLW link

But exactly how complex and fragile?

KatjaGrace3 Nov 2019 18:20 UTC

73 points

32 comments3 min readLW link 1 review

(meteuphoric.com)

Course recommendations for Friendliness researchers

Louie9 Jan 2013 14:33 UTC

96 points

112 comments10 min readLW link

AI Safety Research Camp—Project Proposal

David_Kristoffersson2 Feb 2018 4:25 UTC

29 points

11 comments8 min readLW link

AI Summer Fellows Program

colm21 Mar 2018 15:32 UTC

21 points

0 comments1 min readLW link

The genie knows, but doesn’t care

Rob Bensinger6 Sep 2013 6:42 UTC

93 points

519 comments8 min readLW link

Alignment Newsletter #13: 07/02/18

Rohin Shah2 Jul 2018 16:10 UTC

70 points

12 comments8 min readLW link

(mailchi.mp)

An Increasingly Manipulative Newsfeed

Michaël Trazzi1 Jul 2019 15:26 UTC

62 points

16 comments5 min readLW link

The simple picture on AI safety

Alex Flint27 May 2018 19:43 UTC

31 points

10 comments2 min readLW link

Elon Musk donates $10M to the Future of Life Institute to keep AI beneficial

Paul Crowley15 Jan 2015 16:33 UTC

78 points

52 comments1 min readLW link

Strategic implications of AIs’ ability to coordinate at low cost, for example by merging

Wei_Dai25 Apr 2019 5:08 UTC

66 points

45 comments2 min readLW link 1 review

Modeling AGI Safety Frameworks with Causal Influence Diagrams

Ramana Kumar21 Jun 2019 12:50 UTC

43 points

6 comments1 min readLW link

(arxiv.org)

Henry Kissinger: AI Could Mean the End of Human History

ESRogs15 May 2018 20:11 UTC

17 points

12 comments1 min readLW link

(www.theatlantic.com)

Toy model of the AI control problem: animated version

Stuart_Armstrong10 Oct 2017 11:06 UTC

25 points

8 comments1 min readLW link

A Visualization of Nick Bostrom’s Superintelligence

[deleted]23 Jul 2014 0:24 UTC

62 points

28 comments3 min readLW link

AI Alignment Research Overview (by Jacob Steinhardt)

Ben Pace6 Nov 2019 19:24 UTC

43 points

0 comments7 min readLW link

(docs.google.com)

A general model of safety-oriented AI development

Wei_Dai11 Jun 2018 21:00 UTC

65 points

8 comments1 min readLW link

Counterfactual Oracles = online supervised learning with random selection of training episodes

Wei_Dai10 Sep 2019 8:29 UTC

47 points

26 comments3 min readLW link

Siren worlds and the perils of over-optimised search

Stuart_Armstrong7 Apr 2014 11:00 UTC

73 points

417 comments7 min readLW link

Top 9+2 myths about AI risk

Stuart_Armstrong29 Jun 2015 20:41 UTC

68 points

45 comments2 min readLW link

Rohin Shah on reasons for AI optimism

abergal31 Oct 2019 12:10 UTC

40 points

58 comments1 min readLW link

(aiimpacts.org)

Plausibly, almost every powerful algorithm would be manipulative

Stuart_Armstrong6 Feb 2020 11:50 UTC

38 points

25 comments3 min readLW link

The Magnitude of His Own Folly

Eliezer Yudkowsky30 Sep 2008 11:31 UTC

88 points

128 comments6 min readLW link

AI alignment landscape

paulfchristiano13 Oct 2019 2:10 UTC

40 points

3 comments1 min readLW link

(ai-alignment.com)

Launched: Friendship is Optimal

iceman15 Nov 2012 4:57 UTC

73 points

32 comments1 min readLW link

Friendship is Optimal: A My Little Pony fanfic about an optimization process

iceman8 Sep 2012 6:16 UTC

111 points

152 comments1 min readLW link

Do Earths with slower economic growth have a better chance at FAI?

Eliezer Yudkowsky12 Jun 2013 19:54 UTC

58 points

176 comments4 min readLW link

Idea: Open Access AI Safety Journal

Gordon Seidoh Worley23 Mar 2018 18:27 UTC

28 points

11 comments1 min readLW link

G.K. Chesterton On AI Risk

Scott Alexander1 Apr 2017 19:00 UTC

17 points

0 comments7 min readLW link

The Hidden Complexity of Wishes

Eliezer Yudkowsky24 Nov 2007 0:12 UTC

145 points

137 comments7 min readLW link

The Friendly AI Game

bentarm15 Mar 2011 16:45 UTC

50 points

178 comments1 min readLW link

Q&A with Jürgen Schmidhuber on risks from AI

XiXiDu15 Jun 2011 15:51 UTC

59 points

45 comments4 min readLW link

[Question] What should an Einstein-like figure in Machine Learning do?

Razied5 Aug 2020 23:52 UTC

3 points

3 comments1 min readLW link

Takeaways from safety by default interviews

AI Impacts and abergal

3 Apr 2020 17:20 UTC

28 points

2 comments13 min readLW link

(aiimpacts.org)

Field-Building and Deep Models

Ben Pace13 Jan 2018 21:16 UTC

21 points

12 comments4 min readLW link

Critique my Model: The EV of AGI to Selfish Individuals

ozziegooen8 Apr 2018 20:04 UTC

19 points

9 comments4 min readLW link

‘Dumb’ AI observes and manipulates controllers

Stuart_Armstrong13 Jan 2015 13:35 UTC

52 points

19 comments2 min readLW link

2019 AI Alignment Literature Review and Charity Comparison

Larks19 Dec 2019 3:00 UTC

130 points

18 comments62 min readLW link

Book review: Architects of Intelligence by Martin Ford (2018)

Ofer11 Aug 2020 17:30 UTC

15 points

0 comments2 min readLW link

Qualitative Strategies of Friendliness

Eliezer Yudkowsky30 Aug 2008 2:12 UTC

30 points

56 comments12 min readLW link

Dreams of Friendliness

Eliezer Yudkowsky31 Aug 2008 1:20 UTC

25 points

81 comments9 min readLW link

Conceptual issues in AI safety: the paradigmatic gap

vedevazz24 Jun 2018 15:09 UTC

33 points

0 comments1 min readLW link

(www.foldl.me)

On unfixably unsafe AGI architectures

Steven Byrnes19 Feb 2020 21:16 UTC

33 points

8 comments5 min readLW link

A toy model of the treacherous turn

Stuart_Armstrong8 Jan 2016 12:58 UTC

36 points

13 comments6 min readLW link

Allegory On AI Risk, Game Theory, and Mithril

James_Miller13 Feb 2017 20:41 UTC

45 points

57 comments3 min readLW link

1hr talk: Intro to AGI safety

Steven Byrnes18 Jun 2019 21:41 UTC

35 points

4 comments24 min readLW link

The Evil AI Overlord List

Stuart_Armstrong20 Nov 2012 17:02 UTC

44 points

80 comments1 min readLW link

What I would like the SIAI to publish

XiXiDu1 Nov 2010 14:07 UTC

36 points

225 comments3 min readLW link

Evaluating the feasibility of SI’s plan

JoshuaFox10 Jan 2013 8:17 UTC

38 points

188 comments4 min readLW link

Q&A with experts on risks from AI #1

XiXiDu8 Jan 2012 11:46 UTC

45 points

67 comments9 min readLW link

Algo trading is a central example of AI risk

Vanessa Kosoy28 Jul 2018 20:31 UTC

27 points

5 comments1 min readLW link

Will the world’s elites navigate the creation of AI just fine?

lukeprog31 May 2013 18:49 UTC

36 points

266 comments2 min readLW link

Let’s talk about “Convergent Rationality”

David Scott Krueger (formerly: capybaralet)12 Jun 2019 21:53 UTC

36 points

33 comments6 min readLW link

Breaking Oracles: superrationality and acausal trade

Stuart_Armstrong25 Nov 2019 10:40 UTC

25 points

15 comments1 min readLW link

Q&A with Stan Franklin on risks from AI

XiXiDu11 Jun 2011 15:22 UTC

36 points

10 comments2 min readLW link

Muehlhauser-Goertzel Dialogue, Part 1

lukeprog16 Mar 2012 17:12 UTC

42 points

161 comments33 min readLW link

[LINK] NYT Article about Existential Risk from AI

[deleted]28 Jan 2013 10:37 UTC

38 points

23 comments1 min readLW link

Reframing the Problem of AI Progress

Wei_Dai12 Apr 2012 19:31 UTC

32 points

47 comments1 min readLW link

New AI risks research institute at Oxford University

lukeprog16 Nov 2011 18:52 UTC

36 points

10 comments1 min readLW link

Thoughts on the Feasibility of Prosaic AGI Alignment?

iamthouthouarti21 Aug 2020 23:25 UTC

8 points

10 comments1 min readLW link

Memes and Rational Decisions

inferential9 Jan 2015 6:42 UTC

35 points

18 comments10 min readLW link

Levels of AI Self-Improvement

avturchin29 Apr 2018 11:45 UTC

11 points

0 comments39 min readLW link

Optimising Society to Constrain Risk of War from an Artificial Superintelligence

JohnCDraper30 Apr 2020 10:47 UTC

3 points

1 comment51 min readLW link

Some Thoughts on Singularity Strategies

Wei_Dai13 Jul 2011 2:41 UTC

41 points

29 comments3 min readLW link

A trick for Safer GPT-N

Razied23 Aug 2020 0:39 UTC

7 points

1 comment2 min readLW link

against “AI risk”

Wei_Dai11 Apr 2012 22:46 UTC

35 points

91 comments1 min readLW link

“Smarter than us” is out!

Stuart_Armstrong25 Feb 2014 15:50 UTC

41 points

57 comments1 min readLW link

Analysing: Dangerous messages from future UFAI via Oracles

Stuart_Armstrong22 Nov 2019 14:17 UTC

22 points

16 comments4 min readLW link

Q&A with Abram Demski on risks from AI

XiXiDu17 Jan 2012 9:43 UTC

33 points

71 comments9 min readLW link

Q&A with experts on risks from AI #2

XiXiDu9 Jan 2012 19:40 UTC

22 points

29 comments7 min readLW link

AI Safety Discussion Day

Linda Linsefors15 Sep 2020 14:40 UTC

20 points

0 comments1 min readLW link

A long reply to Ben Garfinkel on Scrutinizing Classic AI Risk Arguments

Søren Elverlin27 Sep 2020 17:51 UTC

17 points

6 comments1 min readLW link

Online AI Safety Discussion Day

Linda Linsefors8 Oct 2020 12:11 UTC

5 points

0 comments1 min readLW link

Military AI as a Convergent Goal of Self-Improving AI

avturchin13 Nov 2017 12:17 UTC

5 points

3 comments1 min readLW link

Neural program synthesis is a dangerous technology

syllogism12 Jan 2018 16:19 UTC

10 points

6 comments2 min readLW link

New, Brief Popular-Level Introduction to AI Risks and Superintelligence

LyleN23 Jan 2015 15:43 UTC

33 points

3 comments1 min readLW link

FAI Research Constraints and AGI Side Effects

JustinShovelain3 Jun 2015 19:25 UTC

26 points

59 comments7 min readLW link

European Master’s Programs in Machine Learning, Artificial Intelligence, and related fields

Master Programs ML/AI14 Nov 2020 15:51 UTC

33 points

8 comments1 min readLW link

The mind-killer

Paul Crowley2 May 2009 16:49 UTC

29 points

160 comments2 min readLW link

[Question] Should I do it?

MrLight19 Nov 2020 1:08 UTC

−3 points

16 comments2 min readLW link

Rationalising humans: another mugging, but not Pascal’s

Stuart_Armstrong14 Nov 2017 15:46 UTC

7 points

1 comment3 min readLW link

Machine learning could be fundamentally unexplainable

George3d616 Dec 2020 13:32 UTC

26 points

15 comments15 min readLW link

(cerebralab.com)

[Question] What do you make of AGI:unaligned::spaceships:not enough food?

Ronny Fernandez22 Feb 2020 14:14 UTC

4 points

3 comments1 min readLW link

Risk Map of AI Systems

VojtaKovarik and Jan_Kulveit

15 Dec 2020 9:16 UTC

27 points

3 comments8 min readLW link

[Question] Does it become easier, or harder, for the world to coordinate around not building AGI as time goes on?

Eli Tyre29 Jul 2019 22:59 UTC

86 points

31 comments3 min readLW link 2 reviews

Grey Goo Requires AI

harsimony15 Jan 2021 4:45 UTC

8 points

11 comments4 min readLW link

(harsimony.wordpress.com)

AISU 2021

Linda Linsefors30 Jan 2021 17:40 UTC

28 points

2 comments1 min readLW link

Nonperson Predicates

Eliezer Yudkowsky27 Dec 2008 1:47 UTC

53 points

176 comments6 min readLW link

Engaging First Introductions to AI Risk

Rob Bensinger19 Aug 2013 6:26 UTC

31 points

21 comments3 min readLW link

Formal Solution to the Inner Alignment Problem

michaelcohen18 Feb 2021 14:51 UTC

47 points

123 comments2 min readLW link

[Question] What are the biggest current impacts of AI?

Sam Clarke7 Mar 2021 21:44 UTC

15 points

5 comments1 min readLW link

[Question] Is a Self-Iterating AGI Vulnerable to Thompson-style Trojans?

sxae25 Mar 2021 14:46 UTC

15 points

7 comments3 min readLW link

AI oracles on blockchain

Caravaggio6 Apr 2021 20:13 UTC

5 points

0 comments3 min readLW link

What if AGI is near?

Wulky Wilkinsen14 Apr 2021 0:05 UTC

11 points

5 comments1 min readLW link

[Question] Is there anything that can stop AGI development in the near term?

Wulky Wilkinsen22 Apr 2021 20:37 UTC

5 points

5 comments1 min readLW link

[Question] [timeboxed exercise] write me your model of AI human-existential safety and the alignment problems in 15 minutes

Quinn4 May 2021 19:10 UTC

6 points

2 comments1 min readLW link

AI Safety Research Project Ideas

Owain_Evans21 May 2021 13:39 UTC

58 points

2 comments3 min readLW link

Survey on AI existential risk scenarios

Sam Clarke, Alexis Carlier and Jonas Schuett

8 Jun 2021 17:12 UTC

60 points

11 comments7 min readLW link

[Question] What are some claims or opinions about multi-multi delegation you’ve seen in the memeplex that you think deserve scrutiny?

Quinn27 Jun 2021 17:44 UTC

17 points

6 comments2 min readLW link

Mauhn Releases AI Safety Documentation

Berg Severens3 Jul 2021 21:23 UTC

4 points

0 comments1 min readLW link

A gentle apocalypse

pchvykov16 Aug 2021 5:03 UTC

3 points

5 comments3 min readLW link

Could you have stopped Chernobyl?

Carlos Ramirez27 Aug 2021 1:48 UTC

29 points

17 comments8 min readLW link

The Governance Problem and the “Pretty Good” X-Risk

Zach Stein-Perlman29 Aug 2021 18:00 UTC

5 points

2 comments11 min readLW link

Distinguishing AI takeover scenarios

Sam Clarke and Sammy Martin

8 Sep 2021 16:19 UTC

67 points

11 comments14 min readLW link

The alignment problem in different capability regimes

Buck9 Sep 2021 19:46 UTC

87 points

12 comments5 min readLW link

How truthful is GPT-3? A benchmark for language models

Owain_Evans16 Sep 2021 10:09 UTC

56 points

24 comments6 min readLW link

Investigating AI Takeover Scenarios

Sammy Martin17 Sep 2021 18:47 UTC

27 points

1 comment27 min readLW link

AI takeoff story: a continuation of progress by other means

Edouard Harris27 Sep 2021 15:55 UTC

75 points

13 comments10 min readLW link

A brief review of the reasons multi-objective RL could be important in AI Safety Research

Ben Smith29 Sep 2021 17:09 UTC

29 points

8 comments10 min readLW link

The Dark Side of Cognition Hypothesis

Cameron Berg3 Oct 2021 20:10 UTC

19 points

1 comment16 min readLW link

Truthful AI: Developing and governing AI that does not lie

Owain_Evans, owencb and Lanrian

18 Oct 2021 18:37 UTC

81 points

9 comments10 min readLW link

AMA on Truthful AI: Owen Cotton-Barratt, Owain Evans & co-authors

Owain_Evans22 Oct 2021 16:23 UTC

31 points

15 comments1 min readLW link

Truthful and honest AI

abergal, Nick_Beckstead and Owain_Evans

29 Oct 2021 7:28 UTC

41 points

1 comment13 min readLW link

What is the most evil AI that we could build, today?

ThomasJ1 Nov 2021 19:58 UTC

−2 points

14 comments1 min readLW link

What are red flags for Neural Network suffering?

Marius Hobbhahn8 Nov 2021 12:51 UTC

26 points

15 comments12 min readLW link

Hardcode the AGI to need our approval indefinitely?

MichaelStJules11 Nov 2021 7:04 UTC

2 points

2 comments1 min readLW link

What would we do if alignment were futile?

Grant Demaree14 Nov 2021 8:09 UTC

73 points

43 comments3 min readLW link

Two Stupid AI Alignment Ideas

aphyer16 Nov 2021 16:13 UTC

24 points

3 comments4 min readLW link

Super intelligent AIs that don’t require alignment

Yair Halberstadt16 Nov 2021 19:55 UTC

10 points

2 comments6 min readLW link

AI Tracker: monitoring current and near-future risks from superscale models

Edouard Harris and Jeremie Harris

23 Nov 2021 19:16 UTC

64 points

13 comments3 min readLW link

(aitracker.org)

HIRING: Inform and shape a new project on AI safety at Partnership on AI

Madhulika Srikumar24 Nov 2021 8:27 UTC

6 points

0 comments1 min readLW link

How to measure FLOP/s for Neural Networks empirically?

Marius Hobbhahn29 Nov 2021 15:18 UTC

16 points

5 comments7 min readLW link

Modeling Failure Modes of High-Level Machine Intelligence

Ben Cottier, Daniel_Eth and Sammy Martin

6 Dec 2021 13:54 UTC

54 points

1 comment12 min readLW link

HIRING: Inform and shape a new project on AI safety at Partnership on AI

madhu_lika7 Dec 2021 19:37 UTC

1 point

0 comments1 min readLW link

Universality and the “Filter”

maggiehayes16 Dec 2021 0:47 UTC

10 points

3 comments11 min readLW link

Reviews of “Is power-seeking AI an existential risk?”

Joe Carlsmith16 Dec 2021 20:48 UTC

76 points

20 comments1 min readLW link

2+2: Ontological Framework

Lyrialtus1 Feb 2022 1:07 UTC

−15 points

2 comments12 min readLW link

Can the laws of physics/nature prevent hell?

superads916 Feb 2022 20:39 UTC

−7 points

10 comments2 min readLW link

How harmful are improvements in AI? + Poll

tilker and Marius Hobbhahn

15 Feb 2022 18:16 UTC

15 points

4 comments8 min readLW link

Preserving and continuing alignment research through a severe global catastrophe

A_donor6 Mar 2022 18:43 UTC

36 points

11 comments5 min readLW link

Ask AI companies about what they are doing for AI safety?

Michael Chen9 Mar 2022 15:14 UTC

50 points

0 comments2 min readLW link

Is There a Valley of Bad Civilizational Adequacy?

lbThingrb11 Mar 2022 19:49 UTC

13 points

1 comment2 min readLW link

[Question] Danger(s) of theorem-proving AI?

Yitz16 Mar 2022 2:47 UTC

8 points

9 comments1 min readLW link

It Looks Like You’re Trying To Take Over The World

gwern9 Mar 2022 16:35 UTC

389 points

125 comments1 min readLW link

(www.gwern.net)

We Are Conjecture, A New Alignment Research Startup

Connor Leahy8 Apr 2022 11:40 UTC

186 points

24 comments4 min readLW link

Is technical AI alignment research a net positive?

cranberry_bear12 Apr 2022 13:07 UTC

4 points

2 comments2 min readLW link

The Peerless

carado13 Apr 2022 1:07 UTC

18 points

2 comments1 min readLW link

(carado.moe)

[Question] Can someone explain to me why MIRI is so pessimistic of our chances of survival?

iamthouthouarti14 Apr 2022 20:28 UTC

10 points

7 comments1 min readLW link

[Question] Convince me that humanity isn’t doomed by AGI

Yitz15 Apr 2022 17:26 UTC

60 points

53 comments1 min readLW link

Reflections on My Own Missing Mood

Lone Pine21 Apr 2022 16:19 UTC

51 points

25 comments5 min readLW link

Code Generation as an AI risk setting

Not Relevant17 Apr 2022 22:27 UTC

91 points

16 comments2 min readLW link

[Question] What is being improved in recursive self improvement?

Lone Pine25 Apr 2022 18:30 UTC

7 points

7 comments1 min readLW link

AI Alternative Futures: Scenario Mapping Artificial Intelligence Risk—Request for Participation (Closed)

Kakili27 Apr 2022 22:07 UTC

10 points

2 comments8 min readLW link

Video and Transcript of Presentation on Existential Risk from Power-Seeking AI

Joe Carlsmith8 May 2022 3:50 UTC

20 points

1 comment29 min readLW link

Interpretability’s Alignment-Solving Potential: Analysis of 7 Scenarios

Evan R. Murphy12 May 2022 20:01 UTC

47 points

0 comments59 min readLW link

Agency As a Natural Abstraction

Thane Ruthenis13 May 2022 18:02 UTC

55 points

9 comments13 min readLW link

[Link post] Promising Paths to Alignment—Connor Leahy | Talk

frances_lorenz14 May 2022 16:01 UTC

34 points

0 comments1 min readLW link

DeepMind’s generalist AI, Gato: A non-technical explainer

frances_lorenz, Nora Belrose and jonmenaster

16 May 2022 21:21 UTC

57 points

6 comments6 min readLW link

Actionable-guidance and roadmap recommendations for the NIST AI Risk Management Framework

Dan H and Tony Barrett

17 May 2022 15:26 UTC

25 points

0 comments3 min readLW link

In defence of flailing

acylhalide18 Jun 2022 5:26 UTC

10 points

14 comments4 min readLW link

Why I’m Optimistic About Near-Term AI Risk

harsimony15 May 2022 23:05 UTC

57 points

28 comments1 min readLW link

Pivotal acts using an unaligned AGI?

Simon Fischer21 Aug 2022 17:13 UTC

25 points

3 comments8 min readLW link

Reshaping the AI Industry

Thane Ruthenis29 May 2022 22:54 UTC

143 points

34 comments21 min readLW link

Explaining inner alignment to myself

Jeremy Gillen24 May 2022 23:10 UTC

9 points

2 comments10 min readLW link

Right now, you’re sitting on a REDONKULOUS opportunity to help solve AGI (and rake in $$$)

Trevor126 May 2022 21:55 UTC

43 points

13 comments2 min readLW link

A Story of AI Risk: InstructGPT-N

peterbarnett26 May 2022 23:22 UTC

24 points

0 comments8 min readLW link

We will be around in 30 years

mukashi7 Jun 2022 3:47 UTC

12 points

205 comments2 min readLW link

Transformer Research Questions from Stained Glass Windows

StefanHex8 Jun 2022 12:38 UTC

4 points

0 comments2 min readLW link

Towards Gears-Level Understanding of Agency

Thane Ruthenis16 Jun 2022 22:00 UTC

23 points

4 comments18 min readLW link

A plausible story about AI risk.

DeLesley Hutchins10 Jun 2022 2:08 UTC

14 points

1 comment4 min readLW link

Summary of “AGI Ruin: A List of Lethalities”

Stephen McAleese10 Jun 2022 22:35 UTC

34 points

2 comments8 min readLW link

Poorly-Aimed Death Rays

Thane Ruthenis11 Jun 2022 18:29 UTC

43 points

5 comments4 min readLW link

Contra EY: Can AGI destroy us without trial & error?

Nikita Sokolsky13 Jun 2022 18:26 UTC

124 points

76 comments15 min readLW link

A Modest Pivotal Act

anonymousaisafety13 Jun 2022 19:24 UTC

−15 points

1 comment5 min readLW link

Alignment research for “meta” purposes

acylhalide16 Jun 2022 14:03 UTC

15 points

0 comments1 min readLW link

Specific problems with specific animal comparisons for AI policy

Trevor119 Jun 2022 1:27 UTC

3 points

1 comment2 min readLW link

[Question] AI misalignment risk from GPT-like systems?

fiso6419 Jun 2022 17:35 UTC

10 points

8 comments1 min readLW link

Causal confusion as an argument against the scaling hypothesis

RobertKirk and David Scott Krueger (formerly: capybaralet)

20 Jun 2022 10:54 UTC

83 points

30 comments18 min readLW link

[LQ] Some Thoughts on Messaging Around AI Risk

DragonGod25 Jun 2022 13:53 UTC

5 points

3 comments6 min readLW link

All AGI safety questions welcome (especially basic ones) [July 2022]

plex and Robert Miles

16 Jul 2022 12:57 UTC

84 points

132 comments3 min readLW link

Reframing the AI Risk

Thane Ruthenis1 Jul 2022 18:44 UTC

26 points

7 comments6 min readLW link

Follow along with Columbia EA’s Advanced AI Safety Fellowship!

RohanS2 Jul 2022 17:45 UTC

3 points

0 comments2 min readLW link

(forum.effectivealtruism.org)

Can we achieve AGI Alignment by balancing multiple human objectives?

Ben Smith3 Jul 2022 2:51 UTC

11 points

1 comment4 min readLW link

New US Senate Bill on X-Risk Mitigation [Linkpost]

Evan R. Murphy4 Jul 2022 1:25 UTC

35 points

12 comments1 min readLW link

(www.hsgac.senate.gov)

My Most Likely Reason to Die Young is AI X-Risk

AISafetyIsNotLongtermist4 Jul 2022 17:08 UTC

61 points

24 comments4 min readLW link

(forum.effectivealtruism.org)

Please help us communicate AI xrisk. It could save the world.

otto.barten4 Jul 2022 21:47 UTC

4 points

7 comments2 min readLW link

When is it appropriate to use statistical models and probabilities for decision making ?

Younes Kamel5 Jul 2022 12:34 UTC

10 points

7 comments4 min readLW link

(youneskamel.substack.com)

Acceptability Verification: A Research Agenda

David Udell and evhub

12 Jul 2022 20:11 UTC

45 points

0 comments1 min readLW link

(docs.google.com)

Goal Alignment Is Robust To the Sharp Left Turn

Thane Ruthenis13 Jul 2022 20:23 UTC

46 points

15 comments4 min readLW link

Conditioning Generative Models for Alignment

Jozdien18 Jul 2022 7:11 UTC

47 points

8 comments20 min readLW link

A Critique of AI Alignment Pessimism

ExCeph19 Jul 2022 2:28 UTC

8 points

1 comment9 min readLW link

What Environment Properties Select Agents For World-Modeling?

Thane Ruthenis23 Jul 2022 19:27 UTC

24 points

1 comment12 min readLW link

Alignment being impossible might be better than it being really difficult

Martín Soto25 Jul 2022 23:57 UTC

12 points

2 comments2 min readLW link

AGI ruin scenarios are likely (and disjunctive)

So8res27 Jul 2022 3:21 UTC

151 points

37 comments6 min readLW link

[Question] How likely do you think worse-than-extinction type fates to be?

span11 Aug 2022 4:08 UTC

3 points

3 comments1 min readLW link

Three pillars for avoiding AGI catastrophe: Technical alignment, deployment decisions, and coordination

Alex Lintz3 Aug 2022 23:15 UTC

17 points

0 comments12 min readLW link

Convergence Towards World-Models: A Gears-Level Model

Thane Ruthenis4 Aug 2022 23:31 UTC

37 points

1 comment13 min readLW link

Complexity No Bar to AI (Or, why Computational Complexity matters less than you think for real life problems)

Noosphere897 Aug 2022 19:55 UTC

17 points

14 comments3 min readLW link

(www.gwern.net)

How To Go From Interpretability To Alignment: Just Retarget The Search

johnswentworth10 Aug 2022 16:08 UTC

146 points

32 comments3 min readLW link

Anti-squatted AI x-risk domains index

plex12 Aug 2022 12:01 UTC

50 points

3 comments1 min readLW link

Infant AI Scenario

Nathan112312 Aug 2022 21:20 UTC

1 point

0 comments3 min readLW link

The Dumbest Possible Gets There First

Artaxerxes13 Aug 2022 10:20 UTC

35 points

7 comments2 min readLW link

Interpretability Tools Are an Attack Channel

Thane Ruthenis17 Aug 2022 18:47 UTC

42 points

22 comments1 min readLW link

Alignment’s phlogiston

Eleni Angelou18 Aug 2022 22:27 UTC

10 points

2 comments2 min readLW link

Benchmarking Proposals on Risk Scenarios

Paul Bricman20 Aug 2022 10:01 UTC

25 points

2 comments14 min readLW link

What’s the Least Impressive Thing GPT-4 Won’t be Able to Do

Algon20 Aug 2022 19:48 UTC

76 points

85 comments1 min readLW link

My Plan to Build Aligned Superintelligence

apollonianblues21 Aug 2022 13:16 UTC

18 points

7 comments8 min readLW link

The Alignment Problem Needs More Positive Fiction

Netcentrica21 Aug 2022 22:01 UTC

5 points

2 comments5 min readLW link

It Looks Like You’re Trying To Take Over The Narrative

George3d624 Aug 2022 13:36 UTC

2 points

20 comments9 min readLW link

(www.epistem.ink)

AI Risk in Terms of Unstable Nuclear Software

Thane Ruthenis26 Aug 2022 18:49 UTC

29 points

1 comment6 min readLW link

Taking the parameters which seem to matter and rotating them until they don’t

Garrett Baker26 Aug 2022 18:26 UTC

117 points

48 comments1 min readLW link

Annual AGI Benchmarking Event

Lawrence Phillips27 Aug 2022 0:06 UTC

24 points

3 comments2 min readLW link

(www.metaculus.com)

Help Understanding Preferences And Evil

Netcentrica27 Aug 2022 3:42 UTC

6 points

7 comments2 min readLW link

[Question] What would you expect a massive multimodal online federated learner to be capable of?

Aryeh Englander27 Aug 2022 17:31 UTC

13 points

4 comments1 min readLW link

Are Generative World Models a Mesa-Optimization Risk?

Thane Ruthenis29 Aug 2022 18:37 UTC

13 points

2 comments3 min readLW link

Three scenarios of pseudo-alignment

Eleni Angelou3 Sep 2022 12:47 UTC

9 points

0 comments3 min readLW link

Worlds Where Iterative Design Fails

johnswentworth30 Aug 2022 20:48 UTC

150 points

26 comments10 min readLW link

Alignment is hard. Communicating that, might be harder

Eleni Angelou1 Sep 2022 16:57 UTC

7 points

8 comments3 min readLW link

Agency engineering: is AI-alignment “to human intent” enough?

catubc2 Sep 2022 18:14 UTC

9 points

10 comments6 min readLW link

Sticky goals: a concrete experiment for understanding deceptive alignment

evhub2 Sep 2022 21:57 UTC

35 points

13 comments3 min readLW link

A Game About AI Alignment (& Meta-Ethics): What Are the Must Haves?

JonathanErhardt5 Sep 2022 7:55 UTC

18 points

13 comments2 min readLW link

How Josiah became an AI safety researcher

Neil Crawford6 Sep 2022 17:17 UTC

4 points

0 comments1 min readLW link

Community Building for Graduate Students: A Targeted Approach

Neil Crawford6 Sep 2022 17:17 UTC

6 points

0 comments3 min readLW link

It’s (not) how you use it

Eleni Angelou7 Sep 2022 17:15 UTC

8 points

1 comment2 min readLW link

Oversight Leagues: The Training Game as a Feature

Paul Bricman9 Sep 2022 10:08 UTC

20 points

6 comments10 min readLW link

Ideological Inference Engines: Making Deontology Differentiable*

Paul Bricman12 Sep 2022 12:00 UTC

6 points

0 comments14 min readLW link

AI Risk Intro 1: Advanced AI Might Be Very Bad

TheMcDouglas and LRudL

11 Sep 2022 10:57 UTC

44 points

13 comments30 min readLW link

AI Safety field-building projects I’d like to see

Akash11 Sep 2022 23:43 UTC

44 points

7 comments6 min readLW link

[Linkpost] A survey on over 300 works about interpretability in deep networks

scasper12 Sep 2022 19:07 UTC

96 points

7 comments2 min readLW link

(arxiv.org)

Representational Tethers: Tying AI Latents To Human Ones

Paul Bricman16 Sep 2022 14:45 UTC

30 points

0 comments16 min readLW link

Risk aversion and GPT-3

hatta_afiq13 Sep 2022 20:50 UTC

1 point

0 comments1 min readLW link

[Question] Would a Misaligned SSI Really Kill Us All?

DragonGod14 Sep 2022 12:15 UTC

6 points

7 comments6 min readLW link

Emily Brontë on: Psychology Required for Serious™ AGI Safety Research

robertzk14 Sep 2022 14:47 UTC

2 points

0 comments1 min readLW link

Precise P(doom) isn’t very important for prioritization or strategy

harsimony14 Sep 2022 17:19 UTC

14 points

6 comments1 min readLW link

Responding to ‘Beyond Hyperanthropomorphism’

ukc1001414 Sep 2022 20:37 UTC

8 points

0 comments16 min readLW link

Capability and Agency as Cornerstones of AI risk — My current model

wilm15 Sep 2022 8:25 UTC

10 points

4 comments12 min readLW link

Understanding Conjecture: Notes from Connor Leahy interview

Akash15 Sep 2022 18:37 UTC

104 points

24 comments15 min readLW link

[Question] Updates on FLI’s Value Aligment Map?

rodeo_flagellum17 Sep 2022 22:27 UTC

17 points

4 comments1 min readLW link

Summaries: Alignment Fundamentals Curriculum

Leon Lang18 Sep 2022 13:08 UTC

43 points

3 comments1 min readLW link

(docs.google.com)

Leveraging Legal Informatics to Align AI

John Nay18 Sep 2022 20:39 UTC

11 points

0 comments3 min readLW link

(forum.effectivealtruism.org)

Here Be AGI Dragons

Oren Montano21 Sep 2022 22:28 UTC

−2 points

0 comments5 min readLW link

AI Risk Intro 2: Solving The Problem

TheMcDouglas and LRudL

22 Sep 2022 13:55 UTC

13 points

0 comments27 min readLW link

Interlude: But Who Optimizes The Optimizer?

Paul Bricman23 Sep 2022 15:30 UTC

15 points

0 comments10 min readLW link

[Question] Why Do AI researchers Rate the Probability of Doom So Low?

Aorou24 Sep 2022 2:33 UTC

7 points

6 comments3 min readLW link

On Generality

Oren Montano26 Sep 2022 4:06 UTC

2 points

0 comments5 min readLW link

Oren’s Field Guide of Bad AGI Outcomes

Oren Montano26 Sep 2022 4:06 UTC

0 points

0 comments1 min readLW link

(Structural) Stability of Coupled Optimizers

Paul Bricman30 Sep 2022 11:28 UTC

25 points

0 comments10 min readLW link

Distribution Shifts and The Importance of AI Safety

Leon Lang29 Sep 2022 22:38 UTC

17 points

2 comments12 min readLW link

Eli’s review of “Is power-seeking AI an existential risk?”

elifland30 Sep 2022 12:21 UTC

58 points

0 comments3 min readLW link

(docs.google.com)

[Question] Any further work on AI Safety Success Stories?

Krieger2 Oct 2022 9:53 UTC

7 points

6 comments1 min readLW link

Announcing the AI Safety Nudge Competition to Help Beat Procrastination

Marc-Everin Carauleanu1 Oct 2022 1:49 UTC

10 points

0 comments1 min readLW link

Boolean Primitives for Coupled Optimizers

Paul Bricman7 Oct 2022 18:02 UTC

9 points

0 comments8 min readLW link

Generative, Episodic Objectives for Safe AI

Michael Glass5 Oct 2022 23:18 UTC

11 points

3 comments8 min readLW link

[Linkpost] “Blueprint for an AI Bill of Rights”—Office of Science and Technology Policy, USA (2022)

rodeo_flagellum5 Oct 2022 16:42 UTC

8 points

4 comments2 min readLW link

(www.whitehouse.gov)

What does it mean for an AGI to be ‘safe’?

So8res7 Oct 2022 4:13 UTC

72 points

32 comments3 min readLW link

Possible miracles

Akash and Thomas Larsen

9 Oct 2022 18:17 UTC

62 points

33 comments8 min readLW link

Instrumental convergence in single-agent systems

Edouard Harris and simonsdsuo

12 Oct 2022 12:24 UTC

27 points

4 comments8 min readLW link

(www.gladstone.ai)

Cataloguing Priors in Theory and Practice

Paul Bricman13 Oct 2022 12:36 UTC

13 points

8 comments7 min readLW link

Misalignment-by-default in multi-agent systems

Edouard Harris and simonsdsuo

13 Oct 2022 15:38 UTC

19 points

8 comments20 min readLW link

(www.gladstone.ai)

Instrumental convergence: scale and physical interactions

Edouard Harris and simonsdsuo

14 Oct 2022 15:50 UTC

15 points

0 comments17 min readLW link

(www.gladstone.ai)

Power-Seeking AI and Existential Risk

Antonio Franca11 Oct 2022 22:50 UTC

5 points

0 comments9 min readLW link

Niceness is unnatural

So8res13 Oct 2022 1:30 UTC

98 points

18 comments8 min readLW link

Greed Is the Root of This Evil

Thane Ruthenis13 Oct 2022 20:40 UTC

21 points

4 comments8 min readLW link

Provably Honest—A First Step

Srijanak De5 Nov 2022 19:18 UTC

10 points

2 comments8 min readLW link

[Question] How easy is it to supervise processes vs outcomes?

Noosphere8918 Oct 2022 17:48 UTC

3 points

0 comments1 min readLW link

Response to Katja Grace’s AI x-risk counterarguments

Erik Jenner and Johannes_Treutlein

19 Oct 2022 1:17 UTC

75 points

18 comments15 min readLW link

POWERplay: An open-source toolchain to study AI power-seeking

Edouard Harris24 Oct 2022 20:03 UTC

22 points

0 comments1 min readLW link

(github.com)

Worldview iPeople—Future Fund’s AI Worldview Prize

Toni MUENDEL28 Oct 2022 1:53 UTC

−22 points

4 comments9 min readLW link

AI Researchers On AI Risk

Scott Alexander22 May 2015 11:16 UTC

18 points

0 comments16 min readLW link

AI as a Civilizational Risk Part 2/6: Behavioral Modification

PashaKamyshev30 Oct 2022 16:57 UTC

9 points

0 comments10 min readLW link

AI as a Civilizational Risk Part 3/6: Anti-economy and Signal Pollution

PashaKamyshev31 Oct 2022 17:03 UTC

7 points

4 comments14 min readLW link

AI as a Civilizational Risk Part 4/6: Bioweapons and Philosophy of Modification

PashaKamyshev1 Nov 2022 20:50 UTC

7 points

1 comment8 min readLW link

AI as a Civilizational Risk Part 5/6: Relationship between C-risk and X-risk

PashaKamyshev3 Nov 2022 2:19 UTC

2 points

0 comments7 min readLW link

AI as a Civilizational Risk Part 6/6: What can be done

PashaKamyshev3 Nov 2022 19:48 UTC

2 points

3 comments4 min readLW link

Am I secretly excited for AI getting weird?

porby29 Oct 2022 22:16 UTC

103 points

4 comments4 min readLW link

My (naive) take on Risks from Learned Optimization

artkpv31 Oct 2022 10:59 UTC

7 points

0 comments5 min readLW link

Clarifying AI X-risk

zac_kenton, Rohin Shah, David Lindner, Vikrant Varma, Vika, Mary Phuong, Ramana Kumar and Elliot Catt

1 Nov 2022 11:03 UTC

104 points

23 comments4 min readLW link

Threat Model Literature Review

zac_kenton, Rohin Shah, David Lindner, Vikrant Varma, Vika, Mary Phuong, Ramana Kumar and Elliot Catt

1 Nov 2022 11:03 UTC

63 points

4 comments25 min readLW link

a casual intro to AI doom and alignment

carado1 Nov 2022 16:38 UTC

11 points

0 comments4 min readLW link

(carado.moe)

Why do we post our AI safety plans on the Internet?

Peter S. Park3 Nov 2022 16:02 UTC

3 points

4 comments11 min readLW link

My summary of “Pragmatic AI Safety”

Eleni Angelou5 Nov 2022 12:54 UTC

2 points

0 comments5 min readLW link

4 Key Assumptions in AI Safety

Prometheus7 Nov 2022 10:50 UTC

20 points

5 comments7 min readLW link

Loss of control of AI is not a likely source of AI x-risk

squek7 Nov 2022 18:44 UTC

−6 points

0 comments5 min readLW link

AI Safety Unconference NeurIPS 2022

Orpheus7 Nov 2022 15:39 UTC

24 points

0 comments1 min readLW link

Value Formation: An Overarching Model

Thane Ruthenis15 Nov 2022 17:16 UTC

29 points

20 comments34 min readLW link

Is AI Gain-of-Function research a thing?

MadHatter12 Nov 2022 2:33 UTC

9 points

2 comments2 min readLW link

The limited upside of interpretability

Peter S. Park15 Nov 2022 18:46 UTC

13 points

11 comments1 min readLW link

Conjecture: a retrospective after 8 months of work

Connor Leahy, Sid Black, Gabriel Alfour and Chris Scammell

23 Nov 2022 17:10 UTC

183 points

9 comments8 min readLW link

Conjecture Second Hiring Round

Connor Leahy, Sid Black, Gabriel Alfour and Chris Scammell

23 Nov 2022 17:11 UTC

91 points

0 comments1 min readLW link

Corrigibility Via Thought-Process Deference

Thane Ruthenis24 Nov 2022 17:06 UTC

14 points

5 comments9 min readLW link

Discussing how to align Transformative AI if it’s developed very soon

elifland and CharlotteS

28 Nov 2022 16:17 UTC

37 points

2 comments30 min readLW link

[Question] Is there any policy for a fair treatment of AIs whose friendliness is in doubt?

nahoj18 Nov 2022 19:01 UTC

15 points

9 comments1 min readLW link

[Question] Will the first AGI agent have been designed as an agent (in addition to an AGI)?

nahoj3 Dec 2022 20:32 UTC

1 point

8 comments1 min readLW link

AI can exploit safety plans posted on the Internet

Peter S. Park4 Dec 2022 12:17 UTC

−19 points

4 comments1 min readLW link

Race to the Top: Benchmarks for AI Safety

Isabella Duan4 Dec 2022 18:48 UTC

12 points

2 comments1 min readLW link

[Question] Who are some prominent reasonable people who are confident that AI won’t kill everyone?

Optimization Process5 Dec 2022 9:12 UTC

72 points

54 comments1 min readLW link

Aligned Behavior is not Evidence of Alignment Past a Certain Level of Intelligence

Ronny Fernandez5 Dec 2022 15:19 UTC

19 points

5 comments7 min readLW link

Foresight for AGI Safety Strategy: Mitigating Risks and Identifying Golden Opportunities

jacquesthibs5 Dec 2022 16:09 UTC

12 points

4 comments8 min readLW link

AI Safety in a Vulnerable World: Requesting Feedback on Preliminary Thoughts

Jordan Arel6 Dec 2022 22:35 UTC

3 points

2 comments3 min readLW link

Fear helped to reduce the nuclear threat, can it do the same to AI alignment?

Igor Ivanov9 Dec 2022 10:04 UTC

6 points

8 comments5 min readLW link

Reflections on the PIBBSS Fellowship 2022

Nora_Ammann and particlemania

11 Dec 2022 21:53 UTC

32 points

0 comments18 min readLW link

[Question] Best introductory overviews of AGI safety?

Jakub Kraus13 Dec 2022 19:01 UTC

17 points

5 comments2 min readLW link

(forum.effectivealtruism.org)

Avoiding Psychopathic AI

Cameron Berg19 Dec 2022 17:01 UTC

28 points

3 comments20 min readLW link

Why I think that teaching philosophy is high impact

Eleni Angelou19 Dec 2022 3:11 UTC

5 points

0 comments2 min readLW link

[Question] Will research in AI risk jinx it? Consequences of training AI on AI risk arguments

Yann Dubois19 Dec 2022 22:42 UTC

5 points

6 comments1 min readLW link

AGI Timelines in Governance: Different Strategies for Different Timeframes

simeon_c and AmberDawn

19 Dec 2022 21:31 UTC

63 points

28 comments10 min readLW link

New AI risk intro from Vox [link post]

Jakub Kraus21 Dec 2022 6:00 UTC

5 points

1 comment2 min readLW link

(www.vox.com)

[Question] Oracle AGI—How can it escape, other than security issues? (Steganography?)

RationalSieve25 Dec 2022 20:14 UTC

3 points

5 comments1 min readLW link

Accurate Models of AI Risk Are Hyperexistential Exfohazards

Thane Ruthenis25 Dec 2022 16:50 UTC

33 points

38 comments9 min readLW link

Safety of Self-Assembled Neuromorphic Hardware

Can Rager26 Dec 2022 18:51 UTC

14 points

2 comments10 min readLW link

(forum.effectivealtruism.org)

Introduction: Bias in Evaluating AGI X-Risks

Remmelt and flandry19

27 Dec 2022 10:27 UTC

0 points

0 comments3 min readLW link

Mere exposure effect: Bias in Evaluating AGI X-Risks

Remmelt and flandry19

27 Dec 2022 14:05 UTC

0 points

2 comments1 min readLW link

Bandwagon effect: Bias in Evaluating AGI X-Risks

Remmelt and flandry19

28 Dec 2022 7:54 UTC

−7 points

0 comments1 min readLW link

In Defense of Wrapper-Minds

Thane Ruthenis28 Dec 2022 18:28 UTC

26 points

34 comments3 min readLW link

Friendly and Unfriendly AGI are Indistinguishable

ErgoEcho29 Dec 2022 22:13 UTC

−4 points

4 comments4 min readLW link

(neologos.co)

Internal Interfaces Are a High-Priority Interpretability Target

Thane Ruthenis29 Dec 2022 17:49 UTC

29 points

6 comments7 min readLW link

CFP for Rebellion and Disobedience in AI workshop

Ram Rachum29 Dec 2022 16:08 UTC

15 points

0 comments1 min readLW link

Reactive devaluation: Bias in Evaluating AGI X-Risks

Remmelt and flandry19

30 Dec 2022 9:02 UTC

−15 points

9 comments1 min readLW link

Curse of knowledge and Naive realism: Bias in Evaluating AGI X-Risks

Remmelt and flandry19

31 Dec 2022 13:33 UTC

−7 points

1 comment1 min readLW link

(www.lesswrong.com)

[Question] Are Mixture-of-Experts Transformers More Interpretable Than Dense Transformers?

simeon_c31 Dec 2022 11:34 UTC

6 points

4 comments1 min readLW link

Challenge to the notion that anything is (maybe) possible with AGI

Remmelt and flandry19

1 Jan 2023 3:57 UTC

−25 points

4 comments1 min readLW link

(mflb.com)

Summary of 80k’s AI problem profile

Jakub Kraus1 Jan 2023 7:30 UTC

7 points

0 comments5 min readLW link

(forum.effectivealtruism.org)

Belief Bias: Bias in Evaluating AGI X-Risks

Remmelt and flandry19

2 Jan 2023 8:59 UTC

−11 points

1 comment1 min readLW link

Status quo bias; System justification: Bias in Evaluating AGI X-Risks

Remmelt and flandry19

3 Jan 2023 2:50 UTC

−11 points

0 comments1 min readLW link

Causal representation learning as a technique to prevent goal misgeneralization

PabloAMC4 Jan 2023 0:07 UTC

16 points

0 comments7 min readLW link

Illusion of truth effect and Ambiguity effect: Bias in Evaluating AGI X-Risks

Remmelt5 Jan 2023 4:05 UTC

−13 points

2 comments1 min readLW link

AI Safety Camp, Virtual Edition 2023

Linda Linsefors6 Jan 2023 11:09 UTC

37 points

9 comments3 min readLW link

(aisafety.camp)

AI Safety Camp: Machine Learning for Scientific Discovery

Eleni Angelou6 Jan 2023 3:21 UTC

2 points

0 comments1 min readLW link

Anchoring focalism and the Identifiable victim effect: Bias in Evaluating AGI X-Risks

Remmelt7 Jan 2023 9:59 UTC

1 point

2 comments1 min readLW link

Big list of AI safety videos

Jakub Kraus9 Jan 2023 6:12 UTC

11 points

2 comments1 min readLW link

(docs.google.com)

The Alignment Problem from a Deep Learning Perspective (major rewrite)

SoerenMind, Richard_Ngo and LawrenceC

10 Jan 2023 16:06 UTC

70 points

8 comments39 min readLW link

(arxiv.org)

[Question] Could Simulating an AGI Taking Over the World Actually Lead to a LLM Taking Over the World?

simeon_c13 Jan 2023 6:33 UTC

15 points

1 comment1 min readLW link

Reflections on Trusting Trust & AI

Itay Yona16 Jan 2023 6:36 UTC

10 points

1 comment3 min readLW link

(mentaleap.ai)

OpenAI’s Alignment Plan is not S.M.A.R.T.

Søren Elverlin18 Jan 2023 6:39 UTC

9 points

19 comments4 min readLW link

Gradient Filtering

Jozdien and janus

18 Jan 2023 20:09 UTC

49 points

15 comments13 min readLW link

Paretoism, a “safe” AI governance paradigm

Jonas Metzger19 Jan 2023 23:09 UTC

3 points

3 comments9 min readLW link

6-paragraph AI risk intro for MAISI

Jakub Kraus19 Jan 2023 9:22 UTC

11 points

0 comments2 min readLW link

(www.maisi.club)

List of technical AI safety exercises and projects

Jakub Kraus19 Jan 2023 9:35 UTC

35 points

5 comments1 min readLW link

(docs.google.com)

NYT: Google will “recalibrate” the risk of releasing AI due to competition with OpenAI

Michael Huang22 Jan 2023 8:38 UTC

47 points

2 comments1 min readLW link

(www.nytimes.com)

Next steps after AGISF at UMich

Jakub Kraus25 Jan 2023 20:57 UTC

10 points

0 comments5 min readLW link

(docs.google.com)

What is the ground reality of countries taking steps to recalibrate AI development towards Alignment first?

Nebuch29 Jan 2023 13:26 UTC

7 points

4 comments3 min readLW link

No comments.