AI Risk

TagLast edit: 16 Jul 2020 10:29 UTC by Ben Pace

AI Risk is analysis of the risks associated with building powerful AI systems.

What failure looks like

paulfchristiano17 Mar 2019 20:18 UTC

258 points

48 comments8 min readLW link 2 nominations 2 reviews

Superintelligence FAQ

Scott Alexander20 Sep 2016 19:00 UTC

63 points

11 comments27 min readLW link

Specification gaming examples in AI

Vika3 Apr 2018 12:30 UTC

39 points

9 comments1 min readLW link

Intuitions about goal-directed behavior

rohinmshah1 Dec 2018 4:25 UTC

46 points

15 comments6 min readLW link

Epistemological Framing for AI Alignment Research

adamShimi8 Mar 2021 22:05 UTC

51 points

7 comments9 min readLW link

What can the principal-agent literature tell us about AI risk?

Alexis Carlier8 Feb 2020 21:28 UTC

99 points

31 comments16 min readLW link

Developmental Stages of GPTs

orthonormal26 Jul 2020 22:03 UTC

131 points

73 comments7 min readLW link

[Question] Will OpenAI’s work unintentionally increase existential risks related to AI?

adamShimi11 Aug 2020 18:16 UTC

48 points

55 comments1 min readLW link

Another (outer) alignment failure story

paulfchristiano7 Apr 2021 20:12 UTC

186 points

37 comments12 min readLW link

How good is humanity at coordination?

Buck21 Jul 2020 20:01 UTC

75 points

43 comments3 min readLW link

A Gym Gridworld Environment for the Treacherous Turn

Michaël Trazzi28 Jul 2018 21:27 UTC

68 points

9 comments3 min readLW link

(github.com)

Are minimal circuits deceptive?

evhub7 Sep 2019 18:11 UTC

56 points

8 comments8 min readLW link

Soft takeoff can still lead to decisive strategic advantage

Daniel Kokotajlo23 Aug 2019 16:39 UTC

113 points

46 comments8 min readLW link 2 nominations 4 reviews

Should we postpone AGI until we reach safety?

otto.barten18 Nov 2020 15:43 UTC

26 points

36 comments3 min readLW link

On Solving Problems Before They Appear: The Weird Epistemologies of Alignment

adamShimi11 Oct 2021 8:20 UTC

73 points

2 comments15 min readLW link

Critiquing “What failure looks like”

Grue_Slinky27 Dec 2019 23:59 UTC

35 points

6 comments3 min readLW link

The Main Sources of AI Risk?

Daniel Kokotajlo and Wei_Dai

21 Mar 2019 18:28 UTC

75 points

22 comments2 min readLW link

Clarifying some key hypotheses in AI alignment

Ben Cottier and rohinmshah

15 Aug 2019 21:29 UTC

77 points

11 comments9 min readLW link

“Taking AI Risk Seriously” (thoughts by Critch)

Raemon29 Jan 2018 9:27 UTC

111 points

68 comments13 min readLW link

Some conceptual highlights from “Disjunctive Scenarios of Catastrophic AI Risk”

Kaj_Sotala12 Feb 2018 12:30 UTC

32 points

4 comments6 min readLW link

(kajsotala.fi)

Non-Adversarial Goodhart and AI Risks

Davidmanheim27 Mar 2018 1:39 UTC

22 points

11 comments6 min readLW link

Six AI Risk/Strategy Ideas

Wei_Dai27 Aug 2019 0:40 UTC

63 points

18 comments4 min readLW link 2 nominations 1 review

[Question] Did AI pioneers not worry much about AI risks?

lisperati9 Feb 2020 19:58 UTC

42 points

9 comments1 min readLW link

Some disjunctive reasons for urgency on AI risk

Wei_Dai15 Feb 2019 20:43 UTC

36 points

24 comments1 min readLW link

Drexler on AI Risk

PeterMcCluskey1 Feb 2019 5:11 UTC

34 points

10 comments9 min readLW link

(www.bayesianinvestor.com)

A shift in arguments for AI risk

Richard_Ngo28 May 2019 13:47 UTC

32 points

7 comments1 min readLW link

(fragile-credences.github.io)

Disentangling arguments for the importance of AI safety

Richard_Ngo21 Jan 2019 12:41 UTC

123 points

23 comments8 min readLW link

AI Safety “Success Stories”

Wei_Dai7 Sep 2019 2:54 UTC

105 points

27 comments4 min readLW link 2 nominations 1 review

Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More

Ben Pace4 Oct 2019 4:08 UTC

179 points

57 comments15 min readLW link 2 nominations 2 reviews

[AN #80]: Why AI risk might be solved without additional intervention from longtermists

rohinmshah2 Jan 2020 18:20 UTC

35 points

94 comments10 min readLW link

(mailchi.mp)

The strategy-stealing assumption

paulfchristiano16 Sep 2019 15:23 UTC

68 points

46 comments12 min readLW link 2 nominations 3 reviews

Thinking soberly about the context and consequences of Friendly AI

Mitchell_Porter16 Oct 2012 4:33 UTC

20 points

39 comments1 min readLW link

Announcement: AI alignment prize winners and next round

cousin_it15 Jan 2018 14:33 UTC

80 points

68 comments2 min readLW link

What Failure Looks Like: Distilling the Discussion

Ben Pace29 Jul 2020 21:49 UTC

74 points

12 comments7 min readLW link

Uber Self-Driving Crash

jefftk7 Nov 2019 15:00 UTC

110 points

1 comment2 min readLW link

(www.jefftk.com)

Reply to Holden on ‘Tool AI’

Eliezer Yudkowsky12 Jun 2012 18:00 UTC

146 points

357 comments17 min readLW link

Stanford Encyclopedia of Philosophy on AI ethics and superintelligence

Kaj_Sotala2 May 2020 7:35 UTC

43 points

19 comments7 min readLW link

(plato.stanford.edu)

AGI Safety Literature Review (Everitt, Lea & Hutter 2018)

Kaj_Sotala4 May 2018 8:56 UTC

13 points

1 comment1 min readLW link

(arxiv.org)

Response to Oren Etzioni’s “How to know if artificial intelligence is about to destroy civilization”

Daniel Kokotajlo27 Feb 2020 18:10 UTC

27 points

5 comments8 min readLW link

Why don’t singularitarians bet on the creation of AGI by buying stocks?

John_Maxwell11 Mar 2020 16:27 UTC

41 points

19 comments4 min readLW link

The problem/solution matrix: Calculating the probability of AI safety “on the back of an envelope”

John_Maxwell20 Oct 2019 8:03 UTC

22 points

4 comments2 min readLW link

Three Stories for How AGI Comes Before FAI

John_Maxwell17 Sep 2019 23:26 UTC

27 points

8 comments6 min readLW link

Brainstorming additional AI risk reduction ideas

John_Maxwell14 Jun 2012 7:55 UTC

19 points

37 comments1 min readLW link

AI Alignment 2018-19 Review

rohinmshah28 Jan 2020 2:19 UTC

121 points

6 comments35 min readLW link

The Fusion Power Generator Scenario

johnswentworth8 Aug 2020 18:31 UTC

109 points

25 comments3 min readLW link

A guide to Iterated Amplification & Debate

Rafael Harth15 Nov 2020 17:14 UTC

59 points

8 comments15 min readLW link

Work on Security Instead of Friendliness?

Wei_Dai21 Jul 2012 18:28 UTC

49 points

107 comments2 min readLW link

An unaligned benchmark

paulfchristiano17 Nov 2018 15:51 UTC

31 points

0 comments9 min readLW link

Artificial Intelligence: A Modern Approach (4th edition) on the Alignment Problem

Zack_M_Davis17 Sep 2020 2:23 UTC

72 points

12 comments5 min readLW link

(aima.cs.berkeley.edu)

Clarifying “What failure looks like”

Sam Clarke20 Sep 2020 20:40 UTC

83 points

13 comments17 min readLW link

Relaxed adversarial training for inner alignment

evhub10 Sep 2019 23:03 UTC

54 points

10 comments27 min readLW link

An overview of 11 proposals for building safe advanced AI

evhub29 May 2020 20:38 UTC

162 points

30 comments38 min readLW link

Risks from Learned Optimization: Introduction

evhub, Chris van Merwijk, vlad_m, Joar Skalse and Scott Garrabrant

31 May 2019 23:44 UTC

144 points

40 comments12 min readLW link 3 nominations 3 reviews

Risks from Learned Optimization: Conclusion and Related Work

evhub, Chris van Merwijk, vlad_m, Joar Skalse and Scott Garrabrant

7 Jun 2019 19:53 UTC

70 points

4 comments6 min readLW link

Deceptive Alignment

evhub, Chris van Merwijk, vlad_m, Joar Skalse and Scott Garrabrant

5 Jun 2019 20:16 UTC

70 points

11 comments17 min readLW link

The Inner Alignment Problem

evhub, Chris van Merwijk, vlad_m, Joar Skalse and Scott Garrabrant

4 Jun 2019 1:20 UTC

77 points

17 comments13 min readLW link

Conditions for Mesa-Optimization

evhub, Chris van Merwijk, vlad_m, Joar Skalse and Scott Garrabrant

1 Jun 2019 20:52 UTC

63 points

47 comments12 min readLW link

AI risk hub in Singapore?

Daniel Kokotajlo29 Oct 2020 11:45 UTC

50 points

18 comments4 min readLW link

Thoughts on Robin Hanson’s AI Impacts interview

Steven Byrnes24 Nov 2019 1:40 UTC

25 points

3 comments7 min readLW link

The AI Safety Game (UPDATED)

Daniel Kokotajlo5 Dec 2020 10:27 UTC

41 points

9 comments3 min readLW link

[Question] Suggestions of posts on the AF to review

adamShimi16 Feb 2021 12:40 UTC

51 points

20 comments1 min readLW link

Google’s Ethical AI team and AI Safety

magfrump20 Feb 2021 9:42 UTC

12 points

16 comments7 min readLW link

Behavioral Sufficient Statistics for Goal-Directedness

adamShimi11 Mar 2021 15:01 UTC

21 points

12 comments9 min readLW link

Review of “Fun with +12 OOMs of Compute”

adamShimi, Joe_Collman and Gyrodiot

28 Mar 2021 14:55 UTC

56 points

18 comments8 min readLW link

April drafts

AI Impacts1 Apr 2021 18:10 UTC

49 points

2 comments1 min readLW link

(aiimpacts.org)

25 Min Talk on MetaEthical.AI with Questions from Stuart Armstrong

June Ku29 Apr 2021 15:38 UTC

17 points

5 comments1 min readLW link

Less Realistic Tales of Doom

Mark Xu6 May 2021 23:01 UTC

97 points

13 comments4 min readLW link

Rogue AGI Embodies Valuable Intellectual Property

Mark Xu and CarlShulman

3 Jun 2021 20:37 UTC

69 points

9 comments3 min readLW link

Environmental Structure Can Cause Instrumental Convergence

TurnTrout22 Jun 2021 22:26 UTC

70 points

44 comments16 min readLW link

(arxiv.org)

Alex Turner’s Research, Comprehensive Information Gathering

adamShimi23 Jun 2021 9:44 UTC

15 points

3 comments3 min readLW link

Sam Altman and Ezra Klein on the AI Revolution

Zack_M_Davis27 Jun 2021 4:53 UTC

37 points

16 comments1 min readLW link

(www.nytimes.com)

Approaches to gradient hacking

adamShimi14 Aug 2021 15:16 UTC

10 points

7 comments8 min readLW link

[Question] What are good alignment conference papers?

adamShimi28 Aug 2021 13:35 UTC

6 points

2 comments1 min readLW link

Why the technological singularity by AGI may never happen

hippke3 Sep 2021 14:19 UTC

5 points

14 comments1 min readLW link

[Question] Conditional on the first AGI being aligned correctly, is a good outcome even still likely?

iamthouthouarti6 Sep 2021 17:30 UTC

2 points

1 comment1 min readLW link

Bayeswatch 7: Wildfire

lsusr8 Sep 2021 5:35 UTC

44 points

6 comments3 min readLW link

[Book Review] “The Alignment Problem” by Brian Christian

lsusr20 Sep 2021 6:36 UTC

60 points

15 comments6 min readLW link

Is progress in ML-assisted theorem-proving beneficial?

MakoYass28 Sep 2021 1:54 UTC

10 points

2 comments1 min readLW link

Interview with Skynet

lsusr30 Sep 2021 2:20 UTC

48 points

1 comment2 min readLW link

Epistemic Strategies of Selection Theorems

adamShimi18 Oct 2021 8:57 UTC

28 points

1 comment12 min readLW link

Access to AI: a human right?

dmtea25 Jul 2020 9:38 UTC

5 points

3 comments2 min readLW link

Agentic Language Model Memes

FactorialCode1 Aug 2020 18:03 UTC

16 points

1 comment2 min readLW link

Conversation with Paul Christiano

abergal11 Sep 2019 23:20 UTC

44 points

6 comments30 min readLW link

(aiimpacts.org)

Transcription of Eliezer’s January 2010 video Q&A

curiousepic14 Nov 2011 17:02 UTC

109 points

9 comments56 min readLW link

Responses to Catastrophic AGI Risk: A Survey

lukeprog8 Jul 2013 14:33 UTC

17 points

8 comments1 min readLW link

How can I reduce existential risk from AI?

lukeprog13 Nov 2012 21:56 UTC

60 points

92 comments8 min readLW link

Thoughts on Ben Garfinkel’s “How sure are we about this AI stuff?”

capybaralet6 Feb 2019 19:09 UTC

25 points

17 comments1 min readLW link

Reframing misaligned AGI’s: well-intentioned non-neurotypical assistants

zhukeepa1 Apr 2018 1:22 UTC

46 points

14 comments2 min readLW link

When is unaligned AI morally valuable?

paulfchristiano25 May 2018 1:57 UTC

58 points

52 comments10 min readLW link

Introducing the AI Alignment Forum (FAQ)

habryka, Ben Pace, Raemon and jimrandomh

29 Oct 2018 21:07 UTC

86 points

8 comments6 min readLW link

Swimming Upstream: A Case Study in Instrumental Rationality

TurnTrout3 Jun 2018 3:16 UTC

64 points

7 comments8 min readLW link

Current AI Safety Roles for Software Engineers

ozziegooen9 Nov 2018 20:57 UTC

69 points

9 comments4 min readLW link

[Question] Why is so much discussion happening in private Google Docs?

Wei_Dai12 Jan 2019 2:19 UTC

93 points

21 comments1 min readLW link

Problems in AI Alignment that philosophers could potentially contribute to

Wei_Dai17 Aug 2019 17:38 UTC

71 points

14 comments2 min readLW link

Two Neglected Problems in Human-AI Safety

Wei_Dai16 Dec 2018 22:13 UTC

74 points

24 comments2 min readLW link

Announcement: AI alignment prize round 4 winners

cousin_it20 Jan 2019 14:46 UTC

74 points

41 comments1 min readLW link

Soon: a weekly AI Safety prerequisites module on LessWrong

toonalfrink30 Apr 2018 13:23 UTC

35 points

10 comments1 min readLW link

And the AI would have got away with it too, if...

Stuart_Armstrong22 May 2019 21:35 UTC

75 points

7 comments1 min readLW link

2017 AI Safety Literature Review and Charity Comparison

Larks24 Dec 2017 18:52 UTC

41 points

5 comments23 min readLW link

Should ethicists be inside or outside a profession?

Eliezer Yudkowsky12 Dec 2018 1:40 UTC

79 points

6 comments9 min readLW link

I Vouch For MIRI

Zvi17 Dec 2017 17:50 UTC

34 points

9 comments5 min readLW link

(thezvi.wordpress.com)

Beware of black boxes in AI alignment research

cousin_it18 Jan 2018 15:07 UTC

39 points

10 comments1 min readLW link

AI Alignment Prize: Round 2 due March 31, 2018

Zvi12 Mar 2018 12:10 UTC

28 points

2 comments3 min readLW link

(thezvi.wordpress.com)

Three AI Safety Related Ideas

Wei_Dai13 Dec 2018 21:32 UTC

64 points

38 comments2 min readLW link

A rant against robots

Lê Nguyên Hoang14 Jan 2020 22:03 UTC

61 points

7 comments5 min readLW link

Opportunities for individual donors in AI safety

alexflint31 Mar 2018 18:37 UTC

30 points

3 comments11 min readLW link

But exactly how complex and fragile?

KatjaGrace3 Nov 2019 18:20 UTC

65 points

32 comments3 min readLW link 2 nominations 1 review

(meteuphoric.com)

Course recommendations for Friendliness researchers

Louie9 Jan 2013 14:33 UTC

92 points

112 comments10 min readLW link

AI Safety Research Camp—Project Proposal

David_Kristoffersson2 Feb 2018 4:25 UTC

29 points

11 comments8 min readLW link

AI Summer Fellows Program

colm21 Mar 2018 15:32 UTC

21 points

0 comments1 min readLW link

The genie knows, but doesn’t care

Rob Bensinger6 Sep 2013 6:42 UTC

88 points

519 comments8 min readLW link

Alignment Newsletter #13: 07/02/18

rohinmshah2 Jul 2018 16:10 UTC

70 points

12 comments8 min readLW link

(mailchi.mp)

An Increasingly Manipulative Newsfeed

Michaël Trazzi1 Jul 2019 15:26 UTC

60 points

16 comments5 min readLW link

The simple picture on AI safety

alexflint27 May 2018 19:43 UTC

31 points

10 comments2 min readLW link

Elon Musk donates $10M to the Future of Life Institute to keep AI beneficial

Paul Crowley15 Jan 2015 16:33 UTC

78 points

52 comments1 min readLW link

Strategic implications of AIs’ ability to coordinate at low cost, for example by merging

Wei_Dai25 Apr 2019 5:08 UTC

60 points

45 comments2 min readLW link 2 nominations 1 review

Modeling AGI Safety Frameworks with Causal Influence Diagrams

Ramana Kumar21 Jun 2019 12:50 UTC

43 points

6 comments1 min readLW link

(arxiv.org)

Henry Kissinger: AI Could Mean the End of Human History

ESRogs15 May 2018 20:11 UTC

17 points

12 comments1 min readLW link

(www.theatlantic.com)

Toy model of the AI control problem: animated version

Stuart_Armstrong10 Oct 2017 11:06 UTC

25 points

8 comments1 min readLW link

A Visualization of Nick Bostrom’s Superintelligence

[deleted]23 Jul 2014 0:24 UTC

62 points

28 comments3 min readLW link

AI Alignment Research Overview (by Jacob Steinhardt)

Ben Pace6 Nov 2019 19:24 UTC

43 points

0 comments7 min readLW link

(docs.google.com)

A general model of safety-oriented AI development

Wei_Dai11 Jun 2018 21:00 UTC

65 points

8 comments1 min readLW link

Counterfactual Oracles = online supervised learning with random selection of training episodes

Wei_Dai10 Sep 2019 8:29 UTC

44 points

26 comments3 min readLW link

Siren worlds and the perils of over-optimised search

Stuart_Armstrong7 Apr 2014 11:00 UTC

69 points

417 comments7 min readLW link

Top 9+2 myths about AI risk

Stuart_Armstrong29 Jun 2015 20:41 UTC

64 points

46 comments2 min readLW link

Rohin Shah on reasons for AI optimism

abergal31 Oct 2019 12:10 UTC

40 points

58 comments1 min readLW link

(aiimpacts.org)

Plausibly, almost every powerful algorithm would be manipulative

Stuart_Armstrong6 Feb 2020 11:50 UTC

38 points

25 comments3 min readLW link

The Magnitude of His Own Folly

Eliezer Yudkowsky30 Sep 2008 11:31 UTC

54 points

128 comments6 min readLW link

AI alignment landscape

paulfchristiano13 Oct 2019 2:10 UTC

40 points

3 comments1 min readLW link

(ai-alignment.com)

Launched: Friendship is Optimal

iceman15 Nov 2012 4:57 UTC

64 points

31 comments1 min readLW link

Friendship is Optimal: A My Little Pony fanfic about an optimization process

iceman8 Sep 2012 6:16 UTC

99 points

152 comments1 min readLW link

Do Earths with slower economic growth have a better chance at FAI?

Eliezer Yudkowsky12 Jun 2013 19:54 UTC

54 points

176 comments4 min readLW link

Idea: Open Access AI Safety Journal

G Gordon Worley III23 Mar 2018 18:27 UTC

28 points

11 comments1 min readLW link

G.K. Chesterton On AI Risk

Scott Alexander1 Apr 2017 19:00 UTC

14 points

0 comments7 min readLW link

The Hidden Complexity of Wishes

Eliezer Yudkowsky24 Nov 2007 0:12 UTC

115 points

135 comments7 min readLW link

The Friendly AI Game

bentarm15 Mar 2011 16:45 UTC

50 points

178 comments1 min readLW link

Q&A with Jürgen Schmidhuber on risks from AI

XiXiDu15 Jun 2011 15:51 UTC

54 points

45 comments4 min readLW link

[Question] What should an Einstein-like figure in Machine Learning do?

Razied5 Aug 2020 23:52 UTC

3 points

3 comments1 min readLW link

Takeaways from safety by default interviews

AI Impacts and abergal

3 Apr 2020 17:20 UTC

23 points

2 comments13 min readLW link

(aiimpacts.org)

Field-Building and Deep Models

Ben Pace13 Jan 2018 21:16 UTC

21 points

12 comments4 min readLW link

Critique my Model: The EV of AGI to Selfish Individuals

ozziegooen8 Apr 2018 20:04 UTC

19 points

9 comments4 min readLW link

‘Dumb’ AI observes and manipulates controllers

Stuart_Armstrong13 Jan 2015 13:35 UTC

52 points

19 comments2 min readLW link

2019 AI Alignment Literature Review and Charity Comparison

Larks19 Dec 2019 3:00 UTC

130 points

18 comments62 min readLW link

Book review: Architects of Intelligence by Martin Ford (2018)

ofer11 Aug 2020 17:30 UTC

15 points

0 comments2 min readLW link

Qualitative Strategies of Friendliness

Eliezer Yudkowsky30 Aug 2008 2:12 UTC

14 points

56 comments12 min readLW link

Dreams of Friendliness

Eliezer Yudkowsky31 Aug 2008 1:20 UTC

25 points

81 comments9 min readLW link

Conceptual issues in AI safety: the paradigmatic gap

vedevazz24 Jun 2018 15:09 UTC

33 points

0 comments1 min readLW link

(www.foldl.me)

On unfixably unsafe AGI architectures

Steven Byrnes19 Feb 2020 21:16 UTC

30 points

8 comments5 min readLW link

A toy model of the treacherous turn

Stuart_Armstrong8 Jan 2016 12:58 UTC

35 points

13 comments6 min readLW link

Allegory On AI Risk, Game Theory, and Mithril

James_Miller13 Feb 2017 20:41 UTC

42 points

57 comments3 min readLW link

1hr talk: Intro to AGI safety

Steven Byrnes18 Jun 2019 21:41 UTC

33 points

4 comments24 min readLW link

The Evil AI Overlord List

Stuart_Armstrong20 Nov 2012 17:02 UTC

44 points

80 comments1 min readLW link

What I would like the SIAI to publish

XiXiDu1 Nov 2010 14:07 UTC

36 points

225 comments3 min readLW link

Evaluating the feasibility of SI’s plan

JoshuaFox10 Jan 2013 8:17 UTC

38 points

188 comments4 min readLW link

Q&A with experts on risks from AI #1

XiXiDu8 Jan 2012 11:46 UTC

45 points

67 comments9 min readLW link

Algo trading is a central example of AI risk

Vanessa Kosoy28 Jul 2018 20:31 UTC

24 points

5 comments1 min readLW link

Will the world’s elites navigate the creation of AI just fine?

lukeprog31 May 2013 18:49 UTC

36 points

266 comments2 min readLW link

Let’s talk about “Convergent Rationality”

capybaralet12 Jun 2019 21:53 UTC

35 points

33 comments6 min readLW link

Breaking Oracles: superrationality and acausal trade

Stuart_Armstrong25 Nov 2019 10:40 UTC

25 points

15 comments1 min readLW link

Q&A with Stan Franklin on risks from AI

XiXiDu11 Jun 2011 15:22 UTC

36 points

10 comments2 min readLW link

Muehlhauser-Goertzel Dialogue, Part 1

lukeprog16 Mar 2012 17:12 UTC

42 points

161 comments33 min readLW link

[LINK] NYT Article about Existential Risk from AI

[deleted]28 Jan 2013 10:37 UTC

38 points

23 comments1 min readLW link

Reframing the Problem of AI Progress

Wei_Dai12 Apr 2012 19:31 UTC

32 points

47 comments1 min readLW link

New AI risks research institute at Oxford University

lukeprog16 Nov 2011 18:52 UTC

36 points

10 comments1 min readLW link

Thoughts on the Feasibility of Prosaic AGI Alignment?

iamthouthouarti21 Aug 2020 23:25 UTC

8 points

10 comments1 min readLW link

Memes and Rational Decisions

inferential9 Jan 2015 6:42 UTC

35 points

17 comments10 min readLW link

Levels of AI Self-Improvement

avturchin29 Apr 2018 11:45 UTC

9 points

0 comments39 min readLW link

Optimising Society to Constrain Risk of War from an Artificial Superintelligence

JohnCDraper30 Apr 2020 10:47 UTC

3 points

0 comments51 min readLW link

Some Thoughts on Singularity Strategies

Wei_Dai13 Jul 2011 2:41 UTC

37 points

29 comments3 min readLW link

A trick for Safer GPT-N

Razied23 Aug 2020 0:39 UTC

7 points

1 comment2 min readLW link

against “AI risk”

Wei_Dai11 Apr 2012 22:46 UTC

35 points

91 comments1 min readLW link

“Smarter than us” is out!

Stuart_Armstrong25 Feb 2014 15:50 UTC

41 points

57 comments1 min readLW link

Analysing: Dangerous messages from future UFAI via Oracles

Stuart_Armstrong22 Nov 2019 14:17 UTC

22 points

16 comments4 min readLW link

Q&A with Abram Demski on risks from AI

XiXiDu17 Jan 2012 9:43 UTC

33 points

71 comments9 min readLW link

Q&A with experts on risks from AI #2

XiXiDu9 Jan 2012 19:40 UTC

22 points

29 comments7 min readLW link

AI Safety Discussion Day

Linda Linsefors15 Sep 2020 14:40 UTC

20 points

0 comments1 min readLW link

A long reply to Ben Garfinkel on Scrutinizing Classic AI Risk Arguments

Søren Elverlin27 Sep 2020 17:51 UTC

16 points

6 comments1 min readLW link

Online AI Safety Discussion Day

Linda Linsefors8 Oct 2020 12:11 UTC

5 points

0 comments1 min readLW link

Military AI as a Convergent Goal of Self-Improving AI

avturchin13 Nov 2017 12:17 UTC

5 points

3 comments1 min readLW link

Neural program synthesis is a dangerous technology

syllogism12 Jan 2018 16:19 UTC

9 points

6 comments2 min readLW link

New, Brief Popular-Level Introduction to AI Risks and Superintelligence

LyleN23 Jan 2015 15:43 UTC

33 points

3 comments1 min readLW link

FAI Research Constraints and AGI Side Effects

JustinShovelain3 Jun 2015 19:25 UTC

26 points

59 comments7 min readLW link

European Master’s Programs in Machine Learning, Artificial Intelligence, and related fields

Master Programs ML/AI14 Nov 2020 15:51 UTC

25 points

8 comments1 min readLW link

The mind-killer

Paul Crowley2 May 2009 16:49 UTC

29 points

160 comments2 min readLW link

[Question] Should I do it?

MrLight19 Nov 2020 1:08 UTC

−3 points

16 comments2 min readLW link

Rationalising humans: another mugging, but not Pascal’s

Stuart_Armstrong14 Nov 2017 15:46 UTC

7 points

1 comment3 min readLW link

Machine learning could be fundamentally unexplainable

George16 Dec 2020 13:32 UTC

25 points

15 comments15 min readLW link

(cerebralab.com)

[Question] What do you make of AGI:unaligned::spaceships:not enough food?

Ronny22 Feb 2020 14:14 UTC

4 points

3 comments1 min readLW link

Risk Map of AI Systems

VojtaKovarik and Jan_Kulveit

15 Dec 2020 9:16 UTC

25 points

3 comments8 min readLW link

Edge of the Cliff

akaTrickster5 Jan 2021 17:21 UTC

1 point

0 comments5 min readLW link

[Question] Does it become easier, or harder, for the world to coordinate around not building AGI as time goes on?

Eli Tyre29 Jul 2019 22:59 UTC

86 points

31 comments3 min readLW link 2 nominations 2 reviews

Grey Goo Requires AI

harsimony15 Jan 2021 4:45 UTC

8 points

11 comments4 min readLW link

(harsimony.wordpress.com)

AISU 2021

Linda Linsefors30 Jan 2021 17:40 UTC

28 points

2 comments1 min readLW link

Nonperson Predicates

Eliezer Yudkowsky27 Dec 2008 1:47 UTC

47 points

176 comments6 min readLW link

Engaging First Introductions to AI Risk

Rob Bensinger19 Aug 2013 6:26 UTC

31 points

21 comments3 min readLW link

Formal Solution to the Inner Alignment Problem

michaelcohen18 Feb 2021 14:51 UTC

46 points

123 comments2 min readLW link

[Question] What are the biggest current impacts of AI?

Sam Clarke7 Mar 2021 21:44 UTC

15 points

4 comments1 min readLW link

[Question] Is a Self-Iterating AGI Vulnerable to Thompson-style Trojans?

sxae25 Mar 2021 14:46 UTC

15 points

7 comments3 min readLW link

AI oracles on blockchain

Caravaggio6 Apr 2021 20:13 UTC

4 points

0 comments3 min readLW link

What if AGI is near?

Wulky Wilkinsen14 Apr 2021 0:05 UTC

11 points

5 comments1 min readLW link

[Question] Is there anything that can stop AGI development in the near term?

Wulky Wilkinsen22 Apr 2021 20:37 UTC

5 points

5 comments1 min readLW link

[Question] [timeboxed exercise] write me your model of AI human-existential safety and the alignment problems in 15 minutes

Quinn4 May 2021 19:10 UTC

6 points

2 comments1 min readLW link

AI Safety Research Project Ideas

Owain_Evans21 May 2021 13:39 UTC

56 points

1 comment3 min readLW link

Survey on AI existential risk scenarios

Sam Clarke, Alexis Carlier and Jonas Schuett

8 Jun 2021 17:12 UTC

55 points

10 comments7 min readLW link

[Question] What are some claims or opinions about multi-multi delegation you’ve seen in the memeplex that you think deserve scrutiny?

Quinn27 Jun 2021 17:44 UTC

16 points

6 comments2 min readLW link

Mauhn Releases AI Safety Documentation

Berg Severens3 Jul 2021 21:23 UTC

4 points

0 comments1 min readLW link

A gentle apocalypse

pchvykov16 Aug 2021 5:03 UTC

3 points

5 comments3 min readLW link

Could you have stopped Chernobyl?

Carlos Ramirez27 Aug 2021 1:48 UTC

28 points

17 comments8 min readLW link

The Governance Problem and the “Pretty Good” X-Risk

Zach Stein-Perlman29 Aug 2021 18:00 UTC

5 points

2 comments12 min readLW link

Distinguishing AI takeover scenarios

Sam Clarke and Sammy Martin

8 Sep 2021 16:19 UTC

60 points

9 comments14 min readLW link

The alignment problem in different capability regimes

Buck9 Sep 2021 19:46 UTC

84 points

12 comments5 min readLW link

How truthful is GPT-3? A benchmark for language models

Owain_Evans16 Sep 2021 10:09 UTC

54 points

24 comments6 min readLW link

Investigating AI Takeover Scenarios

Sammy Martin17 Sep 2021 18:47 UTC

26 points

1 comment27 min readLW link

AI takeoff story: a continuation of progress by other means

Edouard Harris27 Sep 2021 15:55 UTC

66 points

13 comments10 min readLW link

A brief review of the reasons multi-objective RL could be important in AI Safety Research

Ben Smith29 Sep 2021 17:09 UTC

14 points

7 comments10 min readLW link

The Dark Side of Cognition Hypothesis

Cameron Berg3 Oct 2021 20:10 UTC

19 points

1 comment16 min readLW link

Truthful AI: Developing and governing AI that does not lie

Owain_Evans, owencb and Lanrian

18 Oct 2021 18:37 UTC

71 points

3 comments10 min readLW link

No comments.