Reg­u­la­tion and AI Risk

TagLast edit: 30 Sep 2020 23:44 UTC by Swimmer963 (Miranda Dixon-Luinenburg)

Regulation and AI risk is the debate on whether regulation could be used to reduce the risks of Unfriendly AI, and what forms of regulation would be appropriate.

Several authors have advocated AI research to be regulated, but been vague on the details. Yampolskiy & Fox (2012) note that university research programs in the social and medical sciences are overseen by institutional review boards, and propose setting up analogous review boards to evaluate potential AGI research. In order to be successful, AI regulation would have to be global, and there is the potential for an AI arms race between different nations. Partially because of this, McGinnis (2010) argues that the government should not attempt to regulate AGI development. Rather, it should concentrate on providing funding to research projects intended to create safe AGI. Kaushal & Nolan (2015) point out that regulations on AGI development would result in a speed advantage for any project willing to skirt the regulations, and instead propose government funding (possibly in the form of an “AI Manhattan Project”) for AGI projects meeting particular criteria.

While Shulman & Armstrong (2009) argue the unprecedentedly destabilizing effect of AGI could be a cause for world leaders to cooperate more than usual, the opposite argument can be made as well. Gubrud (1997) argues that molecular nanotechnology could make countries more self-reliant and international cooperation considerably harder, and that AGI could contribute to such a development. AGI technology is also much harder to detect than e.g. nuclear technology is—AGI research can be done in a garage, while nuclear weapons require a substantial infrastructure (McGinnis 2010). On the other hand, Scherer (2015) argues that artificial intelligence could nevertheless be susceptible to regulation due to the increasing prominence of governmental entities and large corporations in AI research and development.

Goertzel & Pitt (2012) suggest that for regulation to be enacted, there might need to be an AGI Sputnik moment—a technological achievement that makes the possibility of AGI evident to the public and policy makers. They note that after such a moment, it might not take a very long time for full human-level AGI to be developed, while the negotiations required to enact new kinds of arms control treaties would take considerably longer.


See also

Ways I Ex­pect AI Reg­u­la­tion To In­crease Ex­tinc­tion Risk

1a3orn4 Jul 2023 17:32 UTC
210 points
32 comments7 min readLW link

How ma­jor gov­ern­ments can help with the most im­por­tant century

HoldenKarnofsky24 Feb 2023 18:20 UTC
28 points
0 comments4 min readLW link

[Linkpost] Chi­nese gov­ern­ment’s guidelines on AI

RomanS10 Dec 2021 21:10 UTC
61 points
14 comments1 min readLW link

List of re­quests for an AI slow­down/​halt.

Cleo Nardo14 Apr 2023 23:55 UTC
46 points
6 comments1 min readLW link

AI Sum­mer Harvest

Cleo Nardo4 Apr 2023 3:35 UTC
130 points
10 comments1 min readLW link

Most Peo­ple Don’t Real­ize We Have No Idea How Our AIs Work

Thane Ruthenis21 Dec 2023 20:02 UTC
137 points
42 comments1 min readLW link

Why liber­tar­i­ans are ad­vo­cat­ing for reg­u­la­tion on AI

RobertM14 Jun 2023 20:59 UTC
35 points
13 comments4 min readLW link

2019 AI Align­ment Liter­a­ture Re­view and Char­ity Comparison

Larks19 Dec 2019 3:00 UTC
130 points
18 comments62 min readLW link

Self-reg­u­la­tion of safety in AI research

Gordon Seidoh Worley25 Feb 2018 23:17 UTC
12 points
6 comments2 min readLW link

Learn­ing so­cietal val­ues from law as part of an AGI al­ign­ment strategy

John Nay21 Oct 2022 2:03 UTC
5 points
18 comments54 min readLW link

An­titrust-Com­pli­ant AI In­dus­try Self-Regulation

Cullen7 Jul 2020 20:53 UTC
9 points
3 comments1 min readLW link

AI Align­ment Pod­cast: An Overview of Tech­ni­cal AI Align­ment in 2018 and 2019 with Buck Sh­legeris and Ro­hin Shah

Palus Astra16 Apr 2020 0:50 UTC
58 points
27 comments89 min readLW link

Ex­ces­sive AI growth-rate yields lit­tle so­cio-eco­nomic benefit.

Cleo Nardo4 Apr 2023 19:13 UTC
27 points
22 comments4 min readLW link

Let’s think about slow­ing down AI

KatjaGrace22 Dec 2022 17:40 UTC
542 points
183 comments38 min readLW link3 reviews

AGI in sight: our look at the game board

18 Feb 2023 22:17 UTC
227 points
135 comments6 min readLW link

[Question] What Are Your Prefer­ences Re­gard­ing The FLI Let­ter?

JenniferRM1 Apr 2023 4:52 UTC
−4 points
122 comments16 min readLW link

[Linkpost] Scott Alexan­der re­acts to OpenAI’s lat­est post

Akash11 Mar 2023 22:24 UTC
27 points
0 comments5 min readLW link

Open Agency model can solve the AI reg­u­la­tion dilemma

Roman Leventov8 Nov 2023 20:00 UTC
22 points
1 comment2 min readLW link

Cruxes on US lead for some do­mes­tic AI regulation

Zach Stein-Perlman10 Sep 2023 18:00 UTC
26 points
3 comments2 min readLW link

PCAST Work­ing Group on Gen­er­a­tive AI In­vites Public Input

Christopher King13 May 2023 22:49 UTC
7 points
0 comments1 min readLW link

Brief notes on the Se­nate hear­ing on AI oversight

Diziet16 May 2023 22:29 UTC
77 points
2 comments2 min readLW link

Rishi Su­nak men­tions “ex­is­ten­tial threats” in talk with OpenAI, Deep­Mind, An­thropic CEOs

24 May 2023 21:06 UTC
34 points
1 comment1 min readLW link

Why Job Dis­place­ment Pre­dic­tions are Wrong: Ex­pla­na­tions of Cog­ni­tive Automation

Moritz Wallawitsch30 May 2023 20:43 UTC
−4 points
0 comments8 min readLW link

RoboNet—A new in­ter­net pro­to­col for AI

antoniomax30 May 2023 17:55 UTC
−13 points
1 comment18 min readLW link

What ex­actly does ‘Slow Down’ look like?

Steve M3 Jun 2023 18:11 UTC
7 points
0 comments1 min readLW link

One im­ple­men­ta­tion of reg­u­la­tory GPU restrictions

porby4 Jun 2023 20:34 UTC
32 points
6 comments5 min readLW link

RAMP—RoboNet Ar­tifi­cial Me­dia Protocol

antoniomax7 Jun 2023 19:01 UTC
−1 points
0 comments19 min readLW link

An­thropic | Chart­ing a Path to AI Accountability

Gabriel Mukobi14 Jun 2023 4:43 UTC
34 points
2 comments3 min readLW link

Ban de­vel­op­ment of un­pre­dictable pow­er­ful mod­els?

TurnTrout20 Jun 2023 1:43 UTC
46 points
25 comments4 min readLW link

EU AI Act passed Ple­nary vote, and X-risk was a main topic

Ariel G.21 Jun 2023 18:33 UTC
17 points
0 comments1 min readLW link

Fron­tier AI Regulation

Zach Stein-Perlman10 Jul 2023 14:30 UTC
20 points
4 comments8 min readLW link

An up­com­ing US Supreme Court case may im­pede AI gov­er­nance efforts

NickGabs16 Jul 2023 23:51 UTC
57 points
17 comments2 min readLW link

Scal­ing and Sus­tain­ing Stan­dards: A Case Study on the Basel Accords

Conrad K.16 Jul 2023 22:01 UTC
8 points
1 comment7 min readLW link

Rus­sian par­li­a­men­tar­ian: let’s ban per­sonal com­put­ers and the Internet

RomanS25 Jul 2023 17:30 UTC
11 points
6 comments2 min readLW link

If we had known the at­mo­sphere would ignite

Jeffs16 Aug 2023 20:28 UTC
53 points
49 comments2 min readLW link

AI Reg­u­la­tion May Be More Im­por­tant Than AI Align­ment For Ex­is­ten­tial Safety

otto.barten24 Aug 2023 11:41 UTC
63 points
39 comments5 min readLW link

Re­port on Fron­tier Model Training

YafahEdelman30 Aug 2023 20:02 UTC
121 points
21 comments21 min readLW link

Against the Open Source /​ Closed Source Di­chotomy: Reg­u­lated Source as a Model for Re­spon­si­ble AI Development

alex.herwix4 Sep 2023 20:25 UTC
4 points
12 comments6 min readLW link

[Linkpost] Mark Zucker­berg con­fronted about Meta’s Llama 2 AI’s abil­ity to give users de­tailed guidance on mak­ing an­thrax—Busi­ness Insider

mic26 Sep 2023 12:05 UTC
18 points
11 comments2 min readLW link

A New Model for Com­pute Cen­ter Verification

Damin Curtis10 Oct 2023 19:22 UTC
8 points
0 comments5 min readLW link

Mud­dling Along Is More Likely Than Dystopia

Jeffrey Heninger20 Oct 2023 21:25 UTC
79 points
9 comments8 min readLW link

Thoughts on Hard­ware limits to Prevent AGI?

jrincayc15 Oct 2023 23:45 UTC
4 points
0 comments9 min readLW link

UNGA Gen­eral De­bate speeches on AI

Odd anon16 Oct 2023 6:36 UTC
6 points
0 comments21 min readLW link

Mauhn Re­leases AI Safety Documentation

Berg Severens3 Jul 2021 21:23 UTC
4 points
0 comments1 min readLW link

Hard­code the AGI to need our ap­proval in­definitely?

MichaelStJules11 Nov 2021 7:04 UTC
2 points
2 comments1 min readLW link

[Question] Con­vince me that hu­man­ity is as doomed by AGI as Yud­kowsky et al., seems to believe

Yitz10 Apr 2022 21:02 UTC
92 points
141 comments2 min readLW link

The Reg­u­la­tory Op­tion: A re­sponse to near 0% sur­vival odds

Matthew Lowenstein11 Apr 2022 22:00 UTC
46 points
21 comments6 min readLW link

A Cri­tique of AI Align­ment Pessimism

ExCeph19 Jul 2022 2:28 UTC
9 points
1 comment9 min readLW link

[Question] Would “Man­hat­tan Pro­ject” style be benefi­cial or dele­te­ri­ous for AI Align­ment?

Just Learning4 Aug 2022 19:12 UTC
5 points
1 comment1 min readLW link

Re­spond­ing to ‘Beyond Hyper­an­thro­po­mor­phism’

ukc1001414 Sep 2022 20:37 UTC
8 points
0 comments16 min readLW link

Lev­er­ag­ing Le­gal In­for­mat­ics to Align AI

John Nay18 Sep 2022 20:39 UTC
11 points
0 comments3 min readLW link

Cryp­tocur­rency Ex­ploits Show the Im­por­tance of Proac­tive Poli­cies for AI X-Risk

eSpencer20 Sep 2022 17:53 UTC
1 point
0 comments4 min readLW link

A Kind­ness, or The Inevitable Con­se­quence of Perfect In­fer­ence (a short story)

samhealy12 Dec 2023 23:03 UTC
6 points
0 comments9 min readLW link

[Job]: AI Stan­dards Devel­op­ment Re­search Assistant

Tony Barrett14 Oct 2022 20:27 UTC
2 points
0 comments2 min readLW link

The Slip­pery Slope from DALLE-2 to Deep­fake Anarchy

scasper5 Nov 2022 14:53 UTC
17 points
9 comments11 min readLW link

[Question] Is there any policy for a fair treat­ment of AIs whose friendli­ness is in doubt?

nahoj18 Nov 2022 19:01 UTC
15 points
9 comments1 min readLW link

Is­sues with un­even AI re­source distribution

User_Luke24 Dec 2022 1:18 UTC
3 points
9 comments5 min readLW link

Who Aligns the Align­ment Re­searchers?

Ben Smith5 Mar 2023 23:22 UTC
40 points
0 comments11 min readLW link

s/​acc: Safe Ac­cel­er­a­tionism Manifesto

lorepieri19 Dec 2023 22:19 UTC
−4 points
5 comments2 min readLW link

AI safety ad­vo­cates should con­sider pro­vid­ing gen­tle push­back fol­low­ing the events at OpenAI

civilsociety22 Dec 2023 18:55 UTC
16 points
5 comments3 min readLW link

Open po­si­tions: Re­search An­a­lyst at the AI Stan­dards Lab

22 Dec 2023 16:31 UTC
17 points
0 comments1 min readLW link

[Question] Which bat­tles should a young per­son pick?

EmanuelJankvist29 Dec 2023 20:28 UTC
14 points
5 comments1 min readLW link

AI, In­tel­lec­tual Prop­erty, and the Techno-Op­ti­mist Revolution

Justin-Diamond31 Jan 2024 18:30 UTC
1 point
0 comments1 min readLW link

The open letter

kornai29 Mar 2023 15:09 UTC
−21 points
2 comments1 min readLW link

The 0.2 OOMs/​year target

Cleo Nardo30 Mar 2023 18:15 UTC
82 points
24 comments5 min readLW link

Risks from GPT-4 Byproduct of Re­cur­sively Op­ti­miz­ing AIs

ben hayum7 Apr 2023 0:02 UTC
73 points
9 comments10 min readLW link

Disagree­ments over the pri­ori­ti­za­tion of ex­is­ten­tial risk from AI

Olivier Coutu26 Oct 2023 17:54 UTC
10 points
0 comments6 min readLW link

Re­sponse to “Co­or­di­nated paus­ing: An eval­u­a­tion-based co­or­di­na­tion scheme for fron­tier AI de­vel­op­ers”

Matthew Wearden30 Oct 2023 17:27 UTC
5 points
2 comments6 min readLW link

A list of all the dead­lines in Bi­den’s Ex­ec­u­tive Order on AI

Valentin Baltadzhiev1 Nov 2023 17:14 UTC
26 points
2 comments11 min readLW link

On ex­clud­ing dan­ger­ous in­for­ma­tion from training

ShayBenMoshe17 Nov 2023 11:14 UTC
23 points
5 comments3 min readLW link

[Question] How much of a con­cern are open-source LLMs in the short, medium and long terms?

JavierCC10 May 2023 9:14 UTC
5 points
0 comments1 min readLW link