In­for­ma­tion Hazards

TagLast edit: 3 Jan 2022 13:30 UTC by tilker

An Information Hazard is some true information that could harm people, or other sentient beings, if known. It is tricky to determine policies on information hazards. Some information might genuinely be dangerous, but excessive controls on information has its own perils.

This tag is for discussing the phenomenon of Information Hazards and what to do with them. Not for actual Information Hazards themselves.

An example might be a formula for easily creating cold fusion in your garage, which would be very dangerous. Alternatively, it might be an idea that causes great mental harm to people.

Bostrom’s Typology of Information Hazards

Nick Bostrom coined the term information hazard in a 2011 paper [1] for Review of Contemporary Philosophy. He defines it as follows:

Information hazard: A risk that arises from the dissemination or the potential dissemination of (true) information that may cause harm or enable some agent to cause harm.

Bostrom points out that this is in contrast to the generally accepted principle of information freedom and that, while rare, the possibility of information hazards needs to be considered when making information policies. He proceeds to categorize and define a large number of sub-types of information hazards. For example, he defines artificial intelligence hazard as:

Artificial intelligence hazard: There could be computer-related risks in which the threat would derive primarily from the cognitive sophistication of the program rather than the specific properties of any actuators to which the system initially has access.

The following table is reproduced from Bostrom 2011 [1].

I. By information transfer mode
Data hazard
Idea hazard
Attention hazard
Template hazard
Signaling hazard
Evocation hazard
II. By effect
ADVERSARIAL RISKSCompetiveness hazardEnemy Hazard
Intellectual property hazard
Commitment hazard
Knowing-too-much hazard
RISKS TO SOCIAL ORGANIZATION AND MARKETSNorm hazardInformation asymmetry Hazard
Unveiling hazard
Recognition hazard
Distraction and temptation hazard
Role model hazard
Biasing hazard
De-biasing hazard
Neuropsychological hazard
Information-burying hazard
RISKS TO VALUABLE STATES AND ACTIVITIESPsychological reaction hazardDisappointment hazard
Spoiler hazard
Mindset hazard
Belief-constituted value hazard
(mixed)Embarrassment hazard
RISKS FROM INFORMATION TECHNOLOGY SYSTEMSInformation system hazardInformation infrastructure failure hazard
Information infrastructure misuse hazard
Artificial intelligence hazard

See Also


  1. Bostrom, N. (2011). “Information Hazards: A Typology of Potential Harms from Knowledge”. Review of Contemporary Philosophy 10: 44-79.

Ter­ror­ism, Tylenol, and dan­ger­ous information

Davis_Kingsley12 May 2018 10:20 UTC
142 points
46 comments3 min readLW link


Spiracular17 Sep 2019 2:41 UTC
87 points
14 comments18 min readLW link2 reviews

What are in­for­ma­tion haz­ards?

MichaelA18 Feb 2020 19:34 UTC
41 points
15 comments4 min readLW link

MIRI an­nounces new “Death With Dig­nity” strategy

Eliezer Yudkowsky2 Apr 2022 0:43 UTC
334 points
531 comments18 min readLW link

Ac­cu­rate Models of AI Risk Are Hyper­ex­is­ten­tial Exfohazards

Thane Ruthenis25 Dec 2022 16:50 UTC
30 points
38 comments9 min readLW link

In­for­ma­tion haz­ards: Why you should care and what you can do

23 Feb 2020 20:47 UTC
18 points
4 comments15 min readLW link

Map­ping down­side risks and in­for­ma­tion hazards

20 Feb 2020 14:46 UTC
22 points
0 comments9 min readLW link

Thoughts on the Scope of LessWrong’s In­fo­haz­ard Policies

Ben Pace9 Mar 2020 7:44 UTC
47 points
5 comments8 min readLW link

Needed: AI in­fo­haz­ard policy

Vanessa Kosoy21 Sep 2020 15:26 UTC
62 points
16 comments2 min readLW link

Don’t use ‘in­fo­haz­ard’ for col­lec­tively de­struc­tive info

Eliezer Yudkowsky15 Jul 2022 5:13 UTC
66 points
31 comments1 min readLW link

Con­jec­ture: In­ter­nal In­fo­haz­ard Policy

29 Jul 2022 19:07 UTC
132 points
6 comments19 min readLW link

“In­fo­haz­ard” is a pre­dom­i­nantly con­flict-the­o­retic concept

jessicata2 Dec 2021 17:54 UTC
39 points
17 comments14 min readLW link

The Fu­sion Power Gen­er­a­tor Scenario

johnswentworth8 Aug 2020 18:31 UTC
139 points
28 comments3 min readLW link

Memetic Hazards in Videogames

jimrandomh10 Sep 2010 2:22 UTC
131 points
164 comments3 min readLW link

Oc­cu­pa­tional Infohazards

jessicata18 Dec 2021 20:56 UTC
41 points
133 comments47 min readLW link

Know­ing About Bi­ases Can Hurt People

Eliezer Yudkowsky4 Apr 2007 18:01 UTC
188 points
82 comments2 min readLW link

A point of clar­ifi­ca­tion on in­fo­haz­ard terminology

eukaryote2 Feb 2020 17:43 UTC
49 points
21 comments2 min readLW link

Win­ning vs Truth – In­fo­haz­ard Trade-Offs

eapache7 Mar 2020 22:49 UTC
12 points
11 comments2 min readLW link

Memetic down­side risks: How ideas can evolve and cause harm

25 Feb 2020 19:47 UTC
21 points
3 comments15 min readLW link

Prin­ci­ples of Pri­vacy for Align­ment Research

johnswentworth27 Jul 2022 19:53 UTC
69 points
30 comments7 min readLW link

pub­lish­ing al­ign­ment re­search and exfohazards

carado31 Oct 2022 18:02 UTC
69 points
11 comments1 min readLW link

Con­sume fic­tion wisely

RomanS21 Jan 2022 20:23 UTC
−11 points
53 comments5 min readLW link

A few mis­con­cep­tions sur­round­ing Roko’s basilisk

Rob Bensinger5 Oct 2015 21:23 UTC
90 points
135 comments5 min readLW link

A brief his­tory of eth­i­cally con­cerned scientists

Kaj_Sotala9 Feb 2013 5:50 UTC
103 points
143 comments14 min readLW link

Sal­vage Epistemology

jimrandomh30 Apr 2022 2:10 UTC
98 points
118 comments1 min readLW link

In­fo­haz­ards vs Fork Hazards

jimrandomh5 Jan 2023 9:45 UTC
57 points
16 comments1 min readLW link

Good and bad ways to think about down­side risks

11 Jun 2020 1:38 UTC
17 points
12 comments11 min readLW link

[Question] AI in­ter­pretabil­ity could be harm­ful?

Roman Leventov10 May 2023 20:43 UTC
13 points
2 comments1 min readLW link

[Link and com­men­tary] The Offense-Defense Balance of Scien­tific Knowl­edge: Does Pub­lish­ing AI Re­search Re­duce Mi­suse?

MichaelA16 Feb 2020 19:56 UTC
24 points
4 comments3 min readLW link

SlateS­tarCodex deleted be­cause NYT wants to dox Scott

Rudi C23 Jun 2020 7:51 UTC
89 points
93 comments1 min readLW link

[META] Build­ing a ra­tio­nal­ist com­mu­ni­ca­tion sys­tem to avoid censorship

Donald Hobson23 Jun 2020 14:12 UTC
36 points
33 comments2 min readLW link

Les­sons from the Cold War on In­for­ma­tion Hazards: Why In­ter­nal Com­mu­ni­ca­tion is Critical

Gentzel24 Feb 2018 23:34 UTC
47 points
10 comments4 min readLW link

USA v Pro­gres­sive 1979 excerpt

RyanCarey27 Nov 2017 17:32 UTC
22 points
2 comments2 min readLW link

Shock Level 5: Big Wor­lds and Mo­dal Realism

Roko25 May 2010 23:19 UTC
36 points
158 comments4 min readLW link

Stay­ing Split: Sa­ba­tini and So­cial Justice

[DEACTIVATED] Duncan Sabien8 Jun 2022 8:32 UTC
151 points
28 comments21 min readLW link

Sex­ual Abuse at­ti­tudes might be infohazardous

Pseudonymous Otter19 Jul 2022 18:06 UTC
224 points
64 comments1 min readLW link

Sig­nal­ing Guilt

Krieger8 Oct 2022 20:40 UTC
20 points
6 comments1 min readLW link

Pri­vate al­ign­ment re­search shar­ing and coordination

porby4 Sep 2022 0:01 UTC
58 points
13 comments5 min readLW link

[Question] Is acausal ex­tor­tion pos­si­ble?

sisyphus11 Nov 2022 19:48 UTC
−20 points
34 comments3 min readLW link

[Question] What is our cur­rent best in­fo­haz­ard policy for AGI (safety) re­search?

Roman Leventov15 Nov 2022 22:33 UTC
12 points
2 comments1 min readLW link

Who should write the defini­tive post on Ziz?

NicholasKross15 Dec 2022 6:37 UTC
9 points
45 comments3 min readLW link

[Question] Has pri­vate AGI re­search made in­de­pen­dent safety re­search in­effec­tive already? What should we do about this?

Roman Leventov23 Jan 2023 7:36 UTC
43 points
5 comments5 min readLW link

[Question] Is re­li­gion lo­cally cor­rect for con­se­quen­tial­ists in some in­stances?

Robert Feinstein8 Mar 2023 4:02 UTC
3 points
7 comments1 min readLW link

AI Safety via Luck

Jozdien1 Apr 2023 20:13 UTC
64 points
6 comments11 min readLW link
No comments.