Han­so­nian Pre-Rationality

TagLast edit: 30 Jul 2020 18:52 UTC by habryka

In defining Hansonian Pre-Rationality Robin Hanson offers an intriguing argument that, upon learning that our beliefs were created by an irrational process (be it a religious upbringing or a genetic predisposition to paranoid depression), we should update to agree with the alternate version of ourselves who could have had different beliefs. Agents who agree with alternate selves in this way are “pre-rational”. (NOTE: not to be confused with “pre-rational” meaning “not yet rational” or “less than rational”.)

Suppose you are an AI who was designed by a drunk programmer. Your prior contains an “optimism” parameter which broadly skews how you see the world—set it to −100 and you’d expect world-ending danger around every corner, while +100 would make you expect heaven around every corner. Although your powerful learning algorithm allows you to accurately predict the world, the optimism/​pessimism bias never fully goes away: it skews your views about anything you don’t know.

Unfortunately for you, your programmer set the parameter randomly, rather than attempting to figure out which setting was most accurate or useful. You know for a fact they just mashed the num pad randomly.

How should you think about this?

Re-un­der­stand­ing Robin Han­son’s “Pre-Ra­tion­al­ity”

Wei Dai3 Nov 2009 2:58 UTC
35 points
19 comments3 min readLW link

Con­fu­sions Con­cern­ing Pre-Rationality

abramdemski23 May 2018 0:01 UTC
15 points
29 comments5 min readLW link

Reflec­tions on Pre-Rationality

Wei Dai9 Nov 2009 21:42 UTC
28 points
30 comments3 min readLW link

Believ­ing oth­ers’ priors

rk22 Nov 2018 20:44 UTC
8 points
19 comments7 min readLW link

Towards a mechanis­tic un­der­stand­ing of corrigibility

evhub22 Aug 2019 23:20 UTC
47 points
26 comments6 min readLW link

Mis­takes with Con­ser­va­tion of Ex­pected Evidence

abramdemski8 Jun 2019 23:07 UTC
212 points
25 comments12 min readLW link1 review

Towards an In­ten­tional Re­search Agenda

romeostevensit23 Aug 2019 5:27 UTC
20 points
8 comments3 min readLW link

Seek­ing a “Seek­ing Whence ‘Seek Whence’” Sequence

Will_Newsome25 Jun 2012 11:10 UTC
39 points
33 comments3 min readLW link
No comments.