Spu­ri­ous Counterfactuals

TagLast edit: 20 Jun 2022 16:04 UTC by abramdemski

Spurious Counterfactuals are spuriously low evaluations of the quality of a potential action, which are only provable because they are self-fulfilling (usually due to Lob’s theorem). For example, if I know that I go left, then it is logically true that if I went right, I would get −10 utility (because in classical logic, false statements imply any statement). This suggests that if I fully believed that I went left, then I would indeed go left. By Lob’s theorem, I indeed go left.

Building agents who avoid this line of reasoning, despite having full access to their own source code and the ability to logically reason about their own behavior, is one goal of Embedded Agency.

A Pos­si­ble Re­s­olu­tion To Spu­ri­ous Counterfactuals

JoshuaOSHickman6 Dec 2021 18:26 UTC
15 points
5 comments4 min readLW link

Model­ing nat­u­ral­ized de­ci­sion prob­lems in lin­ear logic

jessicata6 May 2020 0:15 UTC
14 points
2 comments6 min readLW link

An In­tro­duc­tion to Löb’s The­o­rem in MIRI Research

orthonormal23 Mar 2015 22:22 UTC
27 points
27 comments2 min readLW link

Embed­ded Agency (full-text ver­sion)

15 Nov 2018 19:49 UTC
126 points
11 comments54 min readLW link
No comments.