RSS

Embed­ded Agency

Embed­ded Agency is an in­tu­itive no­tion that an un­der­stand­ing of the the­ory of ra­tio­nal agents must ac­count for the fact that the agents we cre­ate (and we our­selves) are parts of the world, and not sep­a­rated from it. This is in con­trast with much cur­rent ba­sic the­ory of AI (such as solomonoff in­duc­tion) which im­plic­itly sup­poses a sep­a­ra­tion be­tween the agent and the-things-the-agent-has-be­liefs about.

Embed­ded Agency is not a fully for­mal­ised re­search agenda, but Scott Garrabrant and Abram Dem­ski have writ­ten the canon­i­cal ex­pla­na­tion of the idea in their se­quence Embed­ded Agency. This points to many of the core con­fu­sions we have about ra­tio­nal agency and at­tempts to tie them into a sin­gle pic­ture.

Embed­ded Agency (full-text ver­sion)

15 Nov 2018 19:49 UTC
103 points
7 comments43 min readLW link

Embed­ded Agents

29 Oct 2018 19:53 UTC
196 points
41 comments1 min readLW link6 nominations2 reviews

Hu­mans Are Embed­ded Agents Too

johnswentworth
23 Dec 2019 19:21 UTC
75 points
19 comments5 min readLW link

De­ci­sion Theory

31 Oct 2018 18:41 UTC
101 points
37 comments1 min readLW link

Sub­sys­tem Alignment

6 Nov 2018 16:16 UTC
121 points
12 comments1 min readLW link

Ro­bust Delegation

4 Nov 2018 16:38 UTC
120 points
10 comments1 min readLW link

Embed­ded World-Models

2 Nov 2018 16:07 UTC
91 points
15 comments1 min readLW link

Embed­ded Curiosities

8 Nov 2018 14:19 UTC
86 points
1 comment2 min readLW link

“em­bed­ded self-jus­tifi­ca­tion,” or some­thing like that

nostalgebraist
3 Nov 2019 3:20 UTC
41 points
14 comments5 min readLW link
(nostalgebraist.tumblr.com)

(Dou­ble-)In­verse Embed­ded Agency Problem

shminux
8 Jan 2020 4:30 UTC
25 points
8 comments2 min readLW link

Embed­ded Agency: Not Just an AI Problem

johnswentworth
27 Jun 2019 0:35 UTC
13 points
10 comments2 min readLW link

Embed­ded Agency via Abstraction

johnswentworth
26 Aug 2019 23:03 UTC
35 points
20 comments11 min readLW link

(A → B) → A

Scott Garrabrant
11 Sep 2018 22:38 UTC
46 points
10 comments2 min readLW link

Bot­world: a cel­lu­lar au­toma­ton for study­ing self-mod­ify­ing agents em­bed­ded in their environment

So8res
12 Apr 2014 0:56 UTC
50 points
55 comments7 min readLW link

When does ra­tio­nal­ity-as-search have non­triv­ial im­pli­ca­tions?

nostalgebraist
4 Nov 2018 22:42 UTC
67 points
11 comments3 min readLW link

Log­i­cal Up­date­less­ness as a Ro­bust Del­e­ga­tion Problem

Scott Garrabrant
27 Oct 2017 21:16 UTC
48 points
2 comments2 min readLW link