RSS

Embed­ded Agency

TagLast edit: 3 Aug 2020 2:34 UTC by Ruby

Embedded Agency is an intuitive notion that an understanding of the theory of rational agents must account for the fact that the agents we create (and we ourselves) are parts of the world, and not separated from it. This is in contrast with much current basic theory of AI (such as solomonoff induction) which implicitly supposes a separation between the agent and the-things-the-agent-has-beliefs about.

Embedded Agency is not a fully formalised research agenda, but Scott Garrabrant and Abram Demski have written the canonical explanation of the idea in their sequence Embedded Agency. This points to many of the core confusions we have about rational agency and attempts to tie them into a single picture.

Embed­ded Agency (full-text ver­sion)

15 Nov 2018 19:49 UTC
114 points
11 comments54 min readLW link

Embed­ded Agents

29 Oct 2018 19:53 UTC
185 points
41 comments1 min readLW link

In­tro­duc­tion to Carte­sian Frames

Scott Garrabrant22 Oct 2020 13:00 UTC
138 points
26 comments22 min readLW link

Hu­mans Are Embed­ded Agents Too

johnswentworth23 Dec 2019 19:21 UTC
75 points
19 comments5 min readLW link

Draft pa­pers for REALab and De­cou­pled Ap­proval on tampering

Jonathan Uesato28 Oct 2020 16:01 UTC
46 points
2 comments1 min readLW link

De­ci­sion Theory

31 Oct 2018 18:41 UTC
105 points
38 comments1 min readLW link

Sub­sys­tem Alignment

6 Nov 2018 16:16 UTC
99 points
12 comments1 min readLW link

Ro­bust Delegation

4 Nov 2018 16:38 UTC
108 points
10 comments1 min readLW link

Embed­ded World-Models

2 Nov 2018 16:07 UTC
85 points
16 comments1 min readLW link

Embed­ded Curiosities

8 Nov 2018 14:19 UTC
85 points
1 comment2 min readLW link

“em­bed­ded self-jus­tifi­ca­tion,” or some­thing like that

nostalgebraist3 Nov 2019 3:20 UTC
36 points
14 comments5 min readLW link
(nostalgebraist.tumblr.com)

(Dou­ble-)In­verse Embed­ded Agency Problem

shminux8 Jan 2020 4:30 UTC
26 points
8 comments2 min readLW link

Embed­ded Agency: Not Just an AI Problem

johnswentworth27 Jun 2019 0:35 UTC
13 points
10 comments2 min readLW link

Embed­ded Agency via Abstraction

johnswentworth26 Aug 2019 23:03 UTC
32 points
20 comments11 min readLW link

(A → B) → A

Scott Garrabrant11 Sep 2018 22:38 UTC
45 points
10 comments2 min readLW link

Bot­world: a cel­lu­lar au­toma­ton for study­ing self-mod­ify­ing agents em­bed­ded in their environment

So8res12 Apr 2014 0:56 UTC
77 points
55 comments7 min readLW link

When does ra­tio­nal­ity-as-search have non­triv­ial im­pli­ca­tions?

nostalgebraist4 Nov 2018 22:42 UTC
64 points
11 comments3 min readLW link

Log­i­cal Up­date­less­ness as a Ro­bust Del­e­ga­tion Problem

Scott Garrabrant27 Oct 2017 21:16 UTC
30 points
2 comments2 min readLW link

Up­dates and ad­di­tions to “Embed­ded Agency”

29 Aug 2020 4:22 UTC
73 points
1 comment3 min readLW link

The whirlpool of reality

G Gordon Worley III27 Sep 2020 2:36 UTC
9 points
2 comments2 min readLW link

Ad­di­tive Oper­a­tions on Carte­sian Frames

Scott Garrabrant26 Oct 2020 15:12 UTC
60 points
6 comments11 min readLW link

Biex­ten­sional Equivalence

Scott Garrabrant28 Oct 2020 14:07 UTC
42 points
13 comments10 min readLW link

Con­trol­lables and Ob­serv­ables, Revisited

Scott Garrabrant29 Oct 2020 16:38 UTC
33 points
5 comments8 min readLW link

Func­tors and Coarse Worlds

Scott Garrabrant30 Oct 2020 15:19 UTC
48 points
4 comments8 min readLW link

Sub-Sums and Sub-Tensors

Scott Garrabrant5 Nov 2020 18:06 UTC
33 points
4 comments8 min readLW link

Mul­ti­plica­tive Oper­a­tions on Carte­sian Frames

Scott Garrabrant3 Nov 2020 19:27 UTC
33 points
23 comments12 min readLW link

Subagents of Carte­sian Frames

Scott Garrabrant2 Nov 2020 22:02 UTC
47 points
4 comments8 min readLW link

Carte­sian Frames Definitions

Rob Bensinger8 Nov 2020 12:44 UTC
24 points
0 comments4 min readLW link

Com­mit­ting, As­sum­ing, Ex­ter­nal­iz­ing, and Internalizing

Scott Garrabrant9 Nov 2020 16:59 UTC
30 points
25 comments10 min readLW link

Eight Defi­ni­tions of Observability

Scott Garrabrant10 Nov 2020 23:37 UTC
33 points
26 comments12 min readLW link

Time in Carte­sian Frames

Scott Garrabrant11 Nov 2020 20:25 UTC
46 points
16 comments7 min readLW link

What Pro­gram Are You?

RobinHanson12 Oct 2009 0:29 UTC
35 points
43 comments2 min readLW link

Time­less De­ci­sion The­ory and Meta-Cir­cu­lar De­ci­sion Theory

Eliezer Yudkowsky20 Aug 2009 22:07 UTC
31 points
37 comments10 min readLW link

Minds: An Introduction

Rob Bensinger11 Mar 2015 19:00 UTC
32 points
1 comment6 min readLW link

Are pre-speci­fied util­ity func­tions about the real world pos­si­ble in prin­ci­ple?

mlogan11 Jul 2018 18:46 UTC
24 points
7 comments4 min readLW link

Ad­di­tive and Mul­ti­plica­tive Subagents

Scott Garrabrant6 Nov 2020 14:26 UTC
19 points
7 comments12 min readLW link

Troll Bridge

abramdemski23 Aug 2019 18:36 UTC
72 points
47 comments12 min readLW link

Coun­ter­fac­tual Plan­ning in AGI Systems

Koen.Holtman3 Feb 2021 13:54 UTC
5 points
0 comments5 min readLW link

Phy­lac­tery De­ci­sion Theory

Bunthut2 Apr 2021 20:55 UTC
14 points
6 comments2 min readLW link

Iden­ti­fi­a­bil­ity Prob­lem for Su­per­ra­tional De­ci­sion Theories

Bunthut9 Apr 2021 20:33 UTC
17 points
13 comments2 min readLW link
No comments.