RSS

Subagents

TagLast edit: 29 Jul 2020 12:45 UTC by Kaj_Sotala

Subagents refers to the idea that rather than thinking of the mind as an entity with one set of goals and beliefs, it includes many independently acting components, each of which might have varying goals and beliefs. One intuitive way of expressing this is the expression “one part of me wants X, but another part of me wants Y instead”.

While the name implies some degree of independent agency on part of the subagents, they may also be viewed as being more passive entities. For example, the “parts” in the above example may be considered different sets of beliefs, accessed one at a time by the same system.

The Multiagent Models of Mind sequence explores the notion of subagents in detail. Akrasia (acting against one’s better judgment, such as by procrastinating) may involve subagent disagreement. Internal Double Crux is one technique for getting subagents to agree with each other.

Why Subagents?

johnswentworth1 Aug 2019 22:17 UTC
169 points
48 comments7 min readLW link1 review

Build­ing up to an In­ter­nal Fam­ily Sys­tems model

Kaj_Sotala26 Jan 2019 12:25 UTC
239 points
84 comments28 min readLW link2 reviews

A non-mys­ti­cal ex­pla­na­tion of in­sight med­i­ta­tion and the three char­ac­ter­is­tics of ex­is­tence: in­tro­duc­tion and preamble

Kaj_Sotala5 May 2020 19:09 UTC
119 points
37 comments12 min readLW link

Men­tal Mountains

Scott Alexander27 Nov 2019 5:30 UTC
126 points
14 comments15 min readLW link1 review
(slatestarcodex.com)

Book Sum­mary: Con­scious­ness and the Brain

Kaj_Sotala16 Jan 2019 14:43 UTC
149 points
21 comments26 min readLW link1 review

Forc­ing your­self to keep your iden­tity small is self-harm

Gordon Seidoh Worley3 Apr 2021 14:03 UTC
35 points
9 comments2 min readLW link

My cur­rent take on In­ter­nal Fam­ily Sys­tems “parts”

Kaj_Sotala26 Jun 2022 17:40 UTC
73 points
8 comments3 min readLW link
(kajsotala.fi)

Subagents, in­tro­spec­tive aware­ness, and blending

Kaj_Sotala2 Mar 2019 12:53 UTC
93 points
18 comments9 min readLW link

Subagents, akra­sia, and co­her­ence in humans

Kaj_Sotala25 Mar 2019 14:24 UTC
123 points
31 comments16 min readLW link

In­te­grat­ing dis­agree­ing subagents

Kaj_Sotala14 May 2019 14:06 UTC
130 points
15 comments21 min readLW link

Subagents, neu­ral Tur­ing ma­chines, thought se­lec­tion, and blindspots

Kaj_Sotala6 Aug 2019 21:15 UTC
77 points
3 comments12 min readLW link

Subagents, trauma and rationality

Kaj_Sotala14 Aug 2019 13:14 UTC
97 points
4 comments19 min readLW link

Book sum­mary: Un­lock­ing the Emo­tional Brain

Kaj_Sotala8 Oct 2019 19:11 UTC
277 points
39 comments21 min readLW link3 reviews

Com­plex Be­hav­ior from Sim­ple (Sub)Agents

moridinamael10 May 2019 21:44 UTC
110 points
13 comments9 min readLW link1 review

[Question] How effec­tive are tul­pas?

Evenflair9 Mar 2020 17:35 UTC
38 points
58 comments2 min readLW link

Si­mu­late and Defer To More Ra­tional Selves

LoganStrohl17 Sep 2014 18:11 UTC
207 points
115 comments5 min readLW link

Con­sis­tently Inconsistent

Kaj_Sotala4 Aug 2011 22:33 UTC
80 points
25 comments5 min readLW link

Shoulder Ad­vi­sors 101

Duncan_Sabien9 Oct 2021 5:30 UTC
186 points
126 comments14 min readLW link2 reviews

[Question] How to se­lect a long-term goal and al­ign my mind to­wards it?

Alexander24 Dec 2021 11:40 UTC
18 points
8 comments2 min readLW link

Se­quence in­tro­duc­tion: non-agent and mul­ti­a­gent mod­els of mind

Kaj_Sotala7 Jan 2019 14:12 UTC
113 points
15 comments7 min readLW link1 review

Sys­tem 2 as work­ing-mem­ory aug­mented Sys­tem 1 reasoning

Kaj_Sotala25 Sep 2019 8:39 UTC
102 points
19 comments16 min readLW link

A mechanis­tic model of meditation

Kaj_Sotala6 Nov 2019 21:37 UTC
121 points
8 comments21 min readLW link

A non-mys­ti­cal ex­pla­na­tion of “no-self” (three char­ac­ter­is­tics se­ries)

Kaj_Sotala8 May 2020 10:37 UTC
99 points
64 comments20 min readLW link1 review

Crav­ing, suffer­ing, and pre­dic­tive pro­cess­ing (three char­ac­ter­is­tics se­ries)

Kaj_Sotala15 May 2020 13:21 UTC
75 points
48 comments19 min readLW link

From self to crav­ing (three char­ac­ter­is­tics se­ries)

Kaj_Sotala22 May 2020 12:16 UTC
50 points
21 comments11 min readLW link

On the con­struc­tion of the self

Kaj_Sotala29 May 2020 13:04 UTC
63 points
17 comments17 min readLW link

Three char­ac­ter­is­tics: impermanence

Kaj_Sotala5 Jun 2020 7:48 UTC
66 points
3 comments18 min readLW link

Con­flicts Between Men­tal Subagents: Ex­pand­ing Wei Dai’s Master-Slave Model

Scott Alexander4 Aug 2010 9:16 UTC
67 points
82 comments10 min readLW link

Con­di­tions un­der which mis­al­igned sub­agents can (not) arise in classifiers

anon111 Jul 2018 1:52 UTC
12 points
2 comments2 min readLW link

Syn­the­sis of sub­agents: exercise

Julija Kobrinovich20 Sep 2019 17:24 UTC
10 points
2 comments14 min readLW link

What Value Subagents?

Gordon Seidoh Worley20 Jul 2017 19:19 UTC
7 points
1 comment4 min readLW link
(mapandterritory.org)

Seven Shiny Stories

Alicorn1 Jun 2010 0:43 UTC
137 points
34 comments7 min readLW link

Embed­ded Agency (full-text ver­sion)

15 Nov 2018 19:49 UTC
154 points
16 comments54 min readLW link

Two Co­or­di­na­tion Styles

abramdemski7 Feb 2018 9:00 UTC
38 points
14 comments7 min readLW link

In­ter­nal­iz­ing In­ter­nal Dou­ble Crux

TurnTrout30 Apr 2018 18:23 UTC
33 points
12 comments4 min readLW link

A Master-Slave Model of Hu­man Preferences

Wei_Dai29 Dec 2009 1:02 UTC
94 points
94 comments3 min readLW link

Self-em­pa­thy as a source of “willpower”

Academian26 Oct 2010 14:20 UTC
70 points
31 comments2 min readLW link

Ro­bust Agency for Peo­ple and Organizations

Raemon19 Jul 2019 1:18 UTC
59 points
10 comments12 min readLW link

Multi-agent pre­dic­tive minds and AI alignment

Jan_Kulveit12 Dec 2018 23:48 UTC
60 points
18 comments10 min readLW link

A Frame­work for In­ter­nal Debugging

Matt Goldenberg16 Jan 2019 16:04 UTC
41 points
3 comments5 min readLW link

On In­ter­nal Fam­ily Sys­tems and multi-agent minds: a re­ply to PJ Eby

Kaj_Sotala29 Oct 2019 14:56 UTC
39 points
31 comments25 min readLW link

City of Lights

Alicorn31 Mar 2010 23:30 UTC
44 points
43 comments4 min readLW link

Embed­ded Agency via Abstraction

johnswentworth26 Aug 2019 23:03 UTC
40 points
20 comments11 min readLW link

In­trap­er­sonal negotiation

datadataeverywhere23 Jan 2011 23:02 UTC
34 points
42 comments4 min readLW link

Neu­ral Ba­sis for Global Workspace Theory

Hazard22 Jun 2020 4:19 UTC
31 points
9 comments8 min readLW link

Ten­ta­tively con­sid­er­ing emo­tional sto­ries (IFS and “get­ting into Self”)

Kaj_Sotala30 Nov 2018 7:40 UTC
37 points
31 comments4 min readLW link
(kajsotala.fi)

Strate­gic ig­no­rance and plau­si­ble deniability

Kaj_Sotala10 Aug 2011 9:30 UTC
60 points
59 comments4 min readLW link

Subagents of Carte­sian Frames

Scott Garrabrant2 Nov 2020 22:02 UTC
48 points
5 comments8 min readLW link

Com­mit­ting, As­sum­ing, Ex­ter­nal­iz­ing, and Internalizing

Scott Garrabrant9 Nov 2020 16:59 UTC
31 points
25 comments10 min readLW link

Eight Defi­ni­tions of Observability

Scott Garrabrant10 Nov 2020 23:37 UTC
34 points
26 comments12 min readLW link

Two Explorations

alkjash16 Dec 2020 21:27 UTC
63 points
8 comments9 min readLW link
(radimentary.wordpress.com)

Men­tal sub­agent im­pli­ca­tions for AI Safety

moridinamael3 Jan 2021 18:59 UTC
11 points
0 comments3 min readLW link

Why Pro­duc­tivity Sys­tems Don’t Stick

Matt Goldenberg16 Jan 2021 17:45 UTC
53 points
21 comments3 min readLW link

Non-Co­er­cive Perfectionism

Matt Goldenberg26 Jan 2021 16:53 UTC
22 points
25 comments3 min readLW link

[Question] Any­one been through IFS or co­her­ence ther­apy?

warrenjordan15 Mar 2021 18:35 UTC
4 points
3 comments1 min readLW link

Re­ward Is Not Enough

Steven Byrnes16 Jun 2021 13:52 UTC
110 points
19 comments10 min readLW link1 review

Ac­tu­ally updating

SaraHax23 Aug 2019 17:46 UTC
52 points
10 comments4 min readLW link

The Game of Masks

Slimepriestess27 Apr 2022 18:03 UTC
50 points
18 comments11 min readLW link
(hivewired.wordpress.com)

An­nounc­ing the Align­ment of Com­plex Sys­tems Re­search Group

4 Jun 2022 4:10 UTC
83 points
18 comments5 min readLW link

The hor­ror of what must, yet can­not, be true

Kaj_Sotala2 Jun 2022 10:20 UTC
50 points
17 comments2 min readLW link
(kajsotala.fi)

Shard The­ory: An Overview

David Udell11 Aug 2022 5:44 UTC
141 points
34 comments10 min readLW link

Many ther­apy schools work with in­ner mul­ti­plic­ity (not just IFS)

17 Sep 2022 10:27 UTC
50 points
15 comments18 min readLW link

In­ter­nal com­mu­ni­ca­tion framework

15 Nov 2022 12:41 UTC
37 points
14 comments12 min readLW link

Slack mat­ters more than any outcome

Valentine31 Dec 2022 20:11 UTC
113 points
52 comments19 min readLW link

Re­marks 1–18 on GPT (com­pressed)

Cleo Nardo20 Mar 2023 22:27 UTC
115 points
23 comments31 min readLW link

Silence

alkjash18 Mar 2018 4:10 UTC
57 points
17 comments4 min readLW link
(radimentary.wordpress.com)

Be­ware So­cial Cop­ing Strategies

Lulie5 Feb 2018 4:48 UTC
48 points
24 comments7 min readLW link

Alien par­a­site tech­ni­cal guy

PhilGoetz27 Jul 2010 16:51 UTC
66 points
55 comments3 min readLW link

TDT for Humans

alkjash28 Feb 2018 5:40 UTC
25 points
7 comments5 min readLW link
(radimentary.wordpress.com)

The Soli­taire Prin­ci­ple: Game The­ory for One

alkjash17 Jan 2018 0:14 UTC
24 points
8 comments9 min readLW link
(radimentary.wordpress.com)

Self and No-Self

Vaniver29 Dec 2019 6:15 UTC
47 points
3 comments2 min readLW link

A Cau­tion­ary Note on Un­lock­ing the Emo­tional Brain

eapache8 Feb 2020 17:21 UTC
51 points
20 comments2 min readLW link

Which Parts Are “Me”?

Eliezer Yudkowsky22 Oct 2008 18:15 UTC
64 points
118 comments5 min readLW link

Make an ap­point­ment with your saner self

MalcolmOcean8 Feb 2019 5:05 UTC
28 points
0 comments4 min readLW link

Ad­di­tive and Mul­ti­plica­tive Subagents

Scott Garrabrant6 Nov 2020 14:26 UTC
20 points
7 comments12 min readLW link

Restricted Anti­na­tal­ism on Subagents

Josephine13 May 2021 1:48 UTC
4 points
1 comment2 min readLW link

Reflec­tion of Hier­ar­chi­cal Re­la­tion­ship via Nuanced Con­di­tion­ing of Game The­ory Ap­proach for AI Devel­op­ment and Utilization

Kyoung-cheol Kim4 Jun 2021 7:20 UTC
2 points
2 comments9 min readLW link

In­te­grat­ing Three Models of (Hu­man) Cognition

jbkjr23 Nov 2021 1:06 UTC
29 points
4 comments32 min readLW link

Prune

alkjash12 Jan 2018 22:50 UTC
61 points
10 comments4 min readLW link
(radimentary.wordpress.com)

Selec­tion pro­cesses for subagents

Ryan Kidd30 Jun 2022 23:57 UTC
34 points
2 comments9 min readLW link

Pro­saic mis­al­ign­ment from the Solomonoff Predictor

Cleo Nardo9 Dec 2022 17:53 UTC
35 points
2 comments5 min readLW link
No comments.