RSS

Co­or­di­na­tion /​ Cooperation

TagLast edit: 5 Aug 2020 12:40 UTC by Yoav Ravid

Coordination is the challenge of distinct actors being able to jointly choose their actions to achieve a favorable outcome. An example of easy coordination is coordinating to drive on the same side of the road. A hard example is coordinating everyone to move a superior yet different social media platform. Many failures of civilization are failures of coordination. A closely related concept is that of cooperation – multiple actors choosing their actions in ways that maximize collective value despite the temptation of greater short-term individual gain by acting to the detriment of the group/​other actors. The Prisoner’s Dilemma is the canonical cooperation/​defection game, but the term is used in other games too, e.g. Stag Hunt.

Coordination and cooperation are fundamental concepts in Game Theory. LessWrong discussion has long been interested in overcoming the gnarly coordination and cooperation challenges that prevent many improvements to optimizing the world. There is also interest because failures to coordinate/​cooperate are likely causes of existential risks such as AI Risk or nuclear war.

See also: Game Theory, Moloch

Another related term in this space is collective action problem.

Related Sequences: Kickstarter for Coordinated Action

Co­or­di­na­tion as a Scarce Resource

johnswentworth25 Jan 2020 23:32 UTC
178 points
17 comments4 min readLW link2 reviews

The Costly Co­or­di­na­tion Mechanism of Com­mon Knowledge

Ben Pace15 Mar 2018 20:20 UTC
166 points
30 comments19 min readLW link2 reviews

Two Co­or­di­na­tion Styles

abramdemski7 Feb 2018 9:00 UTC
38 points
14 comments7 min readLW link

“Win First” vs “Chill First”

lionhearted28 Sep 2020 6:48 UTC
86 points
19 comments3 min readLW link

Iter­ated Trust Kickstarters

Raemon20 Apr 2021 3:18 UTC
73 points
18 comments10 min readLW link

Most Pri­soner’s Dilem­mas are Stag Hunts; Most Stag Hunts are Schel­ling Problems

abramdemski14 Sep 2020 22:13 UTC
156 points
37 comments10 min readLW link3 reviews

Parable of the Dammed

johnswentworth10 Dec 2020 0:08 UTC
105 points
27 comments2 min readLW link

Your Cheer­ful Price

Eliezer Yudkowsky13 Feb 2021 5:41 UTC
227 points
76 comments17 min readLW link

Jean Mon­net: The Guerilla Bureaucrat

Martin Sustrik20 Mar 2021 10:37 UTC
166 points
23 comments18 min readLW link

The Co­or­di­na­tion Fron­tier: Se­quence Intro

Raemon4 Sep 2021 22:11 UTC
157 points
21 comments6 min readLW link

A Key Power of the Pres­i­dent is to Co­or­di­nate the Ex­e­cu­tion of Ex­ist­ing Con­crete Plans

Ben Pace16 Jul 2019 5:06 UTC
124 points
13 comments10 min readLW link

Co­or­di­na­tion Sur­veys: why we should sur­vey to or­ga­nize re­spon­si­bil­ities, not just predictions

Academian7 May 2019 17:43 UTC
68 points
12 comments3 min readLW link3 reviews

In­sights from ‘The Strat­egy of Con­flict’

DanielFilan4 Jan 2018 5:05 UTC
37 points
13 comments7 min readLW link

On De­stroy­ing the World

Chris_Leong28 Sep 2020 7:38 UTC
75 points
86 comments4 min readLW link

Co­or­di­na­tion Schemes Are Cap­i­tal Investments

Raemon6 Sep 2021 23:27 UTC
135 points
28 comments7 min readLW link

Co­or­di­na­tion Skills I Wish I Had For the Pandemic

Raemon13 Nov 2021 23:32 UTC
75 points
6 comments6 min readLW link

Why Our Kind Can’t Cooperate

Eliezer Yudkowsky20 Mar 2009 8:37 UTC
243 points
205 comments9 min readLW link

Co­op­er­at­ing with agents with differ­ent ideas of fair­ness, while re­sist­ing exploitation

Eliezer Yudkowsky16 Sep 2013 8:27 UTC
80 points
45 comments4 min readLW link

Si­mu­lacrum 3 As Stag-Hunt Strategy

johnswentworth26 Jan 2021 19:40 UTC
157 points
30 comments4 min readLW link

What Mul­tipo­lar Failure Looks Like, and Ro­bust Agent-Ag­nos­tic Pro­cesses (RAAPs)

Andrew_Critch31 Mar 2021 23:50 UTC
179 points
59 comments22 min readLW link

A tax­on­omy of co­or­di­na­tion problems

damiensnyder22 Aug 2021 23:10 UTC
29 points
1 comment6 min readLW link

Norm In­no­va­tion and The­ory of Mind

Raemon18 Sep 2021 21:38 UTC
70 points
13 comments7 min readLW link

Con­cen­tra­tion of Force

Duncan_Sabien6 Nov 2021 8:20 UTC
198 points
21 comments12 min readLW link

[Question] Does it be­come eas­ier, or harder, for the world to co­or­di­nate around not build­ing AGI as time goes on?

Eli Tyre29 Jul 2019 22:59 UTC
86 points
31 comments3 min readLW link2 reviews

Tac­ti­cal vs. Strate­gic Cooperation

sarahconstantin12 Aug 2018 16:41 UTC
73 points
18 comments5 min readLW link

Ro­bust Co­op­er­a­tion in the Pri­soner’s Dilemma

orthonormal7 Jun 2013 8:30 UTC
115 points
146 comments7 min readLW link

Black­mail, Nukes and the Pri­soner’s Dilemma

Stuart_Armstrong10 Mar 2010 14:58 UTC
25 points
20 comments2 min readLW link

What counts as defec­tion?

TurnTrout12 Jul 2020 22:03 UTC
81 points
21 comments5 min readLW link1 review

Sec­tions 1 & 2: In­tro­duc­tion, Strat­egy and Governance

JesseClifton17 Dec 2019 21:27 UTC
34 points
5 comments14 min readLW link

Sec­tions 3 & 4: Cred­i­bil­ity, Peace­ful Bar­gain­ing Mechanisms

JesseClifton17 Dec 2019 21:46 UTC
19 points
2 comments12 min readLW link

“You can’t pos­si­bly suc­ceed with­out [My Pet Is­sue]”

Raemon19 Dec 2019 1:12 UTC
50 points
14 comments1 min readLW link

Shap­ing eco­nomic in­cen­tives for col­lab­o­ra­tive AGI

Kaj_Sotala29 Jun 2018 16:26 UTC
45 points
15 comments4 min readLW link

Mul­ti­verse-wide Co­op­er­a­tion via Cor­re­lated De­ci­sion Making

Kaj_Sotala20 Aug 2017 12:01 UTC
6 points
2 comments1 min readLW link
(foundational-research.org)

From Per­sonal to Pri­son Gangs: En­forc­ing Proso­cial Behavior

johnswentworth24 Jan 2019 18:07 UTC
131 points
24 comments5 min readLW link2 reviews

Win­ning is for Losers

Jacob Falkovich11 Oct 2017 4:01 UTC
31 points
15 comments18 min readLW link
(putanumonit.com)

Bayesi­ans vs. Barbarians

Eliezer Yudkowsky14 Apr 2009 23:45 UTC
89 points
285 comments8 min readLW link

The Im­por­tance of Sidekicks

Swimmer9638 Jan 2015 23:21 UTC
216 points
209 comments5 min readLW link

Team Co­he­sion and Ex­clu­sion­ary Egalitarianism

lionhearted17 Sep 2018 4:48 UTC
31 points
6 comments7 min readLW link

Healthy Competition

Raemon20 Oct 2019 20:55 UTC
54 points
13 comments2 min readLW link1 review

Reneg­ing proso­cially by Dun­can Sabien

philip_b18 Jun 2019 18:52 UTC
70 points
5 comments1 min readLW link
(medium.com)

[Question] How could “Kick­starter for Inad­e­quate Equil­ibria” be used for evil or turn out to be net-nega­tive?

Raemon21 Feb 2019 21:36 UTC
25 points
17 comments1 min readLW link

Schel­ling Reach

Ziz7 Jan 2018 18:18 UTC
9 points
3 comments1 min readLW link
(sinceriously.fyi)

Real-World Co­or­di­na­tion Prob­lems are Usu­ally In­for­ma­tion Problems

johnswentworth13 Jun 2019 18:21 UTC
33 points
2 comments4 min readLW link

Vanilla and choco­late and prefer­ence judgements

Swimmer96318 Apr 2011 22:14 UTC
39 points
32 comments6 min readLW link

The Align­ment-Com­pe­tence Trade-Off, Part 1: Coal­i­tion Size and Sig­nal­ing Costs

Gentzel15 Jan 2020 23:10 UTC
30 points
4 comments3 min readLW link
(theconsequentialist.wordpress.com)

Col­lAc­tion his­tory and les­sons learned

Ron17 Jan 2021 15:01 UTC
34 points
6 comments8 min readLW link

Google’s Eth­i­cal AI team and AI Safety

magfrump20 Feb 2021 9:42 UTC
12 points
16 comments7 min readLW link

Partnership

Ruby11 Mar 2021 17:17 UTC
17 points
0 comments3 min readLW link

An Ex­er­cise in Ra­tional Co­op­er­a­tion and Com­mu­ni­ca­tion: Let’s Play Hanabi

ejacob14 Apr 2021 8:36 UTC
14 points
10 comments7 min readLW link

April 15, 2040

Nisan4 May 2021 21:18 UTC
96 points
19 comments2 min readLW link

[Book Re­view] Blueprint for Revolution

lsusr7 Jun 2021 18:14 UTC
28 points
8 comments2 min readLW link

My Mar­riage Vows

Vanessa Kosoy21 Jul 2021 10:48 UTC
79 points
58 comments3 min readLW link

A Bet­ter Web is Coming

matto21 Aug 2021 20:46 UTC
30 points
18 comments6 min readLW link

AISC5 Ret­ro­spec­tive: Mechanisms for Avoid­ing Tragedy of the Com­mons in Com­mon Pool Re­source Problems

27 Sep 2021 16:46 UTC
8 points
3 comments7 min readLW link

The role of tribes in achiev­ing last­ing im­pact and how to cre­ate them

29 Sep 2021 20:48 UTC
14 points
3 comments24 min readLW link

Post­mod­ern Warfare

lsusr25 Oct 2021 9:02 UTC
61 points
25 comments2 min readLW link

First Strike and Se­cond Strike

lsusr25 Nov 2021 9:23 UTC
19 points
5 comments1 min readLW link

Co­or­di­nat­ing the Unequal Treaties

lsusr25 Nov 2021 10:47 UTC
28 points
4 comments2 min readLW link

Pareto im­prove­ments are rarer than they seem

philh27 Jan 2018 22:23 UTC
28 points
14 comments3 min readLW link
(reasonableapproximation.net)

My Overview of the AI Align­ment Land­scape: A Bird’s Eye View

Neel Nanda15 Dec 2021 23:44 UTC
104 points
9 comments16 min readLW link

My Overview of the AI Align­ment Land­scape: Threat Models

Neel Nanda25 Dec 2021 23:07 UTC
38 points
4 comments28 min readLW link

Create a Full Alter­na­tive Stack

Zvi31 Jan 2020 17:10 UTC
74 points
14 comments6 min readLW link1 review
(thezvi.wordpress.com)

Nar­ra­tive Syncing

AnnaSalamon1 May 2022 1:48 UTC
116 points
45 comments7 min readLW link

An­nounc­ing the Align­ment of Com­plex Sys­tems Re­search Group

4 Jun 2022 4:10 UTC
65 points
17 comments5 min readLW link

Inad­e­quate Equil­ibria vs. Gover­nance of the Commons

Martin Sustrik25 May 2018 13:17 UTC
178 points
18 comments14 min readLW link2 reviews

Melt­ing Gold, and Or­ga­ni­za­tional Capacity

Raemon11 Dec 2017 19:16 UTC
68 points
23 comments5 min readLW link

Ideas for an ac­tion co­or­di­na­tion website

Yoav Ravid8 Mar 2019 13:32 UTC
17 points
36 comments5 min readLW link

Norms of Mem­ber­ship for Vol­un­tary Groups

sarahconstantin11 Dec 2018 22:10 UTC
188 points
10 comments7 min readLW link
(srconstantin.wordpress.com)

What is op­er­a­tions?

Swimmer96326 Sep 2019 14:16 UTC
108 points
9 comments7 min readLW link

Defect­ing by Ac­ci­dent—A Flaw Com­mon to An­a­lyt­i­cal People

lionhearted1 Dec 2010 8:25 UTC
115 points
432 comments15 min readLW link

Les­sons from the Cold War on In­for­ma­tion Hazards: Why In­ter­nal Com­mu­ni­ca­tion is Critical

Gentzel24 Feb 2018 23:34 UTC
47 points
10 comments4 min readLW link

Tol­er­ate Tolerance

Eliezer Yudkowsky21 Mar 2009 7:34 UTC
111 points
93 comments2 min readLW link

Lead­ers of Men

Zvi29 Oct 2017 18:10 UTC
43 points
16 comments2 min readLW link
(thezvi.wordpress.com)

The Dar­win Results

Zvi25 Nov 2017 13:30 UTC
47 points
10 comments5 min readLW link
(thezvi.wordpress.com)

I Want To Live In A Baugruppe

Alicorn17 Mar 2017 1:36 UTC
100 points
195 comments3 min readLW link

The Dar­win Pregame

Zvi21 Nov 2017 1:10 UTC
34 points
6 comments5 min readLW link
(thezvi.wordpress.com)

Buri­dan’s ass in co­or­di­na­tion games

jessicata16 Jul 2018 2:51 UTC
52 points
26 comments10 min readLW link

Your Price for Joining

Eliezer Yudkowsky26 Mar 2009 7:16 UTC
67 points
58 comments4 min readLW link

Re­duc­ing col­lec­tive ra­tio­nal­ity to in­di­vi­d­ual op­ti­miza­tion in com­mon-pay­off games us­ing MCMC

jessicata20 Aug 2018 0:51 UTC
59 points
12 comments7 min readLW link

Coun­ter­fac­tu­als about So­cial Media

Zvi22 Apr 2019 12:20 UTC
52 points
23 comments6 min readLW link
(thezvi.wordpress.com)

Some Ways Co­or­di­na­tion is Hard

Zvi13 Jun 2019 13:00 UTC
53 points
13 comments10 min readLW link1 review
(thezvi.wordpress.com)

Can you elimi­nate memetic scarcity, in­stead of fight­ing?

Raemon25 Nov 2019 2:07 UTC
65 points
11 comments4 min readLW link

[Question] If a “Kick­starter for Inad­e­quate Equlibria” was built, do you have a con­crete in­ad­e­quate equil­ibrium to fix?

Raemon21 Feb 2019 21:32 UTC
55 points
40 comments1 min readLW link

So­phie Grouchy on A Build-Break Model of Cooperation

weft25 May 2018 14:55 UTC
23 points
6 comments8 min readLW link
(sophiegrouchy.wordpress.com)

Prop­erty as Co­or­di­na­tion Minimization

Vaniver4 Aug 2020 19:24 UTC
38 points
35 comments4 min readLW link

Let’s split the cake, length­wise, up­wise and slantwise

Stuart_Armstrong25 Oct 2010 13:15 UTC
74 points
29 comments4 min readLW link

[Question] Where are peo­ple think­ing and talk­ing about global co­or­di­na­tion for AI safety?

Wei_Dai22 May 2019 6:24 UTC
98 points
22 comments1 min readLW link

How hard is it for al­tru­ists to dis­cuss go­ing against bad equil­ibria?

abramdemski22 Jun 2019 3:42 UTC
45 points
6 comments11 min readLW link

AI co­op­er­a­tion is already stud­ied in academia as “pro­gram equil­ibrium”

cousin_it30 Jul 2012 15:22 UTC
60 points
44 comments1 min readLW link

Tomb­stone Men­tal­ity

SuperDaveWhite13 Aug 2020 13:46 UTC
6 points
0 comments2 min readLW link

Self-Similar­ity Experiment

Denis Drescher15 Aug 2020 13:19 UTC
12 points
0 comments10 min readLW link

Alle­gory On AI Risk, Game The­ory, and Mithril

James_Miller13 Feb 2017 20:41 UTC
45 points
57 comments3 min readLW link

One Web­site To Rule Them All?

anna_macdonald11 Jan 2019 19:14 UTC
30 points
23 comments10 min readLW link

Col­lec­tive Apa­thy and the Internet

Eliezer Yudkowsky14 Apr 2009 0:02 UTC
41 points
34 comments2 min readLW link

Break­ing Or­a­cles: su­per­ra­tional­ity and acausal trade

Stuart_Armstrong25 Nov 2019 10:40 UTC
25 points
15 comments1 min readLW link

[Question] De­gree of du­pli­ca­tion and co­or­di­na­tion in pro­jects that ex­am­ine com­put­ing prices, AI progress, and re­lated top­ics?

riceissa23 Apr 2019 12:27 UTC
26 points
1 comment2 min readLW link

Su­per­in­tel­li­gence 28: Collaboration

KatjaGrace24 Mar 2015 1:29 UTC
13 points
21 comments6 min readLW link

[Question] Where do (did?) sta­ble, co­op­er­a­tive in­sti­tu­tions come from?

AnnaSalamon3 Nov 2020 22:14 UTC
146 points
74 comments4 min readLW link

Co­op­er­a­tion is for Winners

Jacob Falkovich15 Feb 2019 14:58 UTC
21 points
6 comments4 min readLW link

Spaghetti Towers

eukaryote22 Dec 2018 5:29 UTC
142 points
24 comments3 min readLW link1 review
(eukaryotewritesblog.com)

Ask­ing For Help

Neel Nanda27 Dec 2020 11:32 UTC
23 points
5 comments12 min readLW link
(neelnanda.io)

Prac­ti­cal post on 4 #ProRep meth­ods (Pro­por­tional Rep­re­sen­ta­tion)

Jameson Quinn9 Jan 2021 14:56 UTC
35 points
12 comments11 min readLW link

Creat­ing A Kick­starter for Co­or­di­nated Action

Lumpyproletariat3 Feb 2021 4:57 UTC
12 points
11 comments9 min readLW link

Notes on Schel­ling’s “Strat­egy of Con­flict” (1960)

MichaelA10 Feb 2021 2:48 UTC
22 points
9 comments8 min readLW link

The Pro­to­typ­i­cal Ne­go­ti­a­tion Game

johnswentworth20 Feb 2021 21:33 UTC
81 points
13 comments5 min readLW link

I’m from a par­allel Earth with much higher co­or­di­na­tion: AMA

5 Apr 2021 22:09 UTC
134 points
25 comments61 min readLW link

Re­view and Com­par­i­son: Onto-Car­tog­ra­phy & Promise Theory

Josephine9 May 2021 0:42 UTC
13 points
0 comments5 min readLW link

Con­trol­ling In­tel­li­gent Agents The Only Way We Know How: Ideal Bureau­cratic Struc­ture (IBS)

Justin Bullock24 May 2021 12:53 UTC
10 points
11 comments6 min readLW link

For­mal­iz­ing Ob­jec­tions against Sur­ro­gate Goals

VojtaKovarik2 Sep 2021 16:24 UTC
7 points
22 comments20 min readLW link

Notes on Courtesy

David_Gross16 Dec 2020 4:08 UTC
10 points
0 comments6 min readLW link

Notes on Loyalty

David_Gross15 Nov 2020 19:30 UTC
17 points
2 comments8 min readLW link

AI Pre­dic­tion Ser­vices and Risks of War

isaduan3 Oct 2021 10:26 UTC
3 points
2 comments10 min readLW link

The Coloniza­tion of Cults, Non­profit Or­ga­ni­za­tions, and Society

HS202117 Oct 2021 22:02 UTC
−7 points
2 comments6 min readLW link

Co­or­di­na­tion Mo­ti­va­tion: The Pandemic

Raemon21 Oct 2021 21:57 UTC
61 points
1 comment3 min readLW link

Y2K: Suc­cess­ful Prac­tice for AI Alignment

Darmani5 Nov 2021 6:09 UTC
47 points
5 comments6 min readLW link

Speak­ing of Stag Hunts

Duncan_Sabien6 Nov 2021 8:20 UTC
186 points
376 comments18 min readLW link

[Question] Good books about over­com­ing co­or­di­na­tion prob­lems?

Sean Aubin2 Feb 2022 19:09 UTC
22 points
14 comments1 min readLW link

Notes on So­cial Responsibility

David_Gross19 Mar 2022 14:44 UTC
15 points
4 comments22 min readLW link

Dath Ilan vs. Sid Meier’s Alpha Cen­tauri: Pareto Improvements

David Udell28 Apr 2022 19:26 UTC
31 points
16 comments2 min readLW link

Machines vs Memes Part 3: Imi­ta­tion and Memes

ceru231 Jun 2022 13:36 UTC
5 points
0 comments7 min readLW link

New co­op­er­a­tion mechanism—quadratic fund­ing with­out a match­ing pool

Filip Sondej5 Jun 2022 13:55 UTC
5 points
0 comments5 min readLW link

[AN #150]: The sub­types of Co­op­er­a­tive AI research

Rohin Shah12 May 2021 17:20 UTC
15 points
0 comments6 min readLW link
(mailchi.mp)

CLR’s re­cent work on multi-agent systems

JesseClifton9 Mar 2021 2:28 UTC
51 points
1 comment13 min readLW link

Worked Ex­am­ples of Shap­ley Values

lalaithion24 Jun 2022 17:13 UTC
51 points
8 comments8 min readLW link

Quick Sum­maries of Two Papers on Kant and Game Theory

Erich_Grunewald25 Jun 2022 10:25 UTC
8 points
2 comments4 min readLW link
(www.erichgrunewald.com)

Abadar­ian Trades

David Udell30 Jun 2022 16:41 UTC
5 points
0 comments2 min readLW link
No comments.