Polymath-style attack on the Parliamentary Model for moral uncertainty

Thanks to ESrogs, Ste­fan_Schu­bert, and the Effec­tive Altru­ism sum­mit for the dis­cus­sion that led to this post!

This post is to test out Poly­math-style col­lab­o­ra­tion on LW. The prob­lem we’ve cho­sen to try is for­mal­iz­ing and an­a­lyz­ing Bostrom and Ord’s “Par­li­a­men­tary Model” for deal­ing with moral un­cer­tainty.

I’ll first re­view the Par­li­a­men­tary Model, then give some of Poly­math’s style sug­ges­tions, and fi­nally sug­gest some di­rec­tions that the con­ver­sa­tion could take.

The Par­li­a­men­tary Model

The Par­li­a­men­tary Model is an un­der-speci­fied method of deal­ing with moral un­cer­tainty, pro­posed in 2009 by Nick Bostrom and Toby Ord. Re­post­ing Nick’s sum­mary from Over­com­ing Bias:

Sup­pose that you have a set of mu­tu­ally ex­clu­sive moral the­o­ries, and that you as­sign each of these some prob­a­bil­ity. Now imag­ine that each of these the­o­ries gets to send some num­ber of del­e­gates to The Par­li­a­ment. The num­ber of del­e­gates each the­ory gets to send is pro­por­tional to the prob­a­bil­ity of the the­ory. Then the del­e­gates bar­gain with one an­other for sup­port on var­i­ous is­sues; and the Par­li­a­ment reaches a de­ci­sion by the del­e­gates vot­ing. What you should do is act ac­cord­ing to the de­ci­sions of this imag­i­nary Par­li­a­ment. (Ac­tu­ally, we use an ex­tra trick here: we imag­ine that the del­e­gates act as if the Par­li­a­ment’s de­ci­sion were a stochas­tic vari­able such that the prob­a­bil­ity of the Par­li­a­ment tak­ing ac­tion A is pro­por­tional to the frac­tion of votes for A. This has the effect of elimi­nat­ing the ar­tifi­cial 50% thresh­old that oth­er­wise gives a ma­jor­ity bloc ab­solute power. Yet – un­be­knownst to the del­e­gates – the Par­li­a­ment always takes what­ever ac­tion got the most votes: this way we avoid pay­ing the cost of the ran­dom­iza­tion!)

The idea here is that moral the­o­ries get more in­fluence the more prob­a­ble they are; yet even a rel­a­tively weak the­ory can still get its way on some is­sues that the the­ory think are ex­tremely im­por­tant by sac­ri­fic­ing its in­fluence on other is­sues that other the­o­ries deem more im­por­tant. For ex­am­ple, sup­pose you as­sign 10% prob­a­bil­ity to to­tal util­i­tar­i­anism and 90% to moral ego­ism (just to illus­trate the prin­ci­ple). Then the Par­li­a­ment would mostly take ac­tions that max­i­mize ego­is­tic satis­fac­tion; how­ever it would make some con­ces­sions to util­i­tar­i­anism on is­sues that util­i­tar­i­anism thinks is es­pe­cially im­por­tant. In this ex­am­ple, the per­son might donate some por­tion of their in­come to ex­is­ten­tial risks re­search and oth­er­wise live com­pletely self­ishly.

I think there might be wis­dom in this model. It avoids the dan­ger­ous and un­sta­ble ex­trem­ism that would re­sult from let­ting one’s cur­rent fa­vorite moral the­ory com­pletely dic­tate ac­tion, while still al­low­ing the ag­gres­sive pur­suit of some non-com­mon­sen­si­cal high-lev­er­age strate­gies so long as they don’t in­fringe too much on what other ma­jor moral the­o­ries deem cen­trally im­por­tant.

In a com­ment, Bostrom con­tinues:

there are a num­ber of known is­sues with var­i­ous vot­ing sys­tems, and this is the rea­son I say our model is im­pre­cise and un­der-de­ter­mined. But we have some quite sub­stan­tial in­tu­itions and in­sights into how ac­tual par­li­a­ments work so it is not a com­plete black box. For ex­am­ple, we can see that, other things equal, views that have more del­e­gates tend to ex­ert greater in­fluence on the out­come, etc. There are some fea­tures of ac­tual par­li­a­ments that we want to pos­tu­late away. The fake ran­dom­iza­tion step is one pos­tu­late. We also think we want to stipu­late that the imag­i­nary par­li­a­men­tar­i­ans should not en­gage in black­mail etc. but we don’t have a full speci­fi­ca­tion of this. Also, we have not defined the rule by which the agenda is set. So it is far from a com­plete for­mal model.

It’s an in­ter­est­ing idea, but clearly there are a lot of de­tails to work out. Can we for­mally spec­ify the kinds of ne­go­ti­a­tion that del­e­gates can en­gage in? What about black­mail or pris­on­ers’ dilem­mas be­tween del­e­gates? It what ways does this pro­posed method out­perform other ways of deal­ing with moral un­cer­tainty?

I was dis­cussing this with ESRogs and Ste­fan_Schu­bert at the Effec­tive Altru­ism sum­mit, and we thought it might be fun to throw the ques­tion open to LessWrong. In par­tic­u­lar, we thought it’d be a good test prob­lem for a Poly­math-pro­ject-style ap­proach.

How to Polymath

The Poly­math com­ment style sug­ges­tions are not so differ­ent from LW’s, but num­bers 5 and 6 are par­tic­u­larly im­por­tant. In essence, they point out that the idea of a Poly­math pro­ject is to split up the work into min­i­mal chunks among par­ti­ci­pants, and to get most of the think­ing to oc­cur in com­ment threads. This is as op­posed to a pro­cess in which one com­mu­nity mem­ber goes off for a week, med­i­tates deeply on the prob­lem, and pro­duces a com­plete solu­tion by them­selves. Poly­math rules 5 and 6 are in­struc­tive:

5. If you are plan­ning to think about some as­pect of the prob­lem offline for an ex­tended length of time, let the rest of us know. A poly­math pro­ject is sup­posed to be more than the sum of its in­di­vi­d­ual con­trib­u­tors; the in­sights that you have are sup­posed to be shared amongst all of us, not kept in iso­la­tion un­til you have re­solved all the difficul­ties by your­self. It will un­doubt­edly be the case, es­pe­cially in the later stages of a poly­math pro­ject, that the best way to achieve progress is for one of the par­ti­ci­pants to do some deep thought or ex­ten­sive com­pu­ta­tion away from the blog, but to keep in the spirit of the poly­math pro­ject, it would be good if you could let us know that you are do­ing this, and to up­date us on what­ever progress you make (or fail to make). It may well be that an­other par­ti­ci­pant may have a sug­ges­tion that could save you some effort.

6. An ideal poly­math re­search com­ment should rep­re­sent a “quan­tum of progress”. On the one hand, it should con­tain a non-triv­ial new in­sight (which can in­clude nega­tive in­sights, such as point­ing out that a par­tic­u­lar ap­proach to the prob­lem has some spe­cific difficulty), but on the other hand it should not be a com­plex piece of math­e­mat­ics that the other par­ti­ci­pants will have trou­ble ab­sorb­ing. (This prin­ci­ple un­der­lies many of the pre­ced­ing guidelines.) Ba­si­cally, once your thought pro­cesses reach a point where one could effi­ciently hand the ba­ton on to an­other par­ti­ci­pant, that would be a good time to de­scribe what you’ve just re­al­ised on the blog.

It seems to us as well that an im­por­tant part of the Poly­math style is to have fun to­gether and to use the prin­ci­ple of char­ity liber­ally, so as to cre­ate a space in which peo­ple can safely be wrong, point out flaws, and build up a bet­ter pic­ture to­gether.

Our test project

If you’re still read­ing, then I hope you’re in­ter­ested in giv­ing this a try. The over­all goal is to clar­ify and for­mal­ize the Par­li­a­men­tary Model, and to an­a­lyze its strengths and weak­nesses rel­a­tive to other ways of deal­ing with moral un­cer­tainty. Here are the three most promis­ing ques­tions we came up with:

  1. What prop­er­ties would be de­sir­able for the model to have (e.g. Pareto effi­ciency)?

  2. What should the ex­act mechanism for ne­go­ti­a­tion among del­e­gates?

  3. Are there other mod­els that are prov­ably dom­i­nated by some nice for­mal­iza­tion of the Par­li­a­men­tary Model?

The origi­nal OB post had a cou­ple of com­ments that I thought were worth re­pro­duc­ing here, in case they spark dis­cus­sion, so I’ve posted them.

Fi­nally, if you have meta-level com­ments on the pro­ject as a whole in­stead of Poly­math-style com­ments that aim to clar­ify or solve the prob­lem, please re­ply in the meta-com­ments thread.