Schelling fences on slippery slopes

Slip­pery slopes are them­selves a slip­pery con­cept. Imag­ine try­ing to ex­plain them to an alien:

“Well, we right-think­ing peo­ple are quite sure that the Holo­caust hap­pened, so ban­ning Holo­caust de­nial would shut up some crack­pots and im­prove the dis­course. But it’s one step on the road to things like ban­ning un­pop­u­lar poli­ti­cal po­si­tions or re­li­gions, and we right-think­ing peo­ple op­pose that, so we won’t ban Holo­caust de­nial.“

And the alien might well re­spond: “But you could just ban Holo­caust de­nial, but not ban un­pop­u­lar poli­ti­cal po­si­tions or re­li­gions. Then you right-think­ing peo­ple get the thing you want, but not the thing you don’t want.“

This post is about some of the replies you might give the alien.

Aban­don­ing the Power of Choice

This is the bor­ing one with­out any philo­soph­i­cal in­sight that gets men­tioned only for com­plete­ness’ sake. In this re­ply, giv­ing up a cer­tain point risks los­ing the abil­ity to de­cide whether or not to give up other points.

For ex­am­ple, if peo­ple gave up the right to pri­vacy and al­lowed the gov­ern­ment to mon­i­tor all phone calls, on­line com­mu­ni­ca­tions, and pub­lic places, then if some­one launched a mil­i­tary coup, it would be very difficult to re­sist them be­cause there would be no way to se­cretly or­ga­nize a re­bel­lion. This is also brought up in ar­gu­ments about gun con­trol a lot.

I’m not sure this is prop­erly thought of as a slip­pery slope ar­gu­ment at all. It seems to be a more straight­for­ward “Don’t give up use­ful tools for fight­ing tyranny” ar­gu­ment.

The Le­gend of Mur­der-Gand­hi

Pre­vi­ously on Less Wrong’s The Ad­ven­tures of Mur­der-Gandhi: Gandhi is offered a pill that will turn him into an un­stop­pable mur­derer. He re­fuses to take it, be­cause in his cur­rent in­car­na­tion as a paci­fist, he doesn’t want oth­ers to die, and he knows that would be a con­se­quence of tak­ing the pill. Even if we offered him $1 mil­lion to take the pill, his ab­hor­rence of vi­o­lence would lead him to re­fuse.

But sup­pose we offered Gandhi $1 mil­lion to take a differ­ent pill: one which would de­crease his re­luc­tance to mur­der by 1%. This sounds like a pretty good deal. Even a per­son with 1% less re­luc­tance to mur­der than Gandhi is still pretty paci­fist and not likely to go kil­ling any­body. And he could donate the money to his fa­vorite char­ity and per­haps save some lives. Gandhi ac­cepts the offer.

Now we iter­ate the pro­cess: ev­ery time Gandhi takes the 1%-more-likely-to-mur­der-pill, we offer him an­other $1 mil­lion to take the same pill again.

Maybe origi­nal Gandhi, upon sober con­tem­pla­tion, would de­cide to ac­cept $5 mil­lion to be­come 5% less re­luc­tant to mur­der. Maybe 95% of his origi­nal paci­fism is the only level at which he can be ab­solutely sure that he will still pur­sue his paci­fist ideals.

Un­for­tu­nately, origi­nal Gandhi isn’t the one mak­ing the choice of whether or not to take the 6th pill. 95%-Gandhi is. And 95% Gandhi doesn’t care quite as much about paci­fism as origi­nal Gandhi did. He still doesn’t want to be­come a mur­derer, but it wouldn’t be a dis­aster if he were just 90% as re­luc­tant as origi­nal Gandhi, that stuck-up goody-goody.

What if there were a gen­eral prin­ci­ple that each Gandhi was com­fortable with Gand­his 5% more mur­der­ous than him­self, but no more? Origi­nal Gandhi would start tak­ing the pills, hop­ing to get down to 95%, but 95%-Gandhi would start tak­ing five more, hop­ing to get down to 90%, and so on un­til he’s ram­pag­ing through the streets of Delhi, kil­ling ev­ery­thing in sight.

Now we’re tempted to say Gandhi shouldn’t even take the first pill. But this also seems odd. Are we re­ally say­ing Gandhi shouldn’t take what’s ba­si­cally a free mil­lion dol­lars to turn him­self into 99%-Gandhi, who might well be nearly in­dis­t­in­guish­able in his ac­tions from the origi­nal?

Maybe Gandhi’s best op­tion is to “fence off” an area of the slip­pery slope by es­tab­lish­ing a Schel­ling point—an ar­bi­trary point that takes on spe­cial value as a di­vid­ing line. If he can hold him­self to the pre­com­mit­ment, he can max­i­mize his win­nings. For ex­am­ple, origi­nal Gandhi could swear a mighty oath to take only five pills—or if he didn’t trust even his own leg­endary virtue, he could give all his most valuable pos­ses­sions to a friend and tell the friend to de­stroy them if he took more than five pills. This would com­mit his fu­ture self to stick to the 95% bound­ary (even though that fu­ture self is itch­ing to try to the same pre­com­mit­ment strat­egy to stick to its own 90% bound­ary).

Real slip­pery slopes will re­sem­ble this ex­am­ple if, each time we change the rules, we also end up chang­ing our opinion about how the rules should be changed. For ex­am­ple, I think the Catholic Church may be work­ing off a the­ory of “If we give up this tra­di­tional prac­tice, peo­ple will lose re­spect for tra­di­tion and want to give up even more tra­di­tional prac­tices, and so on.“

Slip­pery Hyper­bolic Dis­count­ing

One evening, I start play­ing Sid Meier’s Civ­i­liza­tion (IV, if you’re won­der­ing—V is ter­rible). I have work to­mor­row, so I want to stop and go to sleep by mid­night.

At mid­night, I con­sider my al­ter­na­tives. For the mo­ment, I feel an urge to keep play­ing Civ­i­liza­tion. But I know I’ll be mis­er­able to­mor­row if I haven’t got­ten enough sleep. Be­ing a hy­per­bolic dis­counter, I value the next ten min­utes a lot, but af­ter that the curve be­comes pretty flat and maybe I don’t value 12:20 much more than I value the next morn­ing at work. Ten min­utes’ sleep here or there doesn’t make any differ­ence. So I say: “I will play Civ­i­liza­tion for ten min­utes - ‘just one more turn’ - and then I will go to bed.“

Time passes. It is now 12:10. Still be­ing a hy­per­bolic dis­counter, I value the next ten min­utes a lot, and sub­se­quent times much less. And so I say: I will play un­til 12:20, ten min­utes sleep here or there not mak­ing much differ­ence, and then sleep.

And so on un­til my em­pire be­strides the globe and the ris­ing sun peeps through my win­dows.

This is pretty much the same pro­cess de­scribed above with Mur­der-Gandhi ex­cept that here the role of the value-chang­ing pill is played by time and my own ten­dency to dis­count hy­per­bol­i­cally.

The solu­tion is the same. If I con­sider the prob­lem early in the evening, I can pre­com­mit to mid­night as a nice round num­ber that makes a good Schel­ling point. Then, when de­cid­ing whether or not to play af­ter mid­night, I can treat my de­ci­sion not as “Mid­night or 12:10”—be­cause 12:10 will always win that par­tic­u­lar race—but as “Mid­night or aban­don­ing the only cred­ible Schel­ling point and prob­a­bly play­ing all night”, which will be suffi­cient to scare me into turn­ing off the com­puter.

(if I con­sider the prob­lem at 12:01, I may be able to pre­com­mit to 12:10 if I am es­pe­cially good at pre­com­mit­ments, but it’s not a very nat­u­ral Schel­ling point and it might be eas­ier to say some­thing like “as soon as I finish this turn” or “as soon as I dis­cover this tech­nol­ogy”).

Coal­i­tions of Re­sis­tance

Sup­pose you are a Zoroas­trian, along with 1% of the pop­u­la­tion. In fact, along with Zoroas­tri­anism your coun­try has fifty other small re­li­gions, each with 1% of the pop­u­la­tion. 49% of your coun­try­men are athe­ist, and hate re­li­gion with a pas­sion.

You hear that the gov­ern­ment is con­sid­er­ing ban­ning the Taoists, who com­prise 1% of the pop­u­la­tion. You’ve never liked the Taoists, vile doubters of the light of Ahura Mazda that they are, so you go along with this. When you hear the gov­ern­ment wants to ban the Sikhs and Jains, you take the same tack.

But now you are in the un­for­tu­nate situ­a­tion de­scribed by Martin Nie­mol­ler:

First they came for the so­cial­ists, and I did not speak out, be­cause I was not a so­cial­ist.
Then they came for the trade union­ists, and I did not speak out, be­cause I was not a trade union­ist.
Then they came for the Jews, and I did not speak out, be­cause I was not a Jew.
Then they came for me, but we had already aban­doned the only defen­si­ble Schel­ling point

With the banned Taoists, Sikhs, and Jains no longer in­vested in the out­come, the 49% athe­ist pop­u­la­tion has enough clout to ban Zoroas­tri­anism and any­one else they want to ban. The bet­ter strat­egy would have been to have all fifty-one small re­li­gions form a coal­i­tion to defend one an­other’s right to ex­ist. In this toy model, they could have done so in an ec­u­me­nial congress, or some other literal strat­egy meet­ing.

But in the real world, there aren’t fifty-one well-delineated re­li­gions. There are billions of peo­ple, each with their own set of opinions to defend. It would be im­prac­ti­cal for ev­ery­one to phys­i­cally co­or­di­nate, so they have to rely on Schel­ling points.

In the origi­nal ex­am­ple with the alien, I cheated by us­ing the phrase “right-think­ing peo­ple”. In re­al­ity, figur­ing out who qual­ifies to join the Right-Think­ing Peo­ple Club is half the bat­tle, and ev­ery­one’s likely to have a differ­ent opinion on it. So far, the prac­ti­cal solu­tion to the co­or­di­na­tion prob­lem, the “only defen­si­ble Schel­ling point”, has been to just have ev­ery­one agree to defend ev­ery­one else with­out wor­ry­ing whether they’re right-think­ing or not, and this is eas­ier than try­ing to co­or­di­nate room for ex­cep­tions like Holo­caust de­niers. Give up on the Holo­caust de­niers, and no one else can be sure what other Schel­ling point you’ve com­mit­ted to, if any...

...un­less they can. In parts of Europe, they’ve banned Holo­caust de­nial for years and ev­ery­one’s been to­tally okay with it. There are also a host of other well-re­spected ex­cep­tions to free speech, like shout­ing “fire” in a crowded the­ater. Pre­sum­ably, these ex­emp­tions are pro­tected by tra­di­tion, so that they have be­come new Schel­ling points there, or are else so ob­vi­ous that ev­ery­one ex­cept Holo­caust de­niers is will­ing to al­low a spe­cial Holo­caust de­nial ex­cep­tion with­out wor­ry­ing it will im­pact their own case.


Slip­pery slopes le­gi­t­i­mately ex­ist wher­ever a policy not only af­fects the world di­rectly, but af­fects peo­ple’s will­ing­ness or abil­ity to op­pose fu­ture poli­cies. Slip­pery slopes can some­times be avoided by es­tab­lish­ing a “Schel­ling fence”—a Schel­ling point that the var­i­ous in­ter­est groups in­volved—or your­self across differ­ent val­ues and times—make a cred­ible pre­com­mit­ment to defend.