Rationalists Are Less Credulous But Better At Taking Ideas Seriously

Con­sider the fol­low­ing com­monly-made ar­gu­ment: cry­on­ics is un­likely to work. Trained ra­tio­nal­ists are signed up for cry­on­ics at rates much greater than the gen­eral pop­u­la­tion. There­fore, ra­tio­nal­ists must be pretty gullible peo­ple, and their claims to be good at eval­u­at­ing ev­i­dence must be ex­ag­ger­a­tions at best.

This ar­gu­ment is wrong, and we can prove it us­ing data from the last two Less Wrong sur­veys.

The ques­tion at hand is whether ra­tio­nal­ist train­ing—rep­re­sented here by ex­ten­sive fa­mil­iar­ity with Less Wrong ma­te­rial—makes peo­ple more likely to be­lieve in cry­on­ics.

We in­ves­ti­gate with a cross-sec­tional study, look­ing at proto-ra­tio­nal­ists ver­sus ex­pe­rienced ra­tio­nal­ists. Define proto-ra­tio­nal­ists as those re­spon­dents to the Less Wrong sur­vey who in­di­cate they have been in the com­mu­nity for less than six months and have zero karma (usu­ally in­dica­tive of never hav­ing posted a com­ment). And define ex­pe­rienced ra­tio­nal­ists as those re­spon­dents to the Less Wrong sur­vey who in­di­cate they have been in the com­mu­nity for over two years and have >1000 karma (usu­ally in­dica­tive of hav­ing writ­ten many well-re­ceived posts).

By these defi­ni­tions, there are 93 proto-ra­tio­nal­ists, who have been in the com­mu­nity an av­er­age of 1.3 months, and 134 ex­pe­rienced ra­tio­nal­ists, who have been in the com­mu­nity an av­er­age of 4.5 years. Proto-ra­tio­nal­ists gen­er­ally have not read any ra­tio­nal­ity train­ing ma­te­rial—only 2093 had read even one-quar­ter of the Less Wrong Se­quences. Ex­pe­rienced ra­tio­nal­ists are, well, more ex­pe­rienced: two-thirds of them have read pretty much all the Se­quence ma­te­rial.

Proto-ra­tio­nal­ists thought that, on av­er­age, there was a 21% chance of an av­er­age cry­on­i­cally frozen per­son be­ing re­vived in the fu­ture. Ex­pe­rienced ra­tio­nal­ists thought that, on av­er­age, there was a 15% chance of same. The differ­ence was marginally sig­nifi­cant (p < 0.1).

Marginal sig­nifi­cance is a copout, but this isn’t our only data source. Last year, us­ing the same defi­ni­tions, proto-ra­tio­nal­ists as­signed a 15% prob­a­bil­ity to cry­on­ics work­ing, and ex­pe­rienced ra­tio­nal­ists as­signed a 12% chance. We see the same pat­tern.

So ex­pe­rienced ra­tio­nal­ists are con­sis­tently less likely to be­lieve in cry­on­ics than proto-ra­tio­nal­ists, and ra­tio­nal­ist train­ing prob­a­bly makes you less likely to be­lieve cry­on­ics will work.

On the other hand, 0% of proto-ra­tio­nal­ists had signed up for cry­on­ics com­pared to 13% of ex­pe­rienced ra­tio­nal­ists. 48% of proto-ra­tio­nal­ists re­jected the idea of sign­ing up for cry­on­ics en­tirely, com­pared to only 25% of ex­pe­rienced ra­tio­nal­ists. So al­though ra­tio­nal­ists are less likely to be­lieve cry­on­ics will work, they are much more likely to sign up for it. Last year’s sur­vey shows the same pat­tern.

This is not nec­es­sar­ily sur­pris­ing. It only in­di­cates that ex­pe­rienced ra­tio­nal­ists and proto-ra­tio­nal­ists treat their be­liefs in differ­ent ways. Proto-ra­tio­nal­ists form a be­lief, play with it in their heads, and then do what­ever they were go­ing to do any­way - usu­ally some var­i­ant on what ev­ery­one else does. Ex­pe­rienced ra­tio­nal­ists form a be­lief, ex­am­ine the con­se­quences, and then act strate­gi­cally to get what they want.

Imag­ine a lot­tery run by an in­com­pe­tent offi­cial who ac­ci­den­tally sets it up so that the av­er­age pay­off is far more than the av­er­age ticket price. For ex­am­ple, maybe the lot­tery sells only ten $1 tick­ets, but the jack­pot is $1 mil­lion, so that each $1 ticket gives you a 10% chance of win­ning $1 mil­lion.

Goofus hears about the lot­tery and re­al­izes that his ex­pected gain from play­ing the lot­tery is $99,999. “Huh,” he says, “the num­bers say I could ac­tu­ally win money by play­ing this lot­tery. What an in­ter­est­ing math­e­mat­i­cal cu­ri­os­ity!” Then he goes off and does some­thing else, since ev­ery­one knows play­ing the lot­tery is what stupid peo­ple do.

Gal­lant hears about the lot­tery, performs the same calcu­la­tion, and buys up all ten tick­ets.

The rele­vant differ­ence be­tween Goofus and Gal­lant is not skill at es­ti­mat­ing the chances of win­ning the lot­tery. We can even change the prob­lem so that Gal­lant is more aware of the un­like­li­hood of win­ning than Goofus—per­haps Goofus mis­tak­enly be­lieves there are only five tick­ets, and so Gal­lant’s su­pe­rior knowl­edge tells him that win­ning the lot­tery is even more un­likely than Goofus thinks. Gal­lant will still play, and Goofus will still pass.

The rele­vant differ­ence is that Gal­lant knows how to take ideas se­ri­ously.

Tak­ing ideas se­ri­ously isn’t always smart. If you’re the sort of per­son who falls for proofs that 1 = 2 , then re­fus­ing to take ideas se­ri­ously is a good way to avoid end­ing up ac­tu­ally be­liev­ing that 1 = 2, and a gen­er­ally ex­cel­lent life choice.

On the other hand, progress de­pends on some­one some­where tak­ing a new idea se­ri­ously, so it’s nice to have peo­ple who can do that too. Helping peo­ple learn this skill and when to ap­ply it is one goal of the ra­tio­nal­ist move­ment.

In this case it seems to have been suc­cess­ful. Proto-ra­tio­nal­ists think there is a 21% chance of a new tech­nol­ogy mak­ing them im­mor­tal—surely an out­come as de­sir­able as any lot­tery jack­pot—con­sider it an in­ter­est­ing cu­ri­os­ity, and go do some­thing else be­cause only weirdos sign up for cry­on­ics.

Ex­pe­rienced ra­tio­nal­ists think there is a lower chance of cry­on­ics work­ing, but some of them de­cide that even a pretty low chance of im­mor­tal­ity sounds pretty good, and act strate­gi­cally on this be­lief.

This is not to ei­ther at­tack or defend the policy of as­sign­ing a non-neg­ligible prob­a­bil­ity to cry­on­ics work­ing. This is meant to show only that the differ­ence in cry­on­ics sta­tus be­tween proto-ra­tio­nal­ists and ex­pe­rienced ra­tio­nal­ists is based on meta-level cog­ni­tive skills in the lat­ter whose de­sir­a­bil­ity is or­thog­o­nal to the ob­ject-level ques­tion about cry­on­ics.

(an ear­lier ver­sion of this ar­ti­cle was posted on my blog last year; I have moved it here now that I have repli­cated the re­sults with a sec­ond sur­vey)