Criticism of some popular LW articles

My com­po­si­tion teacher in col­lege told me that in some pot­tery schools, the teacher holds up your pot, ex­am­ines it, com­ments on it, and then smashes it on the floor. They do this for your first 100 pots.

In that spirit, this post’s epistemic sta­tus is SMASH THIS POT.

As an ex­per­i­ment, I’m choos­ing three pop­u­lar LW posts that hap­pen to be at the top of my feed. I’m look­ing for the quotes I most dis­agree with, and dig­ging into that dis­agree­ment. This is be­cause I no­tice I have a ten­dency to just con­sume cu­rated LW con­tent pas­sively. I’d in­stead like to ap­proach it with a more as­sertively skep­ti­cal mind­set.

Thought I be­lieve in the prin­ci­ple of char­ity, I think that a com­par­a­tive ad­van­tage of LW com­pared with other on­line spaces is as a space where frank and pub­lic ob­ject-level dis­agree­ment is per­mit­ted, as long as we avoid ad hominems or trol­ling.

Fur­ther­more, I’m fo­cus­ing here on what I took away from these posts, rather than what these au­thors in­tended, or what they learned by writ­ing them. In­so­far as the au­thor was speci­fi­cally try­ing to con­vince me to take these state­ments se­ri­ously, they failed. In­so­far as they had some other pur­pose in mind, I have ab­solutely no opinion on the mat­ter.

So with per­sonal apolo­gies to the three ex­cel­lent writ­ers I hap­pened to se­lect for this ex­er­cise, here I go.

1. The Mys­tery of the Haunted Rationalist

Quote I dis­agree with:

So al­though it’s cor­rect to say that the skep­tics’ emo­tions over­whelmed their ra­tio­nal­ity, they wouldn’t have those emo­tions un­less they thought on some level that ghosts were worth get­ting scared about.

No. They had those emo­tions be­cause they thought on some level that dark, un­fa­mil­iar en­vi­ron­ments that their com­mu­nity says are scary might be un­safe. Hence, re­word “This looks sus­pi­ciously like I’m mak­ing an ex­pected util­ity calcu­la­tion. Prob­a­bil­ity of be­ing kil­led by ghost some­thing dan­ger­ous liv­ing in the house * value of my life, com­pared to a mil­lion dol­lars.”

Im­pact on con­clu­sion:

But if that’s true, we’re now up to three differ­ent lev­els of be­lief. The one I pro­fess to my friends, the one that con­trols my an­ti­ci­pa­tion, and the one that in­fluences my emo­tions.

There are no ghosts, pro­fess skep­ti­cism.
There are no ghosts, take the bet.
There are ghosts, run for your life!

In­stead of three lev­els of be­lief, we have three lev­els of sto­ry­tel­ling. The story my friends and I share, the story that con­trols my an­ti­ci­pa­tion, and the story that’s hard­wired into my amyg­dala.

Wouldn’t it be fun to stay the night in a “haunted house?”
There’s most likely no dan­ger, take the bet.
There’s a small but im­por­tant pos­si­bil­ity of dan­ger, run for your life!

Scott’s con­flat­ing the mo­ti­va­tions un­der which an two peo­ple of his back­ground are likely to stay the night in a haunted house, with the cir­cum­stances un­der which two skep­tics might have a se­ri­ous need to dis­prove the ex­is­tence of ghosts by stay­ing the night in a haunted house. The Korean fear of fan death is not very much like the Euro-Amer­i­can fear of haunted houses, which ob­vi­ously are not en­dorsed by lo­cal sci­en­tists or the gov­ern­ment.

2. The Tallest Pygmy Effect

Quote I dis­agree with:

Tallest pygmy effects are frag­ile, es­pe­cially when they are re­li­ant on self-fulfilling prophe­cies or net­work effects. If ev­ery­one sud­denly thought the Euro was the most sta­ble cur­rency, the re­sult­ing switch would desta­bi­lize the dol­lar and hurt both its value and the US econ­omy as a whole.

This is beg­ging the ques­tion. If ev­ery­one sud­denly thought the Euro was the most sta­ble cur­rency, some­thing dra­matic would have had to have hap­pened to shift the stock mar­ket’s as­sess­ment of the fun­da­men­tals of the US vs. EU economies and gov­ern­ments. Economies are nei­ther frag­ile nor pas­sive, and these kinds of mass shifts in opinion on eco­nomic mat­ters don’t blow with the wind. Fur­ther­more, peo­ple are likely to hedge their bets. If the US and EU cur­ren­cies are similar in per­ceived sta­bil­ity, se­ri­ous in­vestors are likely to di­ver­sify.

“Tallest pygmy effect” is an­other term for “ab­solute ad­van­tage,” but with the added bag­gage of be­ing po­ten­tially offen­sive, pump­ing our in­tu­itions to­ward see­ing in­sti­tu­tions as hu­man-like in scale, and be­ing dis­con­nected with the terms “com­par­a­tive ad­van­tage” and “ab­solute” ad­van­tage, which are stan­dard eco­nomic jar­gon and have use­ful tie-ins with widely available ped­a­gog­i­cal ma­te­rial in that field.

Im­pact on con­clu­sion:

We shouldn’t use the term “tallest pygmy effect,” and should be very skep­ti­cal of LessWrong takes on eco­nomic is­sues un­less there’s strong ev­i­dence the pre­sen­ter knows what they’re talk­ing about. This up­dates me in the di­rec­tion of pop­u­lar­ity be­ing a poor proxy for ac­cu­racy or use­ful­ness.

3. Is Rhetoric Worth Learn­ing?

Quote I dis­agree with:

Quote 1:

On LessWrong, peo­ple of­ten make a hard dis­tinc­tion be­tween be­ing cor­rect and be­ing per­sua­sive; one is ra­tio­nal while the other is “dark arts.”

No. On LessWrong, peo­ple use the term “dark arts” to re­fer speci­fi­cally to tech­niques of per­sua­sion that de­liber­ately in­terfere with peo­ple’s truth-seek­ing abil­ities. Ex­am­ples might in­clude per­sonal at­tacks such as sham­ing, os­tra­ciza­tion, or threats; de­liber­ate fal­la­cious rea­son­ing such as fort-and-field or Gish Gal­lop tac­tics; or tech­niques of obfus­ca­tion and straight-up ly­ing.

Be­ing per­sua­sive isn’t “dark arts;” it’s just that good rhetoric is equally use­ful to any­one and is thus a sym­met­ric weapon, one whose use doesn’t in­her­ently help hu­man­ity progress to­ward truth.

Quote 2:

From a so­cietal per­spec­tive, mak­ing any kind of im­prove­ment, at any scale above liter­ally one-man jobs, de­pends on both cor­rect­ness and per­sua­sive­ness. If you want to achieve an out­come, you fail if you pro­pose the wrong method and if you can’t per­suade any­one of the right method.

This is true, but rhetoric is only one small part of per­sua­sion. Since this post is about rhetoric, but its im­por­tance is jus­tified on the ba­sis of the ne­ces­sity of per­sua­sion, I think this is a point that needs to be made.

Quote 3:

Some of the things I think go into talk­ing well:

Emo­tional Skills

  • How to be aware of other peo­ple’s points of view with­out merg­ing with them

  • How to dare to use a loud, clear voice or definite language

  • How to re­strain your­self from anger or upset

[Etc.]

An analo­gous list might be:

Some of the things I think go into do­ing math well:

Ad­ding skills

Know­ing how to add small num­bers in your head.

Know­ing how to trans­form re­peated ad­di­tion into mul­ti­pli­ca­tion.

Know­ing that the more pos­i­tive num­bers you add in the equa­tion, you get a larger and larger out­put.

[Etc.]

While it’s true that some­one who lacks the skills listed prob­a­bly has some ma­jor short­com­ings in the rhetoric or math de­part­ments, I think it’s un­likely that ei­ther of these lists is a use­ful de­com­po­si­tion of these skills. Both pump our in­tu­itions in the di­rec­tion of “if I prac­tice these spe­cific skills, I’ll get bet­ter at math.” Or “if I prac­tice the skills on this list I in­tu­itively think I’m bad at, I’ll get bet­ter at rhetoric over­all.”

In fact, it’s not at all clear to me that the ex­pected value of this list as a ped­a­gog­i­cal tool to teach rhetoric is net pos­i­tive or even gets across the ba­sic idea that they au­thor in­tended. It’s the kind of thing I want to keep my Sys­tem 1 away from so that it doesn’t get sucked in and mis­lead my Sys­tem 2.

Im­pact on con­clu­sion:

Rhetoric might be worth learn­ing, but there’s also a rea­son we have pro­fes­sional ed­i­tors. Divi­sion of la­bor is im­por­tant, and it’s not clear that re­ally good rhetoric is that much bet­ter at per­sua­sion than an oft-chanted slo­gan. In fact, it’s perfectly pos­si­ble that good rhetoric and a cor­rect ar­gu­ment are merely cor­re­lated, both caused by un­der­ly­ing gen­eral in­tel­li­gence and sheer depth of study. It’s also pos­si­ble that rhetoric is not a sym­met­ric weapon, and that it’s eas­ier to dress a cor­rect idea in per­sua­sive rhetoric than to so pre­sent an in­cor­rect idea.

Hey—why do we all seem to as­sume that rhetoric is un­cor­re­lated with truth, any­way?

Aris­to­tle didn’t seem to think so:

Nev­er­the­less, the un­der­ly­ing facts do not lend them­selves equally well to the con­trary views. No; things that are true and things that are bet­ter are, by their na­ture, prac­ti­cally always eas­ier to prove and eas­ier to be­lieve in.

Assess­ment:

Re­spec­tively, I see these posts as fea­tur­ing a mis­guided com­par­i­son, dis­play­ing a lack of schol­ar­ship and putting forth an un­founded as­ser­tion as a stylized fact, and me­an­der­ing around on a topic rather than de­liv­er­ing on the promise im­plied by the ti­tle.

It feels in­tu­itively true that our minds have ~sep­a­rate sys­tems for jus­tify­ing our be­liefs and chang­ing our be­liefs. Read­ing a post while look­ing for things to dis­agree and with the in­ten­tion of stat­ing those points of di­a­gree­ment clearly feels differ­ent from my nor­mal con­sump­tion pat­terns.

I no­tice my­self feel­ing both ner­vous about a nega­tive re­ac­tion to­ward my takes on these posts, and about the pos­si­bil­ity that oth­ers might re­turn the fa­vor when they read my posts.

Over­all, this ex­pe­rience leaves me with two equally con­cern­ing and com­pat­i­ble con­jec­tures.

a. My re­ac­tion to ra­tio­nal­ist con­tent is gov­erned by my frame of mind. If I read them seek­ing wis­dom, then wis­dom I shall find. If I read them to crit­i­cize, then I’ll find things to be crit­i­cal of. Without some more for­mal struc­ture in place, the na­ture of which I’m un­aware, I am not able to “as­sess” con­tent for cor­rect­ness or use­ful­ness. I can only pro­duce pos­i­tive or nega­tive feed­back. This re­minds me of Eliezer’s piece Against Devil’s Ad­vo­cacy, ex­cept that I’m less con­vinced than he seems to be that there’s such a thing as “true think­ing” dis­tinct from the pro­duc­tion of ra­tio­nal­iza­tions. Maybe if we knew what the truth was, we could mea­sure how effi­ciently differ­ent modes of think­ing would get us there. But that’s the prob­lem, right? We don’t know the truth, don’t know the goal, and so it’s very hard to put a finger on this “true think­ing” thing.

b. There is a lot of er­ror-rid­den con­tent on LessWrong. The more true (a) is, then the more true I should ex­pect (b) to be. And if er­ror-rid­den con­tent can in­fluence my frame of mind, then the more true (b) is, the more likely (a) is to be true as well. Read­ing LessWrong is like try­ing to learn a sub­ject by read­ing stu­dent es­says. It’s not a good strat­egy.

This ex­per­i­ment shifts me to­ward see­ing LessWrong as closer to a se­ri­ous-minded fan fic com­mu­nity than a body of schol­ar­ship. You can learn to write re­ally well by pro­duc­ing fan fic. But you prob­a­bly can’t learn to write well by read­ing pri­mar­ily fan fic. Yet some­body needs to read the fan fic to make the writ­ing of it a mean­ingful ex­er­cise. So read­ing and com­ment­ing on LessWrong is some­thing we do as an al­tru­is­tic act, in or­der to sup­port the learn­ing of our com­mu­nity. Post­ing on it, read­ing ex­ter­nal sources, and in­quiring into the con­di­tions of our own lives and tel­ling our own sto­ries is how we trans­form our­selves into bet­ter thinkers.

I be­lieve that read­ing and writ­ing for LessWrong has made me a much bet­ter thinker and en­hanced my lead­er­ship skills. It has my bless­ing. Scott Alexan­der and Eliz­a­beth, my first two vic­tims, are thinkers who I re­spect and whose writ­ings and con­ver­sa­tion I’ve found use­ful. Thanks also to sarah­con­stantin, whose body of writ­ings I first read to­day as far as I know and whose thoughts on rhetoric I found in­ter­est­ing and in­sight­ful.