Against Shooting Yourself in the Foot

Fol­low-up to: Sta­tus Reg­u­la­tion and Anx­ious Underconfidence

Some­how, some­one is go­ing to hor­ribly mi­suse all the ad­vice that is con­tained within this book.

Noth­ing I know how to say will pre­vent this, and all I can do is ad­vise you not to shoot your own foot off; have some com­mon sense; pay more at­ten­tion to ob­ser­va­tion than to the­ory in cases where you’re lucky enough to have both and they hap­pen to con­flict; put your­self and your skills on trial in ev­ery ac­cessible in­stance where you’re likely to get an an­swer within the next minute or the next week; and up­date hard on sin­gle pieces of ev­i­dence if you don’t already have twenty oth­ers.

I ex­pect this book to be of much more use to the un­der­con­fi­dent than the over­con­fi­dent, and con­sid­ered cun­ning plots to route printed copies of this book to only the former class of peo­ple. I’m not sure read­ing this book will ac­tu­ally harm the over­con­fi­dent, since I don’t know of a sin­gle case where any pre­vi­ously over­con­fi­dent per­son was ac­tu­ally res­cued by mod­est episte­mol­ogy and there­after be­came a more effec­tive mem­ber of so­ciety. If any­thing, it might give them a prin­ci­pled episte­mol­ogy that ac­tu­ally makes sense by which to judge those con­texts in which they are, in fact, un­likely to out­perform. In­so­far as I have an emo­tional per­son­al­ity type my­self, it’s more dis­posed to icon­o­clasm than con­for­mity, and in­ad­e­quacy anal­y­sis is what I use to di­rect that im­pulse in pro­duc­tive di­rec­tions.

But for those cer­tain folk who can­not be saved, the ter­minol­ogy in this book will be­come only their next set of ex­cuses; and this, too, is pre­dictable.

If you were never dis­posed to con­for­mity in the first place, and you read this any­way… then I won’t tell you not to think highly of your­self be­fore you’ve already ac­com­plished sig­nifi­cant things. Ad­vice like that wouldn’t have ac­tu­ally been of much use to my­self at age 15, nor would the uni­verse have been a bet­ter place if Eliezer-1995 had made the mis­take of listen­ing to it. But you might talk to peo­ple who have tried to re­form the US med­i­cal sys­tem from within, and hear what things went wrong and why.1 You might re­mem­ber the Free En­ergy Fal­lacy, and that it’s much eas­ier to save your­self than your coun­try. You might re­mem­ber that an as­pect of so­ciety can fall well short of a liquid mar­ket price, and still be far above an am­a­teur’s reach.

I don’t have good, re­peat­able ex­er­cises for train­ing your skill in this field, and that’s one rea­son I worry about the re­sults. But I can tell you this much: bet on ev­ery­thing. Bet on ev­ery­thing where you can or will find out the an­swer. Even if you’re only test­ing your­self against one other per­son, it’s a way of cal­ibrat­ing your­self to avoid both over­con­fi­dence and un­der­con­fi­dence, which will serve you in good stead emo­tion­ally when you try to do in­ad­e­quacy rea­son­ing. Or so I hope.

Beyond this, other skills that feed into in­ad­e­quacy anal­y­sis in­clude “see if the ex­pla­na­tion feels stretched,” “figure out the fur­ther con­se­quences,” “con­sider al­ter­na­tive hy­pothe­ses for the same ob­ser­va­tion,” “don’t hold up a mir­ror to life and cut off the parts of life that don’t fit,” and a gen­eral ac­quain­tance with microe­co­nomics and be­hav­ioral eco­nomics.

The policy of say­ing only what will do no harm is a policy of to­tal silence for any­one who’s even slightly imag­i­na­tive about fore­see­able con­se­quences. I hope this book does more good than harm; that is the most I can hope for it.

For your­self, dear reader, try not to be part of the harm. And if you end up do­ing some­thing that hurts you: stop do­ing it.

Beyond that, though: if you’re try­ing to do some­thing un­usu­ally well (a com­mon enough goal for am­bi­tious sci­en­tists, en­trepreneurs, and effec­tive al­tru­ists), then this will of­ten mean that you need to seek out the most ne­glected prob­lems. You’ll have to make use of in­for­ma­tion that isn’t widely known or ac­cepted, and pass into rel­a­tively un­charted wa­ters. And mod­esty is es­pe­cially detri­men­tal for that kind of work, be­cause it dis­cour­ages act­ing on pri­vate in­for­ma­tion, mak­ing less-than-cer­tain bets, and break­ing new ground. I worry that my ar­gu­ments in this book could cause an over­cor­rec­tion; but I have other, com­pet­ing wor­ries.

The world isn’t mys­te­ri­ously doomed to its cur­rent level of in­ad­e­quacy. In­cen­tive struc­tures have parts, and can be reeng­ineered in some cases, worked around in oth­ers.

Similarly, hu­man bias is not in­her­ently mys­te­ri­ous. You can come to un­der­stand your own strengths and weak­nesses through care­ful ob­ser­va­tion, and schol­ar­ship, and the gen­er­a­tion and test­ing of many hy­pothe­ses. You can avoid over­con­fi­dence and un­der­con­fi­dence in an even-handed way, and rec­og­nize when a sys­tem is in­ad­e­quate at do­ing X for cost Y with­out be­ing ex­ploitable in X, or when it is ex­ploitable-to-some­one but not ex­ploitable-to-you.

Modesty and im­mod­esty are bad heuris­tics be­cause even where they’re cor­rect­ing for a real prob­lem, you’re li­able to over­cor­rect.

Bet­ter, I think, to not worry quite so much about how lowly or im­pres­sive you are. Bet­ter to med­i­tate on the de­tails of what you can do, what there is to be done, and how one might do it.

This con­cludes Inad­e­quate Equil­ibria. The full book is now available in elec­tronic and print form through equil­ibri­

  1. As an ex­am­ple, see Zvi Mow­show­itz’s “The Thing and the Sym­bolic Rep­re­sen­ta­tion of The Thing,” on Me­taMed, a failed med­i­cal con­sult­ing firm that tried to pro­duce un­usu­ally high-qual­ity per­son­al­ized med­i­cal re­ports.