Calibrate your self-assessments

When I moved to Ire­land, I knew that their school sys­tem, and in par­tic­u­lar their ex­am­i­na­tions, would be differ­ent from the ones I was used to. I ed­u­cated my­self on them and by the time I took my first exam I thought I was rea­son­ably pre­pared.

I walked out of my first ex­am­i­na­tion al­most cer­tain I had failed. I re­mem­ber emailing my par­ents, apol­o­giz­ing to them for my failure and promis­ing I would do bet­ter when I re­peated the class.

Then I got my re­sults back, and learned I had passed with hon­ors.

This situ­a­tion re­peated it­self with de­press­ing reg­u­lar­ity over the next few semesters. Took exam, walked out in tears cer­tain I had failed, made angsty com­plaints and apolo­gies, got re­sults back, cel­e­brated. Even­tu­ally I de­cided that I might as well skip steps two to five and go straight to the cel­e­bra­tions.

This was harder than I ex­pected. Just know­ing that my feel­ings of ab­ject failure usu­ally ended out all right did not change those feel­ings of ab­ject failure. I still walked out of each exam with the same gut cer­tainty of dis­aster I had always had. What I did learn to do was ig­nore it: to force my­self to walk home with a smile on my face and re­fuse to let my­self dwell on the feel­ings of failure or take them se­ri­ously. And in this I was suc­cess­ful, and now the feel­ings of ab­ject failure pro­duce only a tiny twinge of stress.

In LW ter­minol­ogy, I am cal­ibrat­ing my self-as­sess­ment of ex­am­i­na­tion suc­cess1.

We ap­pre­ci­ate ob­jec­tive mea­sure­ments, like a per­cent score on an ex­am­i­na­tion, or the run­ning time of a marathon in min­utes and sec­onds. But in the ab­sence of such mea­sure­ments, we use sub­jec­tive men­tal es­ti­mates: how I feel I did on this exam, or how plau­si­ble that the­ory sounds.

The ra­tio­nal­ity liter­a­ture has es­pe­cially fo­cused on one par­tic­u­lar sub­jec­tive men­tal es­ti­mate: our feel­ings of prob­a­bil­ity. For ex­am­ple, some­one may say they feel 80% cer­tain that Ger­many is larger than France. How­ever, if they con­sis­tently an­swer ques­tions like this with 80% con­fi­dence, and only get 60% right, then we say they are mis-cal­ibrated: their sub­jec­tive men­tal es­ti­mate of prob­a­bil­ity has a con­sis­tent mis­match with a more nor­ma­tively cor­rect prob­a­bil­ity. Cal­ibra­tion means re­vis­ing your sub­jec­tive men­tal es­ti­mate un­til it matches the ob­jec­tive value it tries to es­ti­mate; so that when you es­ti­mate some­thing with 80% con­fi­dence, you get it right 80% of the time.

My story about exam scores is also a story about cal­ibra­tion. My sub­jec­tive men­tal es­ti­mate of my exam scores was con­sis­tently too low; I would es­ti­mate I failed when I had re­ally passed by a wide mar­gin. By sup­press­ing my origi­nal men­tal es­ti­mate and re­plac­ing it with one bet­ter in­formed by past ex­pe­rience, I am cal­ibrat­ing my es­ti­mate of exam scores.

Since pass­ing my ex­ams, I’ve iden­ti­fied other ar­eas of my life where I need to cal­ibrate my es­ti­mates:

-- Em­bar­rass­ment. I used to be mor­tified if I an­swered a ques­tion wrong in class, as­sum­ing that peo­ple would judge me on it as long as they knew me. After think­ing about it, I re­al­ized that al­though many peo­ple in my class an­swer ques­tions wrong ev­ery day, I liter­ally can­not re­mem­ber a sin­gle one. If you pointed out any stu­dent in my class, even one of my close friends who I would be ex­pected to pay ex­tra at­ten­tion to, and asked me “Has this per­son ever an­swered a ques­tion wrong in class?” I wouldn’t be able to tell you. This sug­gests they won’t re­mem­ber my mis­takes ei­ther, and that my sub­jec­tive feel­ing of loss of re­spect on an­swer­ing a ques­tion wrong is ex­ag­ger­ated to say the least2.

-- In­ter­est­ing­ness. I tend to think that if I talk about some­thing I’m in­ter­ested in, other peo­ple will be in­ter­ested in it too. No mat­ter how fas­ci­nat­ing the un­der­ly­ing con­cept to me, nor how well I think I’m ex­plain­ing it, this al­most never hap­pens.

-- Flirt­ing. Through painful trial and er­ror, I’ve found that my hunch that a woman likes me is al­most always wrong. Some­one will be flirt­ing very heav­ily with me, and I’ll think “there is no way in the world she’s not into me”, and then it will turn out she will not be into me.

Th­ese aren’t just things I’m of­ten wrong about; mak­ing a list of those would be a Sisyphean task. They’re the things that I’m wrong about that my nat­u­ral in­stincts never auto-cor­rect, so that I know I’m go­ing to keep be­ing wrong un­less I con­sciously cal­ibrate my nat­u­ral in­stincts against a rea­soned opinion.

In gen­eral, I find I am most of­ten mis­cal­ibrated in ar­eas that re­late to self eval­u­a­tion. Cog­ni­tive psy­chol­ogy has a slew of ideas about so-called “self-as­sess­ment bi­ases”. You’ve prob­a­bly heard the self-serv­ing ones where 94% of pro­fes­sors rate their teach­ing abil­ity above av­er­age, or how ev­ery­one thinks they’re an above av­er­age driver, or how (iron­i­cally) ev­ery­one thinks they’re less sus­cep­ti­ble to bi­ases than other peo­ple. But more sur­pris­ingly, I also find cases where peo­ple con­sis­tently un­der­es­ti­mate them­selves—like my own ten­dency to always think I’ve failed my ex­am­i­na­tions. I don’t have a good ex­pla­na­tion of this—I don’t know if it’s strate­gic hu­mil­ity, self-ver­ifi­ca­tion, some un­der­ly­ing de­pres­sion-like state, or what—but I’m pretty sure it ex­ists. And there are two situ­a­tions in which I find it most com­mon and most an­noy­ing.

The first in­volves good looks. Some peo­ple just have no idea how at­trac­tive they are or aren’t. This is most ob­vi­ous in body dys­mor­phic di­s­or­der, a con­di­tion where nor­mal look­ing (or even very at­trac­tive) peo­ple some­how get it into their head that some fea­ture of theirs—their nose, their hair, their weight—is in­hu­manly hideous and that they look like some kind of swamp mon­ster. This is an offi­cially rec­og­nized psy­chi­a­tric di­s­or­der be­cause it’s com­pletely di­vorced from re­al­ity—usu­ally their nose or hair or what­ever looks ab­solutely nor­mal and just like ev­ery­one else’s.

BDD is less rare than peo­ple think, at about one to two per­cent of the pop­u­la­tion, but even peo­ple with­out the full-fledged di­s­or­der can be re­ally bad at de­ter­min­ing how at­trac­tive they are or aren’t. There are a lot of pretty girls who go around say­ing they’re ugly in or­der to trick peo­ple into com­pli­ment­ing them, or to sig­nal that they’re available and not too picky, but I’ve come to re­al­ize that there are also a lot of pretty girls who gen­uinely be­lieve they’re ugly (it’s less ob­vi­ous in men, but I wouldn’t be sur­prised if it were there un­der the sur­face).

And re­search agrees: stud­ies show that peo­ple are uniquely bad at rat­ing their own phys­i­cal at­trac­tive­ness. The opinion of un­bi­ased ob­servers eval­u­at­ing a sub­ject’s at­trac­tive­ness usu­ally cor­re­late at a level of r = .4 to .5; the opinion of the sub­ject her­self cor­re­lates with ev­ery­one else only around the r = .2 level. Other stud­ies us­ing pur­port­edly “ob­jec­tive” mea­sures of at­trac­tive­ness like fa­cial sym­me­try re­port a similarly low level of cor­re­la­tion be­tween the ob­jec­tive mea­sures and the self-re­ports. What self-re­ported phys­i­cal at­trac­tive­ness cor­re­lates strongly with is not ob­jec­tive at­trac­tive­ness, but self-re­ported self-es­teem, with r val­ues around .5 or .6 de­pend­ing on the study.

If you’re not so good at statis­tics—that means that peo­ple of­ten agree on how at­trac­tive a par­tic­u­lar sub­ject is, but that sub­ject’s es­ti­mate of her own at­trac­tive­ness is of­ten com­pletely differ­ent from ev­ery­one else’s (in ei­ther di­rec­tion), and more re­lated to that sub­ject’s self-es­teem than to re­al­ity.

Sites like ho­ or okcu­pid’s MyBestFace have a lot of prob­lems, most ob­vi­ously that they de­pend a lot upon how good a spe­cific photo is. But I think ei­ther is leagues ahead of try­ing to guess how at­trac­tive you are to oth­ers based on how at­trac­tive you feel. If you have any con­cern what­so­ever about how at­trac­tive you are, the worst thing you can do is trust your own brain, es­pe­cially if it’s tel­ling you you’re prob­a­bly pretty ugly when ev­ery­one around you seems to think you’re okay.

Which brings me to the num­ber one most tragic failure of the in­side view I see in my friends, my ac­quain­tances, and the psy­chi­a­tric pa­tients I en­counter.

Niet­zsche said that a ca­sual stroll through an in­sane asy­lum shows that faith does not prove any­thing. Such an ex­pe­rience might also teach peo­ple to be skep­ti­cal of their own sub­jec­tive val­u­a­tion of them­selves—their self-es­teem. If our hy­po­thet­i­cal vis­i­tor doesn’t figure it out af­ter see­ing the de­pressed pa­tients, who are ob­sessed with their own guilt and moral worth­less­ness to the point of con­fess­ing to any crime they hear about be­cause it seems like the sort of thing some­one as awful as them­selves might do, she can go visit the schizophren­ics with delu­sions of grandeur, who in­sist they are the next Je­sus or Ein­stein, or God’s cho­sen rep­re­sen­ta­tive on Earth.

Th­ese peo­ple get locked away be­cause their self-es­teem is at an ex­treme no sane hu­man would ever reach. But your lo­ca­tion out­side the in­sane asy­lum doesn’t prove your own calcu­la­tions of self-es­teem come from rea­son­ing pro­cesses that are any more valid. We all know self-ob­sessed nar­cis­sists with­out any real achieve­ments to their name, and we all know peo­ple who in­sist that they are ugly and stupid and un­like­able even though they don’t seem any worse off than any­one else.

Re­search con­firms that peo­ple’s self-es­teem is poorly cor­re­lated with re­al­ity. Across many ex­per­i­ments with many differ­ent de­signs, peo­ple’s self-re­ported like­abil­ity has no cor­re­la­tion with their like­abil­ity as re­ported by other peo­ple with whom they in­ter­act. This is true whether the ex­per­i­ment mea­sures ar­tifi­cial in­ter­ac­tion in a lab, simu­lated “dates” with peo­ple of the op­po­site sex, or the at­ti­tudes of their room­mates.

There are no stud­ies cor­re­lat­ing self-re­ported moral­ity with ex­per­i­men­tally de­ter­mined moral­ity, but if you want con­duct one, you could prob­a­bly gather enough se­cretly gay evan­gel­i­cal ministers and adulter­ous fam­ily-val­ues poli­ti­ci­ans to make up a pretty good sam­ple size.

If you hate your­self and think you’re worth­less, take a mo­ment to con­sider whether you have any ev­i­dence that you’re ob­jec­tively do­ing any worse than any­one else, or whether you just have a low self-es­teem set point. If the lat­ter ap­pears to be true, then try to re­place the in­side view with the out­side view when wor­ry­ing about how much bother you’re be­ing to other peo­ple or whether you “de­serve” to be happy.

(if your prob­lem is in the other di­rec­tion you may not have as much vested in­ter­est in cor­rect­ing your­self, but do keep in mind that most of the pur­ported benefits of self-con­fi­dence have been ex­ag­ger­ated).


Peo­ple’s sub­jec­tive men­tal es­ti­mates are of­ten way off, es­pe­cially when they’re es­ti­mat­ing qual­ities closely linked to their self-worth. Both ev­ery­day ex­pe­rience and sci­en­tific re­search provide am­ple ev­i­dence of peo­ple who both un­der­es­ti­mate and over­es­ti­mate them­selves in var­i­ous ways. If you worry you may be one of those peo­ple, try and get ob­jec­tive es­ti­mates of the pa­ram­e­ter you’re con­cerned about from other peo­ple or from em­piri­cal test­ing. Then make an effort of will to con­sciously re­place your sub­jec­tive es­ti­mates with your new bet­ter-cal­ibrated es­ti­mates.


1: This could also be in­ter­preted as re­plac­ing the In­side View with the Out­side View and this would also be a good moral to draw from the story; I’m phras­ing it in terms of cal­ibra­tion be­cause it’s more ap­pro­pri­ate for some of the other ex­am­ples later down.

2: See Gilovich, Med­vec, and Sav­it­sky, 2000 for ex­per­i­men­tal proof of the same idea