Everything I ever needed to know, I learned from World of Warcraft: Goodhart’s law

Link post

This is the first in a se­ries of posts about les­sons from my ex­pe­riences in World of War­craft. I’ve been talk­ing about this stuff for a long time—in fo­rum com­ments, in IRC con­ver­sa­tions, etc.—and this se­ries is my at­tempt to make it all a bit more leg­ible. I’ve added foot­notes to ex­plain some of the jar­gon, but if any­thing re­mains in­com­pre­hen­si­ble, let me know in the com­ments.

World of War­craft, es­pe­cially WoW raid­ing[1], is very much a game of num­bers and de­tails.

At first, in the very early days of WoW, peo­ple didn’t nec­es­sar­ily ap­pre­ci­ate this very well, nor did they have any good way to use that fact even if they did ap­pre­ci­ate it. (And—this bit is a tan­gent, but an in­ter­est­ing one—a lot of su­per­sti­tions arose about how game me­chan­ics worked, which abil­ities had which effects, what caused bosses[2] to do this or that, etc.—all the usual hu­man re­sponses to com­plex phe­nom­ena where dis­cern­ing cau­sa­tion is hard.) And, more im­por­tantly and on-topic, there was no re­ally good way to sift the good play­ers from the bad; nor to im­prove one’s own perfor­mance.

This ham­pered pro­gres­sion. (“Pro­gres­sion” is a WoW term of art for “get­ting a boss down, get­ting bet­ter at do­ing so, and ad­vanc­ing to the next challenge; rinse, re­peat”. Hence “pro­gres­sion raid­ing” meant “work­ing on defeat­ing the cur­rently-not-yet-beaten challenges”.)

The com­bat log

Screenshot of the combat log

One cru­cial fea­ture of WoW is the com­bat log. This is a lit­tle win­dow that ap­pears at the bot­tom of your screen; into it, the game out­puts lines that re­port ev­ery­thing that hap­pens to or around your char­ac­ter. All dam­age done or taken, all hits taken or avoided, abil­ities used, etc., etc.—ev­ery­thing. This in­for­ma­tion is out­put in a spe­cific for­mat; and it can be parsed by the add-on sys­tem[3].

Nat­u­rally, then, peo­ple soon be­gan writ­ing add-ons that did parse it—parse it, and or­ga­nize it, and pre­sent var­i­ous statis­ti­cal and ag­grega­tive trans­for­ma­tions of that data in an easy-to-view form—which, im­por­tantly, could be viewed live, as one played.

Thus arose the cat­e­gory of add-ons known as “dam­age me­ters”.

The dam­age meters

Screenshot of a damage meter addon

Of course the “dam­age me­ters” showed other things as well—but view­ing dam­age out­put was the most pop­u­lar and ex­cit­ing use. (What more ex­cit­ing set of data is there, but one that shows how much you’re hurt­ing the mon­sters, with your fire­balls and the strikes of your sword?) The bet­ter class of dam­age-me­ter add-ons not only recorded this data, but also syn­chro­nized and ver­ified it, by com­mu­ni­cat­ing be­tween in­stances of them­selves run­ning on the clients of all the peo­ple in the raid.

Which meant that now you could have a cen­tral­ized dis­play of just what ex­actly ev­ery­one in the raid was do­ing, and how, and how well.

This was a great boon to raid lead­ers and raid guilds ev­ery­where! You have a raid of 40 peo­ple, one of the DPSers[4] is in­com­pe­tent, can’t DPS to save his life, or he’s AFK[5] half the time, or he’s just mess­ing around—who can tell?

With dam­age me­ters—ev­ery­one can tell.

Now, you could sift the bad from the good, the con­scien­tious from the moochers and slack­ers, and so on. And more: some­one’s not perform­ing well but seems to be try­ing, but failing? Well, now you look at his abil­ity break­down[6], you com­pare it to that of the top DPSers, you see what the differ­ence is and you say—no, Bob, don’t use abil­ity X in this situ­a­tion, use abil­ity Y, it does more dam­age.

The problem

All of this is fan­tas­tic. But… it im­me­di­ately and pre­dictably be­gan to be sub­verted by Good­hart’s law.

To wit: if you are look­ing at the DPS me­ters but “max­i­mize DPS” is not perfectly cor­re­lated with “kill the boss” (that be­ing, of course, your goal)… then you have a prob­lem.

This may be ob­vi­ous enough; but it is also in­struc­tive to con­sider the spe­cific ways that those things can come un­cou­pled. So, let me try and enu­mer­ate them.

The Thing is valuable, but it’s not the only valuable thing

There are other things that must be done, that are less glamorous, and may de­tract from do­ing the Thing, but each of which is a sine qua non of suc­cess. (In WoW, this might man­i­fest as: the boss must be dam­aged, but also, adds must be kited—never mind what this means, know only that while a DPSer is do­ing that, he can’t be DPSing!)

And yet more in­sidious elab­o­ra­tions on that pos­si­bil­ity:

We can’t af­ford to specialize

What if, yes, this other thing must be done, but the max­i­mally com­pe­tent raid mem­ber must both do that thing and also the main thing? He won’t DPS as well as he could, but he also can’t just not DPS, be­cause then you fail and die; you can’t say “ok, just do the other thing and for­get DPSing”. In other words, what if the sec­ondary task isn’t just some­thing you can put some­one full-time on?

Out­side of WoW, you might en­counter this in, e.g., a soft­ware de­vel­op­ment con­text: sup­pose you’re mea­sur­ing com­mits, but also doc­u­men­ta­tion must be writ­ten—but you don’t have (nor can you af­ford to hire) a ded­i­cated docs writer! (Similar ex­am­ples abound.)

Then other pos­si­bil­ities:

Tun­nel vi­sion kills

The Thing is valuable, but tun­nel-vi­sion­ing on The Thing means that you will for­get to fo­cus on cer­tain other things, the re­sult be­ing that you are hor­ribly doomed some­how—this is an in­di­vi­d­ual failing, but given rise to by the in­cen­tives of the sin­gu­lar met­ric (i.e., DPS max­i­miza­tion).

(The WoW ex­am­ple is: you have to DPS as hard as pos­si­ble, but you also have to move out the way when the boss does his “ev­ery­one in a 10 foot ra­dius dies to hor­rible fire” abil­ity.)

And yet more in­sidious ver­sions of this one:

Tun­nel vi­sion kills… other people

Yes, if this tun­nel-vi­sion dooms you, per­son­ally, in a pre­dictable and un­avoid­able fash­ion, then it is easy enough to say “do this other thing or else you will pre­dictably also suffer on the sin­gu­lar met­ric” (the dead throw no fire­balls).

But the real prob­lem comes in when ne­glect­ing such a sec­ondary duty cre­ates ex­ter­nal­ities; or when the de­struc­tive effect of the ne­glect can be pushed off on some­one else.

(In WoW: “I won’t run out of the fire and the heal­ers can just heal me and I won’t die and I’ll do more DPS than those who don’t run out”; in an­other con­text, per­haps “I will ne­glect to com­ment my code, or to test it, or to do other main­te­nance tasks; these may be done for me by oth­ers, and mean­while I will max­i­mize my sin­gu­lar met­ric [com­mits]”.)

It’s al­most always the case that you have the com­par­a­tive ad­van­tage in do­ing the sec­ondary thing that avoids the doom; if oth­ers have to pick up your slack there, it’ll be way less effi­cient, over­all.

Op­ti­miza­tion has a price

The Thing is valuable, yes; and it may be that there are ways to in fact in­crease your level of the Thing, re­ally do in­crease it, but at a non-ob­vi­ous cost that is borne by oth­ers. Yes, you are im­prov­ing your effec­tive­ness, but the price is that oth­ers, do­ing other things, now have to work harder, or waste effort on the con­se­quences, etc.

(Many ex­am­ples of this in WoW, such as “start DPSing be­fore you’re sup­posed to, and risk the boss get­ting away from the tank and kil­ling the raid”. In a gen­eral con­text, this is “tak­ing risks, the con­se­quences of which are dire, and the miti­ga­tion of which is a cost borne by oth­ers, not you”.)

Then this one is par­tic­u­larly sub­tle and may be hard to spot:

Every­one wants the chance to show off their skill

The Thing is valuable, and do­ing it well brings judg­ment of com­pe­tence, and there­fore sta­tus. There are roles within the pro­ject’s task al­lo­ca­tion that nat­u­rally give greater op­por­tu­ni­ties to max­i­mize your perfor­mance of the Thing, and there­fore peo­ple seek out those roles prefer­en­tially—even when an op­ti­mal al­lo­ca­tion of roles, by rel­a­tive skill or ap­pro­pri­ate­ness to task, would lead them to be placed in roles that do not let them do the most of the Thing.

(In WoW: if the most skil­led hunter is needed to kite the add, but there are no “who kited the add best” me­ters, only dam­age me­ters… well, then maybe that most skil­led hunter, when called upon to kite the add, says “Bob over there can kite the add bet­ter”—and as a re­sult, be­cause Bob ac­tu­ally is worse at that, the raid fails. In other con­texts… well, many ex­am­ples, of course; glory-seek­ing in pro­ject par­ti­ci­pa­tion, etc.)

Of course there is also:

A good ex­cuse for incompetence

This is the con­verse of the first sce­nario: if the Thing is valuable but you are bad at it, you might de­liber­ately seek out roles in which there is an ex­cuse for not perform­ing it well (be­cause the role’s pri­mary pur­pose is some­thing else)—de­spite the fact that, ac­tu­ally, the ideal per­son in your role also does the Thing (even if not as much as in a Thing-cen­tered role).

  1. “Raid dun­geons” were the most difficult challenges in the game—difficult enough to re­quire up to 40 play­ers to band to­gether and co­op­er­ate, and co­op­er­ate effec­tively, in or­der to over­come them. “Raid­ing” refers to the work of defeat­ing these challenges. Most of what I have to say in­volves raid­ing, be­cause it was this part of WoW that—due to the re­quire­ment for effec­tive group effort (and for other, re­lated, rea­sons)—gave rise to the most in­ter­est­ing so­cial pat­terns, the most illu­mi­nat­ing group dy­nam­ics, etc. ↩︎

  2. “Boss mon­sters” or “bosses” are the pow­er­ful com­puter-con­trol­led op­po­nents which play­ers must defeat in or­der to re­ceive the in-game re­wards which are re­quired to im­prove their char­ac­ters’ ca­pa­bil­ities. The most pow­er­ful and difficult-to-defeat bosses were, of course, raid bosses (see pre­vi­ous foot­note). ↩︎

  3. WoW al­lows play­ers to cre­ate add-ons—pro­grams that en­hance the game’s user in­ter­face, add fea­tures, and so on. Many of these were very pop­u­lar—down­loaded and used by many other play­ers—and some came to be con­sid­ered nec­es­sary tools for suc­cess­ful raid­ing. ↩︎

  4. “Da­m­age Per Se­cond”, i.e. do­ing dam­age to the boss, in or­der to kill it (this be­ing the goal). Along with “tank” and “healer”, “DPS” is one of the three roles that a char­ac­ter might fulfill in a group or raid. A raid needed a cer­tain num­ber of peo­ple in each role, and all were crit­i­cal to suc­cess. ↩︎

  5. “Away From Key­board”, i.e., not ac­tu­ally at the com­puter—which means, ob­vi­ously, that his char­ac­ter is stand­ing mo­tion­less, and not con­tribut­ing to the raid’s efforts in the slight­est. ↩︎

  6. In other words: which of his char­ac­ter’s abil­ities he was us­ing, in what pro­por­tion, etc. Is the mage cast­ing Fire­ball, or Frost­bolt, or Ar­cane Mis­sile? Is the hunter us­ing Ar­cane Shot, and if so, how of­ten? By ex­am­in­ing the record—recorded and shown by the dam­age me­ters—of a char­ac­ter’s abil­ity us­age, it was of­ten very easy to de­ter­mine who was play­ing op­ti­mally, and who was mak­ing mis­takes. ↩︎