The EMH Aten’t Dead

Cross-post­ing from my per­sonal blog, but writ­ten pri­mar­ily for Less Wrong af­ter re­cent dis­cus­sion here.


There are whispers that the Effi­cient-Mar­ket Hy­poth­e­sis is dead. Eliezer’s faith has been shaken. Scott says EMH may have been the real vic­tim of the coro­n­avirus.

The EMH states that “as­set prices re­flect all available in­for­ma­tion”. The di­rect im­pli­ca­tion is that if you don’t have any non-available in­for­ma­tion, you shouldn’t ex­pect to be able to beat the mar­ket, ex­cept by chance.

But some peo­ple were able to pre­empt the corona crash, with­out any spe­cial knowl­edge! Ja­cob men­tioned sel­l­ing some of his stocks be­fore the mar­ket re­acted. Wei Dai bought out-of-the-money ‘put’ op­tions, and took a very hand­some profit. Others shorted the mar­ket.

Th­ese peo­ple were read­ing the same news and re­ports as ev­ery­one else. They prof­ited on the ba­sis of pub­lic in­for­ma­tion that should have been priced in.

And so, the EMH is dead, or dy­ing, or at the very least, has a very nasty-sound­ing cough.

I think that ru­mours of the death of effi­cient mar­kets have been greatly ex­ag­ger­ated. It seems to me the EMH is very much al­ive-and-kick­ing, and the re­cent dis­cus­sion of­ten in­volves com­mon mi­s­un­der­stand­ings that it might be helpful to iron out.

This nec­es­sar­ily in­volves piss­ing on peo­ple’s pa­rade, which is not much fun. So it’s im­por­tant to say up­front that al­though I don’t know Wei Dai, he is no doubt a brilli­ant guy, that Ja­cob is my favourite blog­ger in the di­as­pora, that I would give my left tes­ti­cle to have Scott’s writ­ing tal­ent and ridicu­lous work ethic, that Eliezer is a leg­end whose work I have per­son­ally benefited from greatly, etc.

But in the spirit of the whole ra­tio­nal­ity thing, I want to gen­tly challenge what looks more like a case of ‘back-slaps for the boys’ than a death knell for effi­cient mar­kets.

First: how the heck did the mar­ket get the coro­n­avirus so wrong?

The Great Coron­avirus Trade

Lots of peo­ple ini­tially un­der­re­acted to COVID-19. We are only hu­man. But the stock­mar­ket is not only hu­man—it’s meant to be bet­ter than this.

Here’s Scott, in A Failure, But Not of Pre­dic­tion:

The stock mar­ket is a gi­ant co­or­di­nated at­tempt to pre­dict the econ­omy, and it reached an all-time high on Fe­bru­ary 12, sug­gest­ing that an­a­lysts ex­pected the econ­omy to do great over the fol­low­ing few months. On Fe­bru­ary 20th it fell in a way that sug­gested a mild in­con­ve­nience to the econ­omy, but it didn’t re­ally start plum­met­ing un­til mid-March – the same time the me­dia fi­nally got a clue. Th­ese aren’t empty suits on ca­ble TV with no skin in the game. Th­ese are the best pre­dic­tive in­sti­tu­tions we have, and they got it wrong.

But… this isn’t how it went down. As AllAmer­i­canBreak­fast and oth­ers pointed out in the com­ments, the mar­ket started re­act­ing in the last week of Fe­bru­ary, with news head­lines di­rectly link­ing the de­cline to the ‘coro­n­avirus’. By the time we get to mid-March, we’re not far off the bot­tom.

(You can con­firm this for your­self in a few sec­onds by look­ing at a chart of the rele­vant time pe­riod.)

EDIT: Scott has ex­plained his ra­tio­nale here. Although I still think his ver­sion of events is in­cor­rect as phrased, I want to make it clear I am not ac­cus­ing him of de­liber­ately mas­sag­ing the data or any other such shenani­gans, and the next para­graph about re­vi­sion­ist his­tory etc was only meant to be a gen­eral ob­ser­va­tion about how peo­ple re­sponded. My apolo­gies to Scott for the un­clear word­ing, as well any per­ceived slight against his very good rep­u­ta­tion.

For what­ever rea­son, COVID-19 seems to be a mag­net for re­vi­sion­ist his­tory and/​or wish­ful think­ing. In other com­ments un­der the same post, the no­tion that peo­ple from our ‘tribe’ did es­pe­cially well also comes un­der se­ri­ous ques­tion—in fact, it looks like many of the names men­tioned seem to have jumped on the band­wagon af­ter it was ob­vi­ous, and cer­tainly long af­ter the mar­ket was mov­ing.

The facts are that the mar­ket re­acted faster than al­most all of us. But not be­fore a few pre­scient peo­ple placed their bets!

So now the ques­tion be­comes: why didn’t the mar­ket re­act ear­lier than Fe­bru­ary 20, like those smart peo­ple did?

The null hy­poth­e­sis is that the mar­ket re­acted ex­actly ap­pro­pri­ately on the ba­sis of the in­for­ma­tion available. After all, there were other po­ten­tial pan­demics in the re­cent past that were suc­cess­fully con­tained or erad­i­cated.

On Fe­bru­ary 20, there were only 4 known cases in Italy. We were a long ways from the blood­bath that was com­ing. Maybe it was cor­rect to move cau­tiously un­til fur­ther in­for­ma­tion came in?

Here’s keaswaran:

EDIT: pre­vi­ously mis­at­tributed to AllAmer­i­canBreak­fast (who agrees with it).

[At that time] it may have been plau­si­ble for many peo­ple to think this would con­tinue to play out like SARS – East Asia would solve their prob­lem, ev­ery­one else would watch air­port ar­rivals and quaran­tine them effec­tively, and within a few weeks ev­ery­thing would sta­bi­lize and grad­u­ally go away. By Feb. 27 it was clear that this wasn’t hap­pen­ing, since com­mu­nity spread was very clear from the pub­lic data in Italy and Iran, and prob­a­bly also clear from ge­netic data in the United States and el­se­where.

So we reached a tip­ping point in those next few days, at which point, the mar­ket started re­spond­ing more vi­gor­ously.

If the null hy­poth­e­sis is true, then those early trades were not quite as pre­scient as they look. We might be mak­ing the mis­take of ‘re­sult­ing’, and con­fus­ing the re­al­ity we ended up in with all the oth­ers which were pos­si­ble at the time, in which those traders lost their shirts. It’s re­ally hard to have a use­ful ob­ject-level dis­cus­sion about this, be­cause these events are one-offs (this is the same ar­gu­ment we get into ev­ery year on Scott’s pre­dic­tions threads!) It’s not like we can run the ex­per­i­ment again, and thank good­ness for that.

Nev­er­the­less, Wei Dai sug­gested that this was the fi­nal nail in the coffin of EMH—at least for him.

I want to pause here to give mad props to Wei Dai for be­ing to­tally open about ev­ery­thing, and es­pe­cially for do­ing the fol­low­ing:

  • warn­ing that op­tions are dan­ger­ous, and you can eas­ily lose the lot

  • generic dis­claimer about seek­ing fi­nan­cial advice

  • up­dat­ing the thread af­ter he ended up los­ing 80% of his pa­per profits

  • men­tion­ing se­lec­tion bias, and say­ing we’d be right to dis­count his evidence

Which only leaves the ini­tial claim that “at least for me this puts a fi­nal nail in the coffin of EMH.”

This is a po­lite way of hint­ing that you might be a brilli­ant in­vest­ing wiz­ard with the power to beat the mar­ket. Hon­estly, af­ter mak­ing such a beau­tiful trade—and my gosh it re­ally was beau­tiful—whom amongst us could re­sist that temp­ta­tion? Cer­tainly not me. And any­way, it might even be true!

In mak­ing sense of claims of this na­ture, the first thing we have to es­tab­lish: what does it even mean to be able to beat the mar­ket?

Can Un­cle Ge­orge Beat the Mar­ket?

Un­cle Ge­orge re­ally likes his new iPhone. Man, these things are nifty! The danc­ing poop emoji is hilar­i­ous. On the strength of this in­sight, Ge­orge di­als his bro­ker and loads up on AAPL stock.

the cho­sen one! the scourge of effi­cient mar­kets! the stuff of eu­gene fama’s night­mares!

Over the next year, AAPL stock goes up 15 per cent, while the broader S&P 500 only goes up 10 per cent. Ge­orge be­comes in­suffer­able at fam­ily din­ners as he holds forth on his stock-pick­ing pow­ers. Guess the mar­ket isn’t so ‘effi­cient’ af­ter all, huh. Suck it, Eu­gene Fama!

So: did Un­cle Ge­orge beat the mar­ket?

In the nar­row­est pos­si­ble sense… yes.

In the sense in which we aim to string words to­gether so that they mean things: no, of course not. By this defi­ni­tion, ev­ery sin­gle trade leads to one of the two par­ties ‘beat­ing the mar­ket’. Millions of peo­ple beat the mar­ket while I wrote this sen­tence. I can flip a coin be­tween Pepsi and Coke right now, and have a 50 per cent chance of be­com­ing a mar­ket-beat­ing ge­nius.

The Un­cle Ge­orge ex­am­ple makes it glar­ingly ob­vi­ous that a suc­cess­ful trade does not some­how ‘break’ effi­cient mar­kets. And yet, this is the same naive crit­i­cism con­stantly lev­eled against the EMH: if the mar­ket moves in liter­ally any di­rec­tion, that must mean it was wrong be­fore! My cousin who sold/​bought be­fore it went up/​down beat the mar­ket!

Same goes for the Great Coron­avirus Trade. The fact that some peo­ple got out of the mar­ket early is not even the tiniest bit sur­pris­ing. In­vestors con­stantly think the mar­ket is go­ing to crash, for any num­ber of plau­si­ble rea­sons. This is the de­fault state of af­fairs: we have suc­cess­fully pre­dicted 73 of the last five mar­ket crashes, etc.

Th­ese pre­dic­tions are al­most always wrong:

And al­most all the peo­ple who make them would have been much bet­ter off tak­ing the bor­ing ‘buy and hold for­ever’ strat­egy.

But even a stopped clock is right twice a day. And of course, we’re much more likely to hear about the oc­ca­sional brilli­ant suc­cesses than the near-con­stant dull failures.

So the most naive crit­i­cism of the EMH boils down to ‘it’s pos­si­ble to make a good trade’. This is just a prop­erty of trad­ing. It tells us ex­actly noth­ing about the mar­ket’s effi­ciency, or lack thereof.

But some peo­ple re­ally do beat the mar­ket—and not in the triv­ial sense. They’re not merely stopped clocks, or highly visi­ble ‘sur­vivors’. I’ll sug­gest a defi­ni­tion later on which strips out the effect of ran­dom­ness.

Be­fore we get there—doesn’t the con­ces­sion that peo­ple can non-triv­ially beat the mar­ket already drive a stake through the EMH’s heart?

This is the sec­ond great mi­s­un­der­stand­ing: there is no con­flict be­tween the EMH and beat­ing the mar­ket. That’s how the mar­ket gets effi­cient! You find an in­for­ma­tion asym­me­try that isn’t priced in yet, and in ex­ploit­ing it, you move the mar­ket a lit­tle fur­ther to­wards effi­ciency.

Let’s call this in­for­ma­tion asym­me­try an ‘edge’.

If the EMH is true—or even just true-ish—that doesn’t mean the mar­ket can’t be beat. It means:

You shouldn’t ex­pect to beat the mar­ket with­out a unique edge, ex­cept by chance

Now, this usu­ally gets sim­plified down to ‘you can’t beat the mar­ket’. And most of the time, this sim­plifi­ca­tion is good enough: you might get lucky and win in the Un­cle Ge­orge sense, but over an in­vest­ing life­time, you’ll al­most cer­tainly re­vert to the mean (which isn’t match­ing the mar­ket re­turn—it’s un­der­perform­ing it).

But if you can find some kind of edge, you re­ally can win! So, what might a gen­uine edge look like?

Ano­ma­lies Ex­ist!

The Un­cle Ge­orges of the world don’t have an edge. All of their thoughts have already been thunk by some­one else (prob­a­bly by mil­lions of some­one el­ses). In­stead, their for­tunes are en­tirely at the mercy of the myr­iad other forces that drive stock prices: con­sumer de­mand, work­place ha­rass­ment scan­dals, money print­ers go­ing brrr, the ex­act viru­lence of a novel coro­n­avirus, the price of cheese in Spain last Fri­day af­ter­noon, etc.

All of this stuff—billions of in­puts pro­cessed by the great­est col­lec­tive in­tel­li­gence ever built—is a black box unto us mere mor­tals. It’s im­pos­si­ble to as­sign perfect causal ex­pla­na­tions to stock prices, which means we can pick whichever story suits us best. As a mar­ket re­porter, this was pretty much my whole job: call­ing bro­kers and economists to wrap a plau­si­ble nar­ra­tive around to­tally in­ex­pli­ca­ble events, and gen­er­ate sage nod­ding of heads.

All Un­cle Ge­orge can see is that he placed his bet, and AAPL went up. It was the poop emoji for sure!

And so, Un­cle Ge­orge spends his days dish­ing out hot stock tips on on­line fo­rums, oblivi­ous to the fact that his suc­cess was mean­ingless.

[un­cle georg­ing in­ten­sifies]

What does a real edge look like?

A cen­tury ago, in­vestors started notic­ing they could con­sis­tently pick up bar­gains by run­ning very sim­ple for­mu­las over stock prices. The most fa­mous is the ‘value in­vest­ing’ ap­proach de­vel­oped by Ben Gra­ham, and used by War­ren Buffett and Char­lie Munger. There was a gen­uine, big old in­effi­ciency in the mar­kets, and these guys had a great time ex­ploit­ing it.

I think this might be the image most peo­ple have in their head when they think of ‘beat­ing the mar­ket’—dili­gently study­ing The In­tel­li­gent In­vestor and learn­ing about PE ra­tios or what­ever.

But this is like try­ing to use a stone-age axe against a fighter jet. The Ben Gra­ham in­for­ma­tion asym­me­try has long since dis­ap­peared be­cause…mar­kets are effi­cient(ish)! Once the for­mula was widely known, it stopped work­ing. In­vestors de­vel­oped more so­phis­ti­cated ver­sions, more for­mu­las, more pric­ing mod­els. Once those got out, they stopped work­ing too. Now there’s a great de­bate as to whether even the most com­pli­cated de­scen­dants of value might be to­tally dead. In which case, the anomaly has offi­cially gone for good.

Either way, this is not how Buffett gets his edge, and it hasn’t been for decades. Here’s Munger:

The trou­ble with what I call the clas­sic Ben Gra­ham con­cept is that grad­u­ally the world wised up and those real ob­vi­ous bar­gains dis­ap­peared. You could run your Geiger counter over the rub­ble and it wouldn’t click.

Buffett’s most brilli­ant achieve­ment is weav­ing this folksy leg­end that he is a cute old grandpa who beats the mar­ket by back­ing the best com­pa­nies. Let’s take a look at how mar­ket-beat­ing in­vestors re­ally make their money.

Modern Edges are Com­pletely Ridiculous

great­est show­man on earth­

1. The War­ren Buffett Halo Effect

In re­cent decades, Buffett has made a kil­ling through juicy pri­vate deals which are com­pletely out of reach of the av­er­age in­vestor. Like, six billion dol­lar deals with three billion in prefer­ence rights and a guaran­teed div­i­dend. Like, lob­by­ing the gov­ern­ment to bail out the banks, then carv­ing off a huge piece of the ac­tion. Like, be­ing able to play around with Berk­shire Hath­away’s $115 billion in­surance float. Much of his for­tune is built on tax­payer largesse.

War­ren Buffett’s brand is so pow­er­ful that at this point, his suc­cess is a self-fulfilling prophecy: when Berk­shire in­vests in a stock, ev­ery­one else piles in af­ter him and drive the price up. Buffett even lends out his ‘halo’ to com­pa­nies that need it—most fa­mously dur­ing the GFC—so long as they give him a gen­er­ous dis­count to the mar­ket price, of course. (Matt Lev­ine has writ­ten some fas­ci­nat­ing columns about this).

And yet, and yet… Berk­shire Hath­away has un­der­performed for the last decade. Buffett would have been bet­ter off to take his own ad­vice and put it all in in­dex funds.

2. Hedge funds with armies of drones

There you are, sit­ting in your home office go­ing through Wal­mart’s quar­terly re­port and calcu­lat­ing PE ra­tios or what­ever. Mean­while, the pro­fes­sion­als are us­ing an army of drones to mon­i­tor the move­ment of shop­ping carts in Wal­mart park­ing lots in real time.

See also: send­ing foot sol­diers out to ev­ery branch of a bak­ery chain at the close of busi­ness each day, be­cause the num­bered dock­ets start out at zero, and thus con­tain live sales data un­available to the mar­ket.

And so, when renowned hedge fund man­ager Michael Stein­hardt was asked the most im­por­tant thing av­er­age in­vestors could learn from him, here’s what he sug­gested:

“I’m their com­pe­ti­tion.”

And yet, and yet…al­most all hedge funds un­der­perform. Not all of them are try­ing to beat the mar­ket, but the tools at their dis­posal gives us a sense of the difficulty here.

3. High-fre­quency traders move mountains

If mul­ti­ple peo­ple have ac­cess to the same in­for­ma­tion, the speed in which you can bring it to mar­ket also mat­ters. So, we have high-fre­quency traders.

One firm spent $300 mil­lion lay­ing a di­rect ca­ble be­tween Chicago to New Jersey. They cut straight through moun­tains and crossed rivers. The ca­ble stretched 1331 kilo­me­tres. And they did this to shave four mil­lisec­onds off their trans­mis­sion time.

And yet, and yet…microwaves came along and ren­dered the whole pro­ject ob­so­lete. Try­ing to get an edge is ex­pen­sive.

4. Be­ing will­ing and able to com­mit felonies

In­sider trad­ing is a thing. See also: crim­i­nals who hack or oth­er­wise steal sen­si­tive pri­vate in­for­ma­tion.

And yet, and yet…even when crim­i­nals have ad­vance ac­cess to earn­ings re­ports, they still don’t do all that well, which is ev­i­dence for the very strongest form of the EMH (the one that no-one, in­clud­ing me, be­lieves can pos­si­bly be true).

So…what sort of edge do us lesser mor­tals have?

If we mum­ble some­thing about hav­ing ‘good in­tu­ition’, or ‘sub­scribing to the Wall Street Jour­nal’ then we should con­sider the strong pos­si­bil­ity that we are Un­cle Ge­orge.

If the an­swer in­volves ‘fun­da­men­tal anal­y­sis’ or ‘Fibonacci re­trace­ments’, we’re still in Un­cle Ge­orge ter­ri­tory. The only differ­ence is that do­ing some­thing com­pli­cated makes it eas­ier to in­ter­nally jus­tify the be­lief that we know a se­cret no-one else does. But it is still (prob­a­bly!) a mis­taken be­lief.

The EMH Gets Stronger With Every Attack

So we know for sure that mar­ket-beat­ing edges ex­ist—I’ve even writ­ten them down for ev­ery­one to see!

I can only dream of pos­sess­ing a halo effect so strong that ev­ery­one piles into a stock right af­ter I an­nounce I have graced it with my favour. I don’t have an army of drones at my com­mand, or the abil­ity to bore through moun­tains to shave mil­lisec­onds off my trad­ing times, or a weekly round of golf with the CEO of a For­tune 500 com­pany.

The mar­ket is never perfectly effi­cient. But rel­a­tive to me, it might as well be.

Crit­ics have pointed out plenty of cases in which the EMH doesn’t jive with re­al­ity—and they are ab­solutely right. So this is where it gets re­ally weird.

The EMH is the only the­ory that grows stronger with ev­ery at­tack against it.

Every edge is con­stantly at risk of be­ing gob­bled up by an effi­cient-ish mar­ket. The ones I’ve men­tioned can only be pub­li­cly knowl­edge be­cause they’re some­what defen­si­ble: they’re based on per­sonal re­la­tion­ships, cap­i­tal in­vest­ment, pro­pri­etary tech­nol­ogy, etc. But they dis­ap­pear too.

Most edges can’t even be spo­ken out loud with­out dis­ap­pear­ing. If stocks sys­tem­at­i­cally rise on the third Thurs­day of each month but only un­der a wax­ing moon, and then some­one writes about it in pub­lic, you can kiss that anomaly good­bye. The EMH sucks it into its gi­gan­tic heav­ing maw, and it’s gone for­ever.

In other words: ev­ery time some­one picks a hole in the the­ory and points out an in­effi­ciency, they make the pre­dic­tions gen­er­ated by the EMH more ro­bust! It’s like some freaky shog­goth thing that Just. Won’t. Die.

you may not like it, but this is what peak effi­ciency looks like

Which gets us to the to­tally jus­tified crit­i­cism of the the­ory: the only rea­son the EMH can pull this stunt is be­cause it’s bul­lshit sci­ence.

It’s un­falsifi­able! It re­sponds to crit­i­cism by say­ing, ‘OK, good point, but now that I’ve fac­tored that in, you should be­lieve in my the­ory even more.’

And…we re­ally should?

The only way I can think about the EMH with­out go­ing in­sane is to re­mind my­self that it gen­er­ates a use­ful heuris­tic. It’s not a sta­ble law, like we might find in hard sci­ences. It’s not perfectly ac­cu­rate. At any given point in time, there are always com­pet­ing mod­els that do a bet­ter job of de­scribing re­al­ity. But all those other mod­els can stop work­ing at any mo­ment, with no warn­ing! By the time you find out their pre­dic­tive power is gone, it’s too late, and you prob­a­bly lost a bunch of money! By con­trast, the EMH is a re­li­able model—re­li­ably vague and hand-wavy, yes, but also re­li­ably use­ful.

We know there are in­effi­cien­cies in the mar­ket. In the ful­l­ness of time, they will be ab­sorbed into the gelat­i­nous alien-god’s hive­mind. But be­fore that hap­pens, maybe we can make some money off of them.

So now we come to the fi­nal test. How do you tell if you’ve re­ally found a mar­ket-beat­ing edge—that is, a model of re­al­ity that has more pre­dic­tive power than the EMH—or you’re fool­ing your­self like Un­cle Ge­orge?

If You’re so Smart, Why Aren’t You Rich?

Every­one knows a se­cret about them­selves, or the peo­ple they know well, or can ar­bi­trage some op­por­tu­nity in a niche that few peo­ple are pay­ing at­ten­tion to. Th­ese illiquid pri­vate ‘mar­kets’ are much more fer­tile hunt­ing grounds for asym­me­tries, and some­thing I en­courage ev­ery­one to think about.

But the pub­lic se­cu­rity mar­kets are a gi­gan­tic ag­glomer­a­tion of ev­ery­one’s pre­dic­tions, which con­stantly hoovers up ev­ery new frag­ment of in­for­ma­tion, and re­cal­ibrates it­self in real time. Your challenge is to try and pre­dict why this gi­ant meta-pre­dic­tion is wrong, and in which di­rec­tion.

If you think you can re­li­ably beat, say, an in­dex fund that pas­sively tracks the S&P 500, this is a much stronger claim than it first ap­pears.

For one thing, you’re claiming to be bet­ter than War­ren Buffett, who has failed to pull this off in the last 10 years, de­spite his huge ad­van­tages, and has started say­ing the game is so hard that ev­ery­one should just buy in­dex funds. But that’s noth­ing. You are also claiming that you have the power to beat the great­est col­lec­tive in­tel­li­gence hu­man­ity has ever cre­ated.

This is an ex­traor­di­nary claim, and the thing about ex­traor­di­nary claims is that they re­quire ex­traor­di­nary ev­i­dence.

Un­cle Ge­orge’s AAPL trade ain’t go­ing to cut it. Here is the ex­traor­di­nary ev­i­dence that I would per­son­ally want to see be­fore agree­ing that an in­vestor can beat the mar­ket:

1. Big heaps of money

This is the one area of life where there re­ally is no dodg­ing that most ven­er­a­ble of sick burns: if you’re so smart, why aren’t you rich?

So the first piece of ev­i­dence I would ac­cept is the fact that some­one is very, very rich. Sit­ting atop a big old pile of cash. And they’d prob­a­bly also open a hedge fund so they can take other peo­ple’s money and turn it into mil­lions more.

2. Track record of outperformance

Maybe a gen­uine mar­ket-beater doesn’t have enough start­ing cap­i­tal to make big piles of money on a rel­a­tively slim edge, and for some rea­son is un­able to come up with any scheme to beg or bor­row more? Or the anomaly is real, but dis­ap­pears be­fore they can get filthy rich?

In these sce­nar­ios, I would also ac­cept a com­plete record of out-of-sam­ple in­vest­ment re­turns over time—no back­tests! no se­lect­ing the best trades!—as com­pared against the ap­pro­pri­ate risk-ad­justed bench­mark.

Th­ese ev­i­den­tial stan­dards work both in the case that the EMH is ‘dead’, i.e. you can re­li­ably beat the mar­ket us­ing pub­lic do­main info which ought to already be priced in, and in the case that it’s not, i.e. you re­ally do need novel in­for­ma­tion to get an edge.

As for ev­i­dence that the EMH re­ally is dead…hmm. It’s not a proper the­ory to be­gin with. But I guess it would be ‘dead’ when the pre­dic­tions it gen­er­ates stop be­ing ac­cu­rate or use­ful? Which would look some­thing like: ‘find­ing out that plenty of peo­ple meet the stan­dards above, de­spite hav­ing never been in pos­ses­sion of a scrap of in­for­ma­tion that wasn’t already available to the mar­ket’.

In do­ing so, we’d have to be very care­ful to make sure we aren’t just look­ing at the Un­cle Ge­orges who un­wit­tingly drew the win­ning lot­tery ticket. After all, we should ex­pect plenty of in­vestors to beat the mar­ket for a long time—even for years on end—en­tirely by chance.

The Great Coin-Toss­ing Experiment

Say we held a na­tional coin-flip­ping con­test. After 15 rounds, one in ev­ery ~32,800 peo­ple would have man­aged to call ev­ery sin­gle toss cor­rectly, perfectly pre­dict­ing a se­quence like this:

H T H H T T H H H H T H T T T

Pretty im­pres­sive, huh!

Well, only in a world where we don’t know about prob­a­bil­ity. In that world, we might mis­take blind ran­dom­ness for skill. The lucky few win­ners would be hailed as the heroes of their home­towns, do in­ter­views with breath­less break­fast TV hosts, and ex­plain that it’s all about the pre­cise flick of the wrist. Aspiring flip­pers would queue up to buy the in­evitable best-sel­l­ing book, Flip Me Off, and pay ex­or­bitant sums for one-on-one coach­ing ses­sions with the mas­ter tossers.

De­press­ingly, this is ex­actly what hap­pens in the world of in­vest­ing. What does it mean to achieve the kind of suc­cess which only hap­pens by chance with 0.0003 prob­a­bil­ity? In the United States alone, it means you end up with 10,000 lucky dopes who are in­dis­t­in­guish­able from brilli­ant in­vestors.

And fund man­agers don’t need to do any­where near that well to at­tract a mar­ket-beat­ing aura. They’re in­cen­tivised to swing for the fences, in­creas­ing the odds they beat the mar­ket in some highly visi­ble fash­ion over some shorter pe­riod—say, a lucky sea­son or two. They in­evitably regress to the mean, some­times crash­ing and burn­ing in spec­tac­u­lar fash­ion, but it doesn’t mat­ter so long as they man­age to hose naive in­vestors in the mean­time.

We can never en­tirely rule out the effect of ran­dom­ness—there will always be some tiny chance that War­ren Buffett is re­ally just the world’s great­est coin-flip­per—but we have to draw the line some­where, or the stan­dard is im­pos­si­bly high.

Once the odds of a fluke get pretty slim—some­one is su­per duper rich, and they’ve made a ton of con­sis­tently good trades over time—I’d hap­pily con­grat­u­late them on their mar­ket-beat­ing prowess, give them all my money to in­vest, listen ea­gerly to their ad­vice, etc.

Be­ing good Bayesi­ans, this is ob­vi­ously a spec­trum: if they are not at that point, but trend­ing in the right di­rec­tion, I would be less skep­ti­cal, etc. But the bar has to be pretty high, or there’s no way to sep­a­rate skill from ran­dom­ness.

Scott and Eliezer have both al­luded to their com­ments be­ing in­formed by pri­vate in­for­ma­tion. Here’s a red­dit com­ment in which Scott re­sponds to crit­i­cism of his ‘EMH is the real vic­tim’ riff:

I think we’re in an asym­met­ric po­si­tion, in that I know these peo­ple pretty well, I know they’ve thought about effi­cient mar­ket be­fore, and they’re the sort of peo­ple I would ex­pect to beat the mar­ket if any­one could. I agree that if you just hear some blog­ger say he saw some peo­ple beat the mar­ket once, that’s not much ev­i­dence.

Eliezer definitely un­der­stands the EMH, be­cause the de­scrip­tions of it in Inad­e­quate Equil­ibria are among the most brilli­ant and in­sight­ful I’ve ever come across. And Scott is ob­vi­ously su­per smart.

I would love to know what their ev­i­den­tial stan­dards are, but I’m ex­plic­itly not call­ing them out, or any of the peo­ple men­tioned ear­lier in the post. No-one is un­der the slight­est obli­ga­tion to share pri­vate ev­i­dence, and I would be thrilled if those folks were in­deed the mar­ket-beat­ing cho­sen ones.

But I am say­ing that peo­ple, in gen­eral, make these kind of claims all the time—in good faith and with no mal­i­cious in­tent—and in gen­eral, tak­ing their ad­vice is an ex­traor­di­nar­ily bad idea.

A heuris­tic: if you (or some­one you know) is con­fi­dent they can beat the mar­ket, and yet you no­tice you are not sit­ting atop an enor­mous pile of wealth, it’s at least worth con­sid­er­ing the pos­si­bil­ity that you might be fool­ing your­selves.

The Four Types of Investors

There are very ob­vi­ous and well-known rea­sons why ev­ery­one loves to think they can beat the mar­ket: over­con­fi­dence, con­fir­ma­tion bias, ‘re­sult­ing’, se­lec­tive mem­ory, sur­vivor­ship bias, etc.

Th­ese forces are so pow­er­ful that many peo­ple—my­self in­cluded—blithely ig­nore the vast piles of ev­i­dence that sug­gest beat­ing the mar­ket is in­cred­ibly difficult, and go ahead and try any­way. All of us think we are spe­cial, and (al­most) all of us are wrong.

Even if we don’t per­son­ally har­bour this par­tic­u­lar fan­tasy, there’s also a nat­u­ral ten­dency to want our tribe or our friends to be the brilli­ant vi­sion­ar­ies who were ahead of the ac­tion, pos­sess sweet mar­ket-beat­ing skills, etc.

So we can roughly place in­vestors into one of the fol­low­ing four quad­rants:

(‘Losers’ and ‘win­ners’ here is tongue-in-cheek, and not a value judg­ment: liter­ally, los­ing/​win­ning this game by ei­ther suc­cess­fully beat­ing the mar­ket, or failing to do so)

De­luded losers (‘Suck­ers’)

“Ap­ple stock re­ally is un­der­val­ued, but the mar­ket hasn’t recog­nised it yet. I just got un­lucky—it was be­cause of [elab­o­rate ra­tio­nal­iza­tion]. Also, even if I got it wrong this time, I was re­ally close. Next time!

“What’s that? Do I track my port­fo­lio re­turns over time, and com­pare against the rele­vant risk-ad­justed bench­mark to see whether I’m ac­tu­ally out­perform­ing? Well, there’s no need. I usu­ally do pretty well for my­self, and I’m ex­pect­ing to im­prove—in fact, I just picked up this clas­sic book called The In­tel­li­gent In­vestor…

De­luded win­ners (‘Dumb Luck’)

“I knew Ap­ple stock was un­der­val­ued! And I re­mem­ber that other time I made a re­ally good trade, too. Guess I’m pretty good at this game!

“…What’s that? I might have just got lucky? Hah, no. I even did the Fibonacci re­trace­ments and ev­ery­thing.”

Real­is­tic losers (‘Clear-Eyed Fools’)

“I keep a metic­u­lous record of my port­fo­lio re­turns, which forces me to ac­knowl­edge the fact that even though I oc­ca­sion­ally do well, I am un­der­perform­ing my bench­marks on a risk-ad­justed ba­sis. I am un­der no delu­sions about my prospects of find­ing an edge, and I know I re­ally ought to take War­ren Buffett’s ad­vice and put all my money in in­dex funds.

“But I en­joy play­ing the mar­kets! It’s like how a night in Ve­gas can have nega­tive EV but still be pos­i­tive util­ity, be­cause of all the non-fi­nan­cial fac­tors. So I’m gonna keep gam­bling with a small part of my port­fo­lio, just for shits and gig­gles. In the event that I ‘win’, I will try re­ally hard to re­sist the in­cred­ible in­ter­nal pres­sure to start think­ing of my­self as a brilli­ant in­vest­ing guru.”

Real­is­tic win­ners (‘Cho­sen One’)

“I keep a metic­u­lous record of my port­fo­lio re­turns, which have out­performed the ap­pro­pri­ate risk-ad­justed bench­marks to such a de­gree that I am con­fi­dent I have found a gen­uine in­for­ma­tional asym­me­try. I will of course never tell any­one about it, or it will be­come use­less.

“And I can never be en­tirely sure: it’s also pos­si­ble that I just got lucky. But at the very least, I am sit­ting atop great piles of money, which is pretty nice.”

***

The vast ma­jor­ity of peo­ple who ac­tively trade their ac­count are ‘Suck­ers’. Some smaller num­ber fall into the ‘Dumb Luck’ quad­rant (Un­cle Ge­orge would stay there if he never places an­other trade, but he al­most cer­tainly won’t be able to help him­self.)

The right-hand quad­rants are much more sparsely pop­u­lated. I guess there are a few ‘Clear-Eyed Losers’ float­ing around, and a tiny hand­ful of ‘Cho­sen Ones’.

This rough dis­tri­bu­tion is prob­a­bly not too con­tro­ver­sial. The ques­tion is, which one are you?

(I made a poll on my web­site at this point in the post, just for a bit of fun: the re­sults so far are brilli­ant)

Try­ing to Beat the Mar­ket is Like Crack for Smart People

There is a ten­dency for smart peo­ple to wan­der into ar­eas they know very lit­tle about, and think they can do bet­ter than the ac­tual ex­perts who have years or decades of do­main-spe­cific knowl­edge, on the ba­sis of be­ing very smart, or hav­ing read some blog posts on­line about be­ing more ra­tio­nal.

This would be OK if it was just a bit cringe. I love arm­chair pon­tif­i­cat­ing as much as the next guy! The con­se­quences are usu­ally limited to mildly an­noy­ing the peo­ple who ac­tu­ally know what they’re talk­ing about, and much eye-rol­ling when you triumphantly rein­vent the wheel.

There is some up­side too: rein­vent­ing the wheel is fun, be­cause you get to, like, in­vent wheels. And very oc­ca­sion­ally, it might even be true! No doubt smart out­siders are oc­ca­sion­ally able to breeze into a new field and ex­ploit some ob­vi­ous in­effi­cien­cies.

But…oh boy. It’s re­ally not true of this par­tic­u­lar do­main. And it’s not harm­less ei­ther.

The cen­tral pre­dic­tion gen­er­ated by the EMH is that you should not ex­pect to be able to beat the mar­ket (in the non-triv­ial sense) un­less you have unique in­for­ma­tion or some similar edge.

This pre­dic­tion is tested ev­ery day. We have great piles of ev­i­dence which sug­gest that it is cor­rect: the vast ma­jor­ity of ac­tive in­vestors do re­ally badly.

Cru­cially, it’s not only reg­u­lar schmucks who un­der­perform. So do paid pro­fes­sion­als, and ac­tive man­agers, and hedge funds, and all sorts of brilli­ant peo­ple who have made this their life’s work.

It’s pos­si­ble that I am not mak­ing many friends with this post. I cer­tainly feel pretty ner­vous about pub­lish­ing it. Every­one who thinks they can beat the mar­ket will have their hack­les up! If it helps at all, I am not claiming the high ground. I have made ev­ery dumb in­vest­ing mis­take you could think of, and then a few more be­sides. I am painfully aware of how hard this is.

Th­ese days I would put my­self in the ‘Clear-Eyed Fool’ quad­rant, but only by a finger­nail. It’s a con­stant bat­tle even to stay there. I still do clever things that con­tra­dict my own bor­ing ad­vice, and an­noy­ingly, am re­warded for my hubris just of­ten enough to start en­ter­tain­ing the thought that I’m a brilli­ant in­vest­ing ge­nius af­ter all. Then I force my­self to calcu­late the IRR on my pub­li­cly-traded in­vest­ments, and com­pare it against ap­pro­pri­ate bench­marks, and man­age to get a finger­nail-hold back on bor­ing old re­al­ity.

To the ex­tent that I have suc­ceeded as an in­vestor, and I am do­ing quite nicely thank you, it has only come through forc­ing my­self to ac­knowl­edge the main pre­dic­tion that emerges from the very-much-al­ive-and-kick­ing EMH. The huge and un­der­ap­pre­ci­ated benefit of do­ing so is that I oc­ca­sion­ally di­vert some of my at­ten­tion el­se­where, to do­mains where I ac­tu­ally do have an edge—and then I win.


Discussion

The EMH is a weird, coun­ter­in­tu­itive, freaky-ass shog­goth of a thing and it still con­fuses the heck out of me, even af­ter al­most a decade of writ­ing about fi­nance. I al­most cer­tainly made some mis­takes in the post—in the pro­cess of writ­ing it, I no­ticed sev­eral ways in which my ini­tial be­liefs were sub­tly wrong, which has already been su­per use­ful for me, and helped me un­der­stand some of the (valid) ob­jec­tions peo­ple have raised.

So I would also like to use this post to open up the floor to any and all EMH dis­cus­sion, and try to benefit from the smaller-but-still-pow­er­ful col­lec­tive in­tel­li­gence that is Less Wrong.

I’m go­ing to add com­ments with some of my own ques­tions and un­cer­tainty. It would be nice to be­come less con­fused to­gether, and try to get a bet­ter sense of where we should ap­ply our efforts at the mar­gin.