Prediction Markets: When Do They Work?

Epistemic Sta­tus: Res­i­dent Expert

I’m a lit­tle late on this, which was an old promise to Robin Han­son (not that he asked for it). I was mo­ti­vated to deal with this again by the launch of Augur (REP), the crypto pre­dic­tion mar­ket to­ken. And by the crypto pre­dic­tion mar­ket to­ken, I mean the empty shell of a po­ten­tial fu­ture pre­dic­tion mar­ket to­ken; what they have now is pretty ter­rible but in crypto world that is oc­ca­sion­ally good for a $300 mil­lion mar­ket cap. This is, for now, one of those oc­ca­sions.

The biggest mar­ket there, by far, is on whether Ether will trade above $500 at the end of the year. This is an in­ter­est­ing mar­ket be­cause Augur bets are made in Ether. So even though the mar­ket (as of last time I checked) says it’s 74% per­cent to be trad­ing above $500 and it’s cur­rently $480 (it’s cur­rently Thurs­day on July 26, and I’m not go­ing to go back and keep up­dat­ing these num­bers). When I first saw this the mar­ket was at 63%, which seemed to me like a com­plete steal. Now it’s at 74%, which seems more rea­son­able, which means the first ‘offi­cial DWATV trad­ing tip’ will have to wait. A shame!

A bet­ter way to ask this ques­tion, given how close the price is to $500 now, is what the ra­tio of ‘given Ether is above $500 what does it cost’ to ‘given Ether is be­low $500 what does it cost’ should be. A three to one ra­tio seems plau­si­ble?

The weak­ness (or twist) on mar­kets this im­plies ap­plies to pre­dic­tion mar­kets gen­er­ally. If you bet on an event that is cor­re­lated with the cur­rency you’re bet­ting in, the fair price can be very differ­ent from the true prob­a­bil­ity. It doesn’t have to be price based – think about bet­ting on an elec­tion be­tween a hard money can­di­date and one who will print money, or a pre­dic­tion on a nu­clear war.

If I bet on a nu­clear war, and win, how ex­actly am I get­ting paid?

Robin Han­son, Eliezer Yud­kowsky and Scott Sum­ner are big ad­vo­cates of pre­dic­tion mar­kets. In the­ory, so am I. Pre­dic­tion mar­kets are a won­der­ful thing. By giv­ing peo­ple a mon­e­tary in­cen­tive to solve prob­lems and share in­for­ma­tion, we can learn prob­a­bil­ities (what will GDP be next year?) and con­di­tional prob­a­bil­ities (what will GDP be next year if we pass this tax cut bill?) and use the an­swers to make the best de­ci­sion. This method of mak­ing de­ci­sions is called futarchy.

For­mally, a pre­dic­tion mar­ket al­lows par­ti­ci­pants to buy and sell con­tracts. Those con­tracts then pay out a vari­able amount of money. Typ­i­cally this is ei­ther bi­nary (will Don­ald Trump be elected pres­i­dent?), pay­ing out 100 if the event hap­pens and 0 if it doesn’t, or they are con­tin­u­ous (how many elec­toral col­lege votes will Don­ald Trump get?) and pay pro­por­tion­ally to the an­swer. Some­times there are spe­cial cases where the mar­ket is void and all trans­ac­tions are un­done, at other times strange cases have spe­cial logic to de­ter­mine the pay­out level.

There are three types of pre­dic­tion mar­kets that have got­ten non-zero trac­tion.

The first is poli­tics. There are mar­kets at Pre­dic­tIt and BetFair and Pin­na­cle Sports, and there used to be rel­a­tively deep mar­kets at InTrade. Th­ese mar­kets mat­ter enough to get talked about and at­tract some money when they in­volve ma­jor events like pres­i­den­tial elec­tions, but tend to be quite pa­thetic for any­thing less than that.

The sec­ond is eco­nomics. There are lots of stocks and fu­tures and op­tions and other such prod­ucts available for pur­chase. Fu­tures mar­kets in par­tic­u­lar are pre­dic­tion mar­kets. They don’t call them­selves pre­dic­tion mar­kets, but that is one of the things they are, and the in­for­ma­tion they re­veal is in­valuable. It’s even some­times used to make de­ci­sions.

The third is sports. Most tele­vised sport­ing events have book­mak­ers offer­ing odds and tak­ing bets. They use their own ter­minol­ogy for many things, but these are the clos­est thing to true pre­dic­tion mar­kets out there.

What makes a suc­cess­ful pre­dic­tion mar­ket? What makes an un­suc­cess­ful pre­dic­tion mar­ket? When are they effi­cient? What gets peo­ple in­volved?

To get a thriv­ing mar­ket, you need (at least) these five things.

I

Well-Defined

If you can’t ex­actly define the out­come, you can’t have a pre­dic­tion mar­ket. Even highly un­likely cor­ner cases must be re­solved. Thus, if you want a mar­ket on “Don­ald Trump is elected pres­i­dent of the United States in 2020” you need to know ex­actly what hap­pens if he dies af­ter the elec­tion but be­fore inau­gu­ra­tion, or if there is a re­volt in the elec­toral col­lege, or if the elec­tion is fraud­u­lent or can­cel­led, or if he loses to a differ­ent per­son also named “Don­ald Trump.” That’s not be­cause Trump makes such is­sues more likely. If you were bet­ting on Obama vs. Rom­ney, you’d need to do all the same stuff.

In sports mar­kets, this means writ­ing up a multi-page doc­u­ment de­tailing what hap­pens when a game is rained out, dis­puted, post­poned, tied, you name it, along with all the other rules. If there’s an an­gle left am­bigu­ous, you can bet (well you can’t, but if you could, you’d have good odds) that some­one will try to take ad­van­tage of it even­tu­ally. That leaves ev­ery­one mad and ru­ins good busi­ness re­la­tion­ships. It’s im­por­tant to have clear rules and stick to them.

One of the first mar­kets on Augur asked, “Will England defeat Croa­tia in the World Cup?” Which I im­me­di­ately rec­og­nized as a re­ally bad word­ing, be­cause it was am­bigu­ous. If the game had gone to over­time, or even to penalty kicks, and England had ad­vanced, what hap­pens? In a real sports­book, gen­er­ally bets de­fault to reg­u­la­tion time only, so the match would be ruled a draw, and the an­swer would be “No” even if England later won.

That’s not ac­cept­able. All it takes is one cor­ner case to get peo­ple yel­ling at each other, and drive them away. When they do hap­pen, the sports­book is wise to eat the loss, even if that means pay­ing both sides, and then fix its pro­ce­dures. That’s one benefit of hav­ing a cen­tral au­thor­ity to blame.

I’m also in­clud­ing in this the re­quire­ment that you can be con­fi­dent you can col­lect if you win. As one sports­book’s slo­gan puts it, sweat the game, not the pay­out. Bets with peo­ple who might not pay you re­quire huge edges. They’re about ac­cept­ing the risk to land the whale. They don’t do much for price dis­cov­ery, and are for pro­fes­sion­als only.

II

Quick Resolution

The faster the mar­ket pays out, the more in­ter­est you’ll get. Mar­kets that tie up money for weeks or months, let alone years, see de­ci­sive de­clines in par­ti­ci­pa­tion. Be­fore the sea­son, there are mar­kets for which teams will win the cham­pi­onship, or how many wins each team will have. This seems great, since if you’re right you can have a huge ad­van­tage, and it gives you an in­ter­est in games for much or all of the sea­son.

De­spite that, you will see less money wa­gered on any given sea­son win to­tal than for a ran­dom game played by that team. Usu­ally by an or­der of mag­ni­tude. That’s how valuable quick re­s­olu­tion is. Even in March Mad­ness, where ev­ery­one fills out office brack­ets, the bulk of the real wa­ger­ing goes game by game. There are lots of propo­si­tions, and there’s value there, but only for small amounts. Bet­ting your bracket be­fore the tour­na­ment, de­spite brack­ets be­ing some­thing pres­i­dents do to show they’re in touch, isn’t a thing for se­ri­ous money.

Thus, mar­kets on small events like in­di­vi­d­ual games are big­ger, and more effi­cient, than mar­kets on big­ger and more in­ter­est­ing things like en­tire sea­sons or even a play­off se­ries. The pri­mary pur­pose of the long-term mar­kets isn’t to make money; it’s to provide a ser­vice so peo­ple can see what odds have been as­signed to var­i­ous out­comes.

Ma­jor events like pres­i­den­tial elec­tions have enough in­her­ent in­ter­est to still see solid mar­kets, but only barely. There’s a lot of in­ter­est in what the odds are, but the vol­umes traded are quite thin, so much so that it is in the in­ter­est of par­ti­sans to trade in or­der to move the price and thus change the poli­ti­cal nar­ra­tive.

Eco­nomic mar­kets are the only place longer-term mar­kets pros­per.

Note that if the mar­ket is suffi­ciently liquid, it can act as if it is short term, pro­vided the prices will move quickly enough, since par­ti­ci­pants can then exit their po­si­tions.

III

Prob­a­ble Resolution

Trad­ing in a pre­dic­tion mar­ket ties up cap­i­tal, cre­ates risk and re­quires op­ti­miza­tion pres­sure. I need to pay at­ten­tion to the mar­ket, both to de­cide what fair value is and then to go about max­i­miz­ing and mak­ing good trades.

If that mar­ket is con­di­tional, and trades were only valid if those con­di­tions were met, we have a prob­lem: I’ve wasted my time, money and risk ca­pac­ity, and got­ten noth­ing in re­turn.

One of the mar­kets I liked a lot as a gam­bler was called the Home/​Away line in MLB. The idea was, you added up all the runs scored by the home teams and com­pared them to all the runs scored by the away teams, and bet on which would be higher, or on the sum of all runs scored that day, which was called The Grand Salami. There was lots of value in these lines be­cause peo­ple were us­ing very sim­ple heuris­tics, and if you did first-level statis­tics on runs scored in games and how dis­tri­bu­tions add up, you could get a big edge.

What was con­tin­u­ously frus­trat­ing was that of­ten one game would get rained out, can­cel­ling all your wa­gers. Often I’d have locked in large prof­its, and they’d be lost.

This wasn’t enough to keep me from bet­ting, be­cause I got my money back within a day and the edge was huge. But when funds were tight, it shifted those funds to­wards other things, and ev­ery time I thought about look­ing at the Home/​Away line, my brain fired back ‘are you sure you want to bother?’ so I only cared when my edge was large.

Gam­blers ac­tively pre­fer bet­ting on odds that can’t tie, e.g. bet­ting on a foot­ball team −3.5 or −2.5 rather than −3.0, be­cause the −3 line ties 10% of the time. The book­maker agrees!

If you are in­stead ty­ing up your money for weeks, months or even years, and in­stead of a 10% chance of rain some­where there’s a 50% or even 90% chance the event doesn’t fire, that’s much worse. If your’e deal­ing with a hy­per-com­plex Han­so­nian death trap of a con­di­tional mar­ket where it’s 99%+ to not hap­pen, even with good risk mea­sure­ment tools that don’t tie up more money than nec­es­sary, no one is go­ing to want to put in the work and tie up the funds.

IV

Limited Hid­den Information

In­sider trad­ing of se­cu­ri­ties is ille­gal. This seems at odds with price dis­cov­ery. If I know some­thing you don’t know, then my not trad­ing on it makes the price less ac­cu­rate. One might sug­gest that al­low­ing in­sid­ers to trade would make the price more effi­cient.

The prob­lem is that it drives peo­ple away.

If other peo­ple know some­thing im­por­tant I don’t, then my trades are giv­ing them a way to pick my pocket. When I look at the price and see it is wrong, my prior is ‘there is some­thing I don’t know’ rather than ‘there is some­thing they don’t know or un­der­stand’. I’m the one mak­ing the mis­take. I’m the sucker. So I walk away.

Thus, while the in­di­vi­d­ual trades of in­sid­ers make the mar­ket more effi­cient, they pun­ish oth­ers try­ing to share their in­for­ma­tion and anal­y­sis with the mar­ket. This is bad. Bad enough to kill out­right mar­kets with too much risk of in­sider in­for­ma­tion.

The first sea­son of Sur­vivor, there was a mar­ket on who would win. The pro­duc­tion crew found out. Then there was no mar­ket.

Another im­por­tant case: If a per­son with a large role in choos­ing the out­come can bet in the mar­ket, you might not want to risk bet­ting against him. Or bet at all.

When there is a big in­jury risk in a game, the mar­ket dies un­til the is­sue is re­solved. When the is­sue is re­solved, trad­ing picks back up no mat­ter the out­come.

Even re­duc­tion of un­cer­tainty as such can be im­por­tant. Be­fore im­por­tant events like elec­tions of­ten money will ‘sit on the sideline’ un­til the out­come is known. This can even re­sult in bad out­comes driv­ing prices up. We may not like the new boss, but at least we now know who he is and can go about our busi­ness.

In my ex­pe­rience with pre­dic­tion mar­kets, im­por­tant hid­den in­for­ma­tion other traders could know acts as an out­right veto on the mar­ket. It might not do that if the mar­ket had enough ‘nat­u­ral’ trad­ing vol­ume, but that’s a high bar to clear.

V

Sources of Disagree­ment and Interest

Also known as, Suck­ers at the Table.

Any mar­ket, like a poker table, re­quires sources of dis­agree­ments and prof­its. Without a sucker at the table, why par­ti­ci­pate in the mar­ket? Re­mem­ber, if you can’t spot the sucker in your first half hour at the table, then you are the sucker.

Ideally, you want ei­ther a di­rect sub­sidy to the mar­ket, or nat­u­ral buy­ers and sel­l­ers.

If some­one has a rea­son to trade even at a not so great price, for ex­am­ple air­lines or coun­tries hedg­ing against moves in oil prices, then ev­ery­one can com­pete to make money off of that. The same would go if some­one wanted to hedge against a poli­ti­cal event, or to bet for or against their fa­vorite team on prin­ci­ple – ei­ther to make it in­ter­est­ing, or to get what I used to half-jok­ingly call ‘com­pen­sa­tion for our pain’ when the Mets in­evitably lost again.

Another class of ‘nat­u­ral’ traders are gam­blers or noise traders, who de­mand liquidity for no par­tic­u­lar rea­son. They too can be the sucker.

If peo­ple who want to learn fair prob­a­bil­ities sub­si­dize the mar­ket, like donors sub­si­dize the NGDP fu­tures mar­kets Scott Sum­ner helped cre­ate, that also works.

And of course there’s always the peo­ple who think they know some­thing, and are sadly mis­taken.

What traders need, more than any­thing else, is the abil­ity to tell a story for why their trade is a good idea. To do that, they need to know why they have the op­por­tu­nity to make this trade. What do they know that oth­ers don’t know? What mis­take do they think peo­ple are mak­ing?

For sports, poli­tics and eco­nomics, ev­ery­one has an opinion, so it is easy for them to get the idea that they have the ad­van­tage.

The ge­nius of a bi­nary bet on Ether prices that trades in Ether is that there are a lot of an­gles where one can think you know some­thing the mar­ket doesn’t fully know, and lots of mis­takes you can think other peo­ple are mak­ing. It’s easy to think of many differ­ent an­gles and ap­proaches one could take. One can trade short term, or trade long term, do ar­bi­trage or use it for lev­er­age. Another could be do­ing it as a form of speech, or an ex­per­i­ment, and the group that can reach the mar­ket is doubtless quite bi­ased.

It’s easy to make that leap to ‘I know why he’s will­ing to give me this trade’ and even to ‘I know ex­actly what mis­take he is mak­ing.’ It’s a great choice for a big ini­tial mar­ket.

VI

Sum­mary and Conclusion

Pre­dic­tion mar­kets rely on at­tract­ing a va­ri­ety of par­ti­ci­pants, in­clud­ing both ‘losers’ who have nat­u­ral rea­sons to par­ti­ci­pate, and ‘win­ners’ who will be at­tracted by that value.

Any crit­i­cal is­sue can kill a pre­dic­tion mar­ket dead, or even an en­tire pre­dic­tion mar­ket ecosys­tem.

If your mar­ket isn’t well-defined, ar­gu­ments over price be­come ar­gu­ments over the rules, which turn into very an­gry par­ti­ci­pants. If this hap­pens even a small per­centage of the time, it drives ev­ery­one away.

If your mar­ket doesn’t re­solve quickly, and quickly is on the or­der of days or at most a few weeks, it needs to be mas­sively liquid and re­fer to real world ques­tions peo­ple have nat­u­ral ex­po­sures to, to cre­ate par­ti­ci­pa­tion. It ties up cash and doesn’t offer the rush of a good gam­ble. No one wants to bet on an ob­scure out­come years from now.

If your mar­ket is un­likely to re­solve, par­ti­ci­pants will find other uses for their time and money. The chance here has to be small, well un­der 50%, and much lower if time to re­s­olu­tion isn’t quick. Years-long mar­kets that are un­likely to trig­ger are go­ing to have se­vere is­sues.

If your mar­ket has po­ten­tial hid­den in­for­ma­tion, that is a tax on ev­ery­one who par­ti­ci­pates, who are prey to ad­verse se­lec­tion. Every­one must worry that the mar­ket knows what they don’t know, and that them lik­ing one side of a trade means the per­son on the other side has a se­cret; there are even traders in such mar­kets that fol­low a strat­egy of ‘find the naively cor­rect bet, and bet the other way,’ which is known (or should be known) as The Con­stanza.

If your mar­ket doesn’t draw nat­u­ral in­ter­est and offer sources of dis­agree­ment, to cre­ate a foun­da­tion of par­ti­ci­pa­tion and liquidity, there’s noth­ing to build on and no in­ter­est.

In ad­di­tion to these threats, such mar­kets face reg­u­la­tory and le­gal hur­dles, and face var­i­ous eth­i­cal con­cerns. If you offer one mar­ket that seems to mimic a reg­u­lated trade, such as an op­tion on a stock, or that sounds dis­taste­ful, such as the so-called ‘as­sas­si­na­tion mar­kets,’ that can be all any­one will see when they look at your offer­ings. Even though such con­cerns, frankly, are mostly quite stupid, they’re real and peo­ple care about this a lot. They’ve got­ten ba­si­cally ev­ery past at­tempt at pre­dic­tion mar­kets (other than book­mak­ers and pro­fes­sional eco­nomic trad­ing plat­forms) shut down.

Ac­tive cu­ra­tion is nec­es­sary to deal with many of these is­sues, and to provide sim­ple ease of use and ease of find­ing what one is look­ing for and would be in­ter­ested in.

Sur­gi­cal use of pre­dic­tion mar­kets for key in­for­ma­tion points re­mains a great idea, and in many cases peo­ple love a good bet. But we shouldn’t get too am­bi­tious, and keep an eye on the prac­ti­cal needs of par­ti­ci­pants.