Meetup Notes: Ole Peters on ergodicity

Ole Peters claims that the stan­dard ex­pected util­ity toolbox for eval­u­at­ing wa­gers is a flawed ba­sis for ra­tio­nal de­ci­sion­mak­ing. In par­tic­u­lar, it com­monly fails to take into ac­count that an in­vestor/​bet­tor tak­ing a se­ries of re­peated bets is not an er­godic pro­cess.

Op­ti­miza­tion Pro­cess, in­ter­nety, my­self, and a cou­ple oth­ers spent about 5 hours across a cou­ple of Seat­tle mee­tups in­ves­ti­gat­ing what Peters was say­ing.

Background

Why do we care?

Prox­i­mally, be­cause Nas­sim Taleb is ba­nanas about er­god­ic­ity.

More in­ter­est­ingly, ex­pected util­ity max­i­miza­tion is widely ac­cepted as the ba­sis for ra­tio­nal de­ci­sion­mak­ing. Find­ing flaws (or at least patholo­gies) in this foun­da­tion is there­fore quite high lev­er­age.

A spe­cific ex­am­ple: many peo­ple’s re­tire­ment in­vest­ment strate­gies might be said to be tak­ing the “en­sem­ble av­er­age” as their op­ti­miza­tion tar­get—i.e. their port­fo­lios are built on the as­sump­tion that, ev­ery year, an in­di­vi­d­ual in­vestor should make the choice that, when av­er­aged across (e.g.) 100,000 in­vestors mak­ing that choice for that year, will max­i­mize the mean wealth (or mean util­ity) of in­vestors in the group at the end of that year. It’s claimed that this means that in­di­vi­d­ual re­tire­ment plans can’t work be­cause many in­di­vi­d­u­als will, in ac­tu­al­ity, even­tu­ally be im­pov­er­ished by mar­ket swings, and that so­cial in­surance schemes (e.g. So­cial Se­cu­rity) where the cur­rent rich are trans­fer­ring wealth to the cur­rent poor avoid this pit­fall.

Claims about short­com­ings in ex­pected util­ity max­i­miza­tion are also in­ter­est­ing be­cause I’ve felt vaguely con­fused for a long time about why ex­pected value/​util­ity is the right way to eval­u­ate de­ci­sions; it seems like I might be more strongly in­ter­ested in some­thing like “the 99th per­centile out­come for the over­all util­ity gen­er­ated over my life­time”. Any work that promises to pick at the cor­ners of EU max­i­miza­tion is worth look­ing at.

What does ex­ist­ing non-Peters the­ory say?

The Von Neu­mann-Mor­gen­stern the­o­rem says, loosely, that all ra­tio­nal ac­tors are max­i­miz­ing some util­ity func­tion in ex­pec­ta­tion. It’s al­most cer­tainly not the case that Ole Peters has pro­duced a coun­terex­am­ple, but (again) iden­ti­fy­ing ap­par­ently patholog­i­cal be­hav­ior im­plied by the VNM math would be quite use­ful.

Eco­nomics re­search as a whole tends to take it as given that in­di­vi­d­ual ac­tors are try­ing to max­i­mize, in ex­pec­ta­tion, the log­a­r­ithm of their wealth (or some similar risk-averse func­tion map­ping wealth to util­ity).

Spe­cific claims made by Peters et al.

We were pretty con­fused about this and spent a bunch of in­ves­ti­ga­tion time sim­ply nailing down what was be­ing claimed!

What we learned

1.5x/​0.6x coin flip bet

This is a spe­cific ex­am­ple from https://​​medium.com/​​freshe­co­nomic­think­ing/​​re­vis­it­ing-the-math­e­mat­ics-of-eco­nomic-ex­pec­ta­tions-66bc9ad8f605

Here’s what we con­cluded. [Th­ese tags ex­plain the level of proof we used.]

  • It is in­deed the case that play­ing many, many rounds of this bet com­presses al­most all the win­nings into a tiny cor­ner of prob­a­bil­ity space, with “lost a bunch of money” be­ing the over­whelming ma­jor­ity of out­comes. [math proof]

  • How­ever, no log-wealth-max­i­mizer would ac­cept the bet, ever (at least, not at the stated “bet en­tire bankroll ev­ery time” stakes). [math proof]

  • Bet­ting only a tiny, con­stant chunk of your bankroll ev­ery time in­stead of all your money at once does, as ex­pected, make you richer most of the time. [Monte Carlo simu­la­tion, in­tu­ition]

  • Rea­son­ing about what hap­pens over a gazillion rounds of the game is a lit­tle bunk be­cause you don’t have to com­mit to play a zillion rounds up front. [hand-wav­ing math in­tu­ition]

    • i.e. if some­one is choos­ing, ev­ery round, whether or not to keep play­ing the game, point­ing out that (their de­ci­sion in round N to keep play­ing is dumb be­cause it would be a ter­rible idea to com­mit to play a gazillion ( >> N ) rounds up front) is a red her­ring.

“Rich house, poor player” theorems

The “coin flip” ex­am­ple of the pre­vi­ous sec­tion is claimed to be in­ter­est­ing be­cause most play­ers go bankrupt, de­spite ev­ery wa­ger offered be­ing pos­i­tive ex­pected value to the player.

So then an in­ter­est­ing ques­tion arises: can some rich “house” ex­ploit some less-rich “player” player by offer­ing a pos­i­tive-ex­pected-value wa­ger that the player will always choose to ac­cept, but that leads with near cer­tainty to the player’s bankruptcy when played in­definitely?

(As noted in the last sec­tion, no log-wealth-util­ity player would take even the first bet, so we chose to steel­man/​sim­plify by as­sum­ing that wealth == util­ity (ei­ther ad­just­ing the gam­ble so that it is pos­i­tive ex­pected util­ity, or ad­just­ing the player to have util­ity lin­ear in wealth))

We think it’s pretty ob­vi­ous that, if the house can fund wa­gers whose player-util­ity is un­bounded (ei­ther the house has in­finity money, or the player has some con­ve­nient util­ity func­tion), then, yes, the house can al­most surely bankrupt the player.

So, in­stead, con­sider a house that has some finite amount of money. We have a half-baked math proof ([1] [2]) that there can’t ex­ist a way for the house to al­most-surely (defined as “drive the prob­a­bil­ity of bankruptcy to above (1 - ep­silon) for any given ep­silon”) bankrupt the player.

Tan­gen­tially: there’s a sym­me­try is­sue here: you can just as well say “the house will even­tu­ally go bankrupt” if the house will be re­peat­edly play­ing some game with un­bounded max pay­off with many play­ers. How­ever, note that zero-sum games that nei­ther party deems wise to play are not un­heard of; risk-averse agents don’t want to play any zero-sum games at fair odds!

Paper: The time re­s­olu­tion of the St Peters­burg Paradox

This pa­per claims to ap­ply Peters’s time-av­er­age (in­stead of en­sem­ble-av­er­age) meth­ods to re­solve the St. Peters­burg Para­dox, and to de­rive “util­ity log­a­r­ith­mic in wealth” as a straight­for­ward im­pli­ca­tion of the time-av­er­age rea­son­ing he uses.

We spent about an hour try­ing to di­gest this. Un­for­tu­nately, aca­demic math pa­pers are of­ten im­pen­e­tra­ble even when they’re mak­ing cor­rect state­ments us­ing math­e­mat­i­cal tools the reader is fa­mil­iar with, so we’re not sure of our con­clu­sions.

That said here are some loose notes point­ing to par­tic­u­lar steps we ei­ther couldn’t ver­ify the val­idity of or think are in­valid.

Op­ti­miza­tion Pro­cess also pointed out that equa­tion (6.6) doesn’t re­ally make sense for a lot­tery where the pay­out is always zero.

This pa­per works from the as­sump­tion that the player is try­ing to max­i­mize (in ex­pec­ta­tion) the ex­po­nen­tial growth rate of their wealth. We no­ticed that this is the log-wealth-max­i­mizer—i.e. in or­der to to get from “max­i­mizes growth” to “max­i­mizes the log­a­r­ithm of wealth”, you don’t seem to ac­tu­ally need what­ever deriva­tion Peters’s pa­per is mak­ing.

Conclusions

We still don’t un­der­stand what “the prob­lem with ex­pected util­ity” is that Peters is point­ing at. It seems like ex­pected util­ity with a risk-averse util­ity func­tion is suffi­cient to make ap­pro­pri­ate choices in the 1.5x/​0.6x flip and St. Peters­burg gam­bles.

Peters’s time-av­er­age vs. en­sem­ble-av­er­age St. Peters­burg pa­per ei­ther has bro­ken math, or we don’t un­der­stand it. Either way, we’re still con­fused about the time- vs. en­sem­ble-av­er­age dis­tinc­tion’s ap­pli­ca­tion to gam­bles.

Peters’s St. Peters­burg Para­dox pa­per does de­rive some­thing equiv­a­lent to log-wealth-util­ity from max­i­miz­ing ex­pected growth rate, but maybe this is an elab­o­rate ex­er­cise in beg­ging the ques­tion by as­sum­ing “max­i­mize ex­pected growth rate” as the goal.

I, per­son­ally, am unim­pressed by Peters’s claims, and I don’t in­tend to spend more brain­power in­ves­ti­gat­ing them.