An investment analogy for Pascal’s Mugging
A lottery ticket sometimes has positive expected value, (a $1 ticket might be expected to pay out $1.30). How many tickets should you buy?
Probably none. Informally, all but the richest players can expect to go broke before they win, despite the positive expected value of a ticket.
In more precise terms: In order to maximize the long-term growth rate of your money (or log money), you’ll want to put a very small fraction of your bankroll into lotteries tickets, which will imply an “amount to invest” that is less than the cost of a single ticket, (excluding billionaires). If you put too great a proportion of your resources into a risky but positive expected value asset, the long-term growth rate of your resources can become negative. For an intuitive example, imagine Bill Gates dumping 99% percent of his wealth into a series of positive expected-value bets with single-lottery-ticket-like odds.
This article has some graphs and details on the lottery. This pdf on the Kelly criterion has some examples and general dicussion of this type of problem.
Can we think about Pascal mugging the same way?
The applicability might depend on whether we’re trading resource-generating-resources for non-resource-generating assets. So if we’re offered something like cash, the lottery ticket model (with payout inversely varying with estimated odds) is a decent fit. But what if we’re offered utility in some direct and non-interest-bearing form?
Another limit: For a sufficiency unlikely but positive-expected-value gamble, you can expect the heat death of the universe before actually realizing any of the expected value.
The Kelly criterion assumes logarithmic utility of money. The difference between $100 and $10 is the same as the difference between $10 and $1. And going broke has negative infinite expected value. If you don’t do that, then your best bet is to take any bet with expected value above one. Sure you’ll almost certainly go broke, but the utility from winning every bet would be enough to make up for it.
Even if we assign logarithmic utility to whatever we’re being mugged with, it just makes getting mugged harder. 3^^^3 is such overkill that, unless we’re running out of money and we assign negative infinite utility to this, we’re better off paying the mugger.
You say that as if the only possible options are “utility proportional to wealth, and trying to maximize expected utility” and “utility proportional to log wealth, and trying to maximize expected utility” but of course there are many many options besides those.
One possibly-relevant example: Some people argue that we should use bounded utility functions, on one or more of the following grounds:
Human brains are finite and simply can’t represent an unbounded range of utilities. So if “utility” means anything your brain is actually capable of evaluating, it has an upper bound.
It’s possible (and not all that difficult) to construct paradoxical situations in which the existence of unbounded utilities is an essential feature (e.g., because the expectations you’re trying to construct end up diverging) and bounded utility functions aren’t vulnerable to those problems.
If your utility function is bounded and its upper bound isn’t outrageously large then 3^^^3-type Pascal muggings won’t work on you.
(Note: There are reasons not to want a bounded utility function, too. I am not claiming that we should use them.)
A slight error: the Kelly criterion assigns no utility. It simply maximizes your expected bankroll at any point in the non-immediate future. What you do with that bankroll and how much you care about it is unmentioned by the Kelly criterion.
It does not maximize your expected bankroll. It maximizes the expected value of the log of your bankroll. This is equivalent to saying that it is utilitarian with respect to the utility function of the log of your bankroll.
Ah, I see what you’re getting at. Good catch.
Log is a monotonic function, what maximizes one maximizes the other.
Yes, but what maximizes the expected value of one does not necessarily maximize the expected value of the other. The average of zero and four is two. The average of log(zero) and log(four) is negative infinity. Ergo, if you have one dollar and you have the option of betting it so you get four dollars if you win and nothing if you lose, betting increases the expected value of your bankroll, but reduces the expected value of the log of your bankroll.
This is only because zero is special (in that it gives rise to an infinity). And the whole point of the Kelly Rule is that once you hit zero you lose forever, so following it will never result in you getting to zero dollars. Once we’re dealing only with positive numbers, whatever maximizes the expected value in dollars, maximizes the expected value in log dollars as well.
Average of 1 and 100 is 50.5. Average of log(1) and log(100) is log(10). If you’re offered a choice of a 50:50 chance of 1 vs 10 and a sure amount between 10 and 50.5, then you’re better off with the bet for maximizing expected value, but you’re better off with the sure thing for maximizing the expected log value.
You are confused between maximizing the log of the expected value of your bets and maximizing the expected value of the log of your bets. These are, of course, not the same.
In Kelly Rule bets you do not get paid the log of the outcomes.
The Kelly rule maximizes the expected value of the log of your bets. You get paid the outcome, but you presumably value the log of what you get paid.
The proof section on the Wikipedia article gives the derivation using Python and SymPy:
It is maximizing the function y = p*log(1+b*x) + (1-p)*log(1-x). Maximizing the function y = p*(1+b*x) + (1-p)*(1-x) with the restriction that p is between 0 and 1 will give p as 0 or 1, since it’s a linear function.
The Kelly rule maximizes the log of your bankroll as the number of trials goes to infinity. Note that Wikipedia says:
You’re maximizing “the expected value of the logarithmic bankroll y(x)”.
If you take any of the bets, your bankroll is a probability distribution. Probability distributions have no standard ordering, and cannot be maximized.
Yes, that’s why you’re maximizing the expected value and not an entire probability distribution.
I seem to have misread the second thing you said, which is more helpful.
Yes. The Kelly criterion maximizes the expected value of the logarithmic bankroll. Not the expected value of the bankroll.
And we have come full circle to noting again that because log is a monotonic function maximizing the expected value of the log of the bankroll is exactly the same thing as maximizing the expected value of the bankroll.
The expected value of the bankroll is a simple number, a positive scalar. Maximizing this value is equivalent to maximizing the log of this value.
But the Kelly criterion does not maximize the log of the expected value of the bankroll. It maximizes the expected value of the log of the bankroll. The way to maximize the log of the expected value of the bankroll is to take all of the bets with all of your money.
I’m not really saying anything DanielLC hasn’t said already, but perhaps it will be easier for you to take seriously when it’s not just one other person disagreeing with you.
DanielLC is right and you are wrong.
Kelly says: maximise expected log bankroll. This is not the same as maximising expected bankroll, and it is not true in general that maximising E(X) and maximising E(f(X)) are equivalent when f is monotonic increasing.
If your current bankroll is $1000 and you can choose between (1) a gamble that gives you $100 with probability 3⁄4 and $1000 with probability 1⁄4 and (2) sticking with what you’ve got, maximising expected bankroll will tell you to choose #1 but maximising expected log bankroll will tell you to choose #2.
Actually, no, it does not. The Kelly Rule aims to maximize the expected bankroll (not the log of it) after many bets. It’s a real-world rule and in the real world people want money, not log(money).
I think there are a couple of points of confusion here. The first is between maximizing what and maximizing how. We want to maximize the amount that you have on hand after a long series of bets. Figuring out what bets to accept and which to decline involves logs, but in the end you just want to have max(money). The second is between maximizing over a long series of bets and maximizing in one single bet (trial).
Your example is screwed up. First, no one would ever choose #1, and second, the Kelly Rule isn’t about picking between two bets, it’s about how much of your bankroll to invest in a particular bet.
In any case, this subthread seems to have fallen into the negative-karma hole.
A few comments on the remainder of this subthread.
Different people are talking about different things, without making the distinctions perfectly clear. Specifically, when Lumifer says “it maximizes your expected bankroll, and maximizing your expected log bankroll is the same” I believe he means “it maximizes your expected bankroll in the long run, and maximizing your expected log bankroll in the long run is the same because in the long run you almost always get approximately your expectation”. Whereas DanielLC and I have been observing that on a single iteration the Kelly rule maximizes your expected log bankroll and does not maximize your expected bankroll. (But Lumifer hasn’t been saying “in the long run” every time, and we haven’t been saying “on a single bet” every time, hence a lot of what I think is pseudo-disagreement.)
It is simply not true in general that maximizing E(X) is the same as maximizing E(f(X)) when f is a monotone increasing function. It is true when X is not actually random, which is kinda what happens in the long run here.
It is simply not true that the Kelly rule “assumes logarithmic utility of money”. If your utility happens to be proportional to log(money) then the Kelly rule maximizes your expected utility even for a single bet, but provided your dependence of utility on money is monotone increasing, in the long run your utility will almost always be maximized by following the Kelly rule.
It isn’t clear to me that the only reason why maximizing E(X) and maximizing E(log(X)) are different is that “zero is special”, even when we are considering what happens in the long run. Specifically, suppose your individual bets have some nasty distribution whose tails are too fat for the variance to be defined; then it needn’t be true that your performance almost always looks like its expectation. Isn’t there a counterexample lurking somewhere around here?
I think it’s a distraction to talk about what the Kelly rule is “trying” to do. You can say that it almost always maximizes your bankroll in the long run. You can say that it maximizes your expected utility in the short run, if its dependence on wealth is logarithmic. You can say that it gives you the warm fuzzy feeling of doing something known to be optimal. The Kelly rule doesn’t care; it is what it is, whatever your goal is. And what it is is an injunction to choose each individual bet so as to maximize E(log(bankroll)).
The principle of maximizing E(log(bankroll)) can be applied in situations where the choice you have is something other than simply “how much shall I bet on this otherwise-fixed gamble?”. I don’t think it’s a serious abuse of terminology to call this “the Kelly rule” since it’s simply a generalization of Kelly to this broader class of situations.
[EDITED to fix formatting a bit.]
In particular its possible for log(X) to have well-defined variance but not X, and for E(log(X)) but not E(X) to be defined.
Re 1: looking back at the subthread, yes, I think that was the source of much confusion. I did mean maximization in the long run and for quite a while did not realize that you and DanielLC were talking about the maximization in a single iteration.
Re 4: The expectation operator is just a weighted sum (in the discrete case) or an integral (in the continuous case). I don’t think it cares about the fatness of tails or whether some moments are defined or not.
Speaking generally, log(E(X)) is not the same thing as E(log(X)) (see Jensen’s Inequality), but that’s a different question. The question we have is that if you have some set of parameters theta that X is conditional on, does maximizing for E(X) lead you to different optimal thetas than maximizing for E(log(X))?
Re 6: Well, you have to be careful that Kelly Rule assumptions hold. It works as it works because capital growth is multiplicative, not additive, and because you expect to have many iterations of betting, for example.
The expectation operator doesn’t care about fatness of tails (well, it kinda doesn’t, but note that e.g. the expectation of a random variable with Cauchy distribution is undefined, precisely because of those very fat tails), but the theorem that says that in the long run your wealth is almost always close to its expectation may fail for fat-tail-related reasons.
In the present case where we’re looking at long-run results only, the answer might be “no” (but—see above—I’m not sure it actually is). But in general, if you allow X to be any random variable rather than some kind of long-run average of well-behaved things, it is absolutely not in any way true that maximizing E(X) leads to the same parameter choices as maximizing E(log(X)).
If you want your choice to be optimal, sure. But all I’m saying is that using “the Kelly rule” to mean “making the choice that maximizes expected log bankroll” seems like a reasonable bit of terminology. Whether using the Kelly rule, in this sense, is a good idea in any given case will of course depend on all sorts of details.
Good point about Cauchy. If even the mean is undefined, all bets are off :-)
Can I get an example? Say, X is a random positive real number. For which distribution which parameters that maximize E(X) will not maximize E(log(X))?
I don’t know about that. The Kelly Rule means a specific strategy in a specific setting and diluting and fuzzifying that specificity doesn’t seem useful.
That is exactly what the Kelly criterion provides examples of. Let p be the probability of winning some binary bet and k the multiple of your bet that is returned to you if you win. Given an initial bankroll of 1, let theta be the proportion of it you are going to bet. Let the distribution of your bankroll after the bet be X. With probability p, X is 1+theta(k-1), and with probability 1-p, X is 1-theta. theta is a parameter of this distribution. (So are p and k, but we are interested in maximising over theta for given p and k.)
If pk > 1 then theta = 1 maximises E(X), but theta = (pk-1)/(k-1) maximises E(log(X)).
The graphs of E(X) and E(log(X)) as functions of theta look nothing like each other. The first is a linear ascending gradient, and the second rises to a maximum and then plunges to -∞.
Yep, I was wrong. Now I need to figure out why I thought I was right..
May have gotten confused because log is monotonically increasing e.g. log likelihood maximized at the same spot as likelihood. So log E(X) is maximized at the same spot as E(X). But log and E do not commute (Jensen’s inequality is not called Jensen’s equality, after all).
Was probably part of it—I think the internal cheering for the wrong position included the words “But log likelihood!” :-/
Sure. So, just to be clear, the situation is: We have real-valued random variable X depending on a single real-valued parameter t. And I claim it is possible (indeed, usual) that the choice of t that maximizes E(log X) is not the same as the choice of t that maximizes E(X).
My X will have two possible values for any given t, both with probability 1⁄2. They are t exp t and exp −2t.
E(log X) = 1⁄2 (log(t exp t) + log(exp −2t)) = 1⁄2 (log t + t − 2t) = 1⁄2 (log t—t). This is maximized at t=1. (It’s also undefined for t<=0; I’ll fix that in a moment.)
E(X) is obviously monotone increasing for large positive t, so it’s “maximized at t=+oo”. (It doesn’t have an actual maximum; I’ll fix that in a moment.)
OK, now let me fix those two parenthetical quibbles. I said X depends on t, but actually it turns out that t = 100.5 + 100 sin u, where u is an angle (i.e., varies mod 2pi). Now E(X) is maximized when sin u = 1, so for u = pi2; and E(log X) is maximized when 100 sin u = −99.5, so for two values of u close to -pi/2. (Two local maxima, with equal values of E(log X).)
Okay, I accept that I’m wrong and you’re right. Now the interesting part is that my mathematical intuition is not that great, but this is a pretty big fail even for it. So in between googling for crow recipes, I think I need to poke around my own mind and figure out which wrong turn did it happily take… I suspect I got confused about the expectation operator, but to confirm I’ll need to drag my math intuition into the interrogation room and start asking it pointed questions.
Upvoted for public admission of error :-).
(In the unlikely event that I can help with the brain-fixing, e.g. by supplying more counterexamples to things, let me know.)
As a trivial example, let’s say you are choosing between distribution A and distribution B.
In distribution A, X=100 with probability 0.5, and X=epsilon with probability 0.5
In distribution B, X=10 with probability 1
The average value of X under distribution A is 50, whereas the average value of X under distribution B is 10. If you want to maximize E(X) you will therefore choose distribution A
The average value of log X under distribution A is negative infinity, whereas the average value of log X under distribution B is 1. If you want to maximize E(log X) you will choose distribution B.
Edited to add: The idea behind Von Neumann Morgenstern Expected Utility Theory is that optimizing your expected utility does not imply the same choices as optimizing the expected payoff. If you maximize for E(X) your utility function is risk neutral, if you maximize for E(log X) your utility function is risk averse etc. If maximizing these two expectations always implied identical choices, it would not be possible to define risk aversion.
As a trivial example, let’s say you are choosing between distribution A and distribution B.
In distribution A, X=100 with probability 0.5, and X=epsilon with probability 0.5
In distribution B, X=10 with probability 1
The average value of X under distribution A is 50, whereas the average value of X under distribution B is 10. If you want to maximize E(X) you will therefore choose distribution A
The average value of log X under distribution A is negative infinity, whereas the average value of log X under distribution B is 1. If you want to maximize E(log X) you will choose distribution B.
This was, as I recall, the first proposed resolution to the St. Petersburg Paradox; but as pointed out by DanielLC, it doesn’t work because you can then just reformulate the statement in terms of utility, instead of dollars.
Some suggest that having a very aggressively bounded utility function is sufficient to solve the problem- if your utility for the current state of things is .9, and your maximum utility is 1, then the amount you’d gamble for a chance to get a utility of 1 is bounded at a rather low amount. But this might be throwing the baby out with the bathwater, and might be asymmetric in a way you really don’t want to be. It seems like any technique that resists Pascal’s Mugging, where you pay a little to avoid a tiny chance of paying a lot, should also resist paying a little to get a tiny chance of getting a lot. (Now that I say that I’m unsure, but that might be because my mind has different heuristics for blackmail, insurance, and investments.)
It would be nice to include some kelly criterion numbers to go with the lottery example.
This post is essentially my response to Pascal’s mugging.
a)If an event is extremely unlikely to occur then trying to maximise expected utility is foolish unless it is repeated multiple times. In the case of the lottery, you need to buy millions of tickets to have a reasonable chance of winning once (ignoring small prizes here). In the mugging, you have to be mugged an absolutely absurd number of times before any of the muggers are even remotely likely to be telling the truth b)The hidden and secret response, which is this:
even if the lottery has positive expected utility, there are likely to be alternate uses for your money which will give better ones! If you have the money to win the lottery, you can invest elsewhere and get better returns. This is true for Pascal’s mugging, where you can go spend your 10 dollars to save lives immediately via a recommendation from give well.