abramdemski(Abram Demski)

Karma: 10,643

abramdemski 27 Feb 2021 1:47 UTC
5 points
on: Never Go Full Kelly
Thanks for writing this!
Just to be pedantic, I wanted to mention: if we take Fractional Kelly as the average-with-market-beliefs thing, it’s actually full Kelly in terms of our final probability estimate, having updated on the market :)
Concerning your first argument, that uncertainty leads to fractional Kelly—is the idea:
1. We have a probability estimate $^p$ , which comes from estimating the true frequency $p$ ,
2. Our uncertainty follows a Beta distribution,
3. We have to commit to a fractional Kelly strategy based on our $^p$ and never update that strategy ever again
?
So the graph shows what happens if we take our uncertainty and keep it as-is, not updating on data, as we continue to update?
Or is it that we keep updating (and hence reduce our uncertainty), but nonetheless, keep our Kelly fraction fixed (so we don’t converge to full Kelly even as we become increasingly certain)?
Also, I don’t understand the graph. (The third graph in your post.) You say that it shows growth rate vs Kelly fraction. Yet it’s labeled “expected utility”. I don’t know what “expected utility” means, since the expected utility should grow unboundedly as we increase the number of iterations.
Or maybe the graph is of a single step of Kelly investment, showing expected log returns? But then wouldn’t Kelly be optimal, given that Kelly maximizes log-wealth in expectation, and in this scenario the estimate $^p$ is going to be right on average, when we sample from the prior?
Anyway, I’m puzzled about this one. What exactly is the take-away? Let’s say a more-or-less Bayesian person (with uncertainty about their utilities and probabilities) buys the various arguments for Kelly, so they say, “In practice, my utility is more or less logarithmic in cash, at least in so far as it pertains to situations where I have repeated opportunities to invest/bet”.
Then they read your argument about parameter uncertainty.
Now lets assume there’s some uncertainty in that. (Bayesians might get a little uncomfortable here—posterior distributions for discrete events are point estimates. Instead imagine you view the event as a Bernoulli random variable with parameter p, and you have a posterior distribution for p.*).
BAYESIAN: Wait, what? I agree I’ll have parameter uncertainty. But we’ve already established that my utility is roughly logarithmic in money. My point estimate (my posterior) for this gamble paying off is $p$ . The optimal bet under these assumptions is Kelly. So what are you saying? Perhaps you’re arguing that my best-estimate probability isn’t really $p$ .
OTHER: No, $p$ is really your best-estimate probability. I’m pointing to your model uncertainty.
BAYESIAN: Perhaps you’re saying that my utility isn’t really logarithmic? That I should be more risk-averse in this situation?
OTHER: No, my argument doesn’t involve anything like that.
BAYESIAN: So what am I missing? Log utility, probability $p$ , therefore Kelly.
OTHER: Look, one of the ways we can argue for Kelly is by studying the iterated investment game, right? We look at the behavior of different strategies in the long term in that game. And we intuitively find that strategies which don’t maximize growth (EG the expected-money-maximizer) look pretty dumb. So we conclude that our values must me closer to the growth-maximizer, ie Kelly, strategy.
BAYESIAN: Right; that’s part of what convinced me that my values must be roughly logarithmic in money.
OTHER: So all I’m trying to do is examine the same game. But this time, rather than assuming we know the frequency of success from the beginning, I’m assuming we’re uncertain about that frequency.
BAYESIAN: Right… look, when I accepted the original Kelly argument, I wasn’t really imagining this circumstance where we face the exact same bet over and over. Rather, I was imagining I face lots of different situations. So long as my probabilities are calibrated, the long-run frequency argument works out the same way. Kelly looks optimal. So what’s your beef with me going “full Kelly” on those estimates?
OTHER: In those terms, I’m examining the case where probabilities aren’t calibrated.
BAYESIAN: That’s not so hard to fix, though. I can make a calibration graph of my long-term performance. I can try to adjust my probability estimates based on that. If my 70% probability events tend to come back true 60% of the time, I adjust for that in the future. I’ve done this. You’ve done this.
OTHER: Do you really think your estimates are calibrated, now?
BAYESIAN: Not precisely, but I could put more work into it if I wanted to. Is this your crux? Would you be happy for me to go Full Kelly if I could show you a perfect x=y line on my calibration graph? Are you saying you can calculate the $α$ value for my fractional Kelly strategy from my calibration graph?
OTHER: … maybe? I’d have to think about how to do the calculation. But look, even if you’re perfectly calibrated in terms of past data, you might be caught off guard by a sudden change in the state of affairs.
BAYESIAN: Hm. So let’s grant that there’s uncertainty in my calibration graph. Are you saying it’s not my current point-estimate of my calibration that matters, but rather, my uncertainty about my calibration?
OTHER: I fear we’re getting overly meta. I do think $α$ should be lower the more uncertain you are about your calibration you are, in addition to lower the lower your point-estimate calibration is. But let’s get a bit more concrete. Look at the graph. I’m showing that you can expect better returns with lower $α$ in this scenario. Is that not compelling?
BAYESIAN (who at this point regresses to just being Abram again): See, that’s my problem. I don’t understand the graph. I’m kind of stuck thinking that it represents someone with their hands tied behind their back, like they can’t perform a Bayes update to improve their estimate $^p$ , or they can’t change their $α$ after the start, or something.

abramdemski 26 Feb 2021 17:06 UTC
4 points
in reply to: riceissa’s comment on: Probability vs Likelihood
Ah, yeah, I agree with your story.

Before the data comes in, the conspiracy theorist may not have a lot of predictions, or may have a lot of wrong predictions.

After the data comes in, though, the conspiracy theorist will have all sorts of stories about why the data fits perfectly with their theory.

My intention in what you quote was to consider the conspiracy theory in its fulness, after it’s been all fleshed out. This is usually the version of conspiracy theories I see.

That second version of the theory will be very likely, but have a very low prior probability. And when someone finds a conspiracy theory like that convincing, part of what’s going on may be that they confuse likelihood and probability. “It all makes sense! All the details fit!”

Whereas the original conspiracy theorist is making a very different kind of mistake.

abramdemski 26 Feb 2021 16:55 UTC
LW: 2 AF: 2
AF
in reply to: Richard_Ngo’s comment on: The Pointers Problem: Human Values Are A Function Of Humans’ Latent Variables
Agreed. The problem is with AI designs which don’t do that. It seems to me like this perspective is quite rare. For example, my post Policy Alignment was about something similar to this, but I got a ton of pushback in the comments—it seems to me like a lot of people really think the AI should use better AI concepts, not human concepts. At least they did back in 2018.

As you mention, this is partly due to overly reductionist world-views. If tables/happiness aren’t reductively real, the fact that the AI is using those concepts is evidence that it’s dumb/insane, right?

Illustrative excerpt from a comment there:

From an “engineering perspective”, if I was forced to choose something right now, it would be an AI “optimizing human utility according to AI beliefs” but asking for clarification when such choice diverges too much from the “policy-approval”.

Probably most of the problem was that my post didn’t frame things that well—I was mainly talking in terms of “beliefs”, rather than emphasizing ontology, which makes it easy to imagine AI beliefs are about the same concepts but just more accurate. John’s description of the pointers problem might be enough to re-frame things to the point where “you need to start from human concepts, and improve them in ways humans endorse” is bordering on obvious.

(Plus I arguably was too focused on giving a specific mathematical proposal rather than the general idea.)

abramdemski 26 Feb 2021 16:19 UTC
12 points
on: Useless knowledge; why people resist education improvement
Welcome to LessWrong! I think the other commenters are being a bit harsh to a first-time poster ;p though perhaps you wouldn’t have it any other way, given your subject matter. So I just want to say that I agree with your overall message; lots of university courses have poor epistemic standards. But then, lots of society has poor epistemic standards, so is this a real surprise?

Additional comment: this view would be a bit more controversial among lesswrong users, but I believe that this also applies to some math courses. Everyone should be taught formal logic, budgeting, probability, and data analysis (and other things), but for most people calculus and matrix math really isn’t needed.

I agree wrt prioritization of math topics, but this is a lot different from the topic of most of your post, which is accuracy. I don’t find this kind of not-commonly-useful argument to be much of a strike against a class. Because if you do take those classes, then you can do those technical jobs (at least, if you get far enough).

Too often it’s the kid in math class complaining “this will never be useful to me”. It’s too easy of an out. If you were training an RL system, you would just put it straight to work and have it do the actual thing it needs to learn thousands of times. But humans get a lot of benefit from broad knowledge, including preparedness for unexpected situations, and having analogies to work from. If you want to do really innovative work, it pays to learn a lot of individually “useless” stuff. (If you don’t want to do really innovative work, OK, sure.)

If someone were approaching school with an attitude of getting as much out of it as possible, it seems like they’d want to identify skills to level up (EG they might identify art history as not being a skill to level up), and then grind those skills as much as they can. Any of those proficiencies, once you grind enough, becomes a marketable skill. (Even if you make an economically poor choice, like law or art, there will be ways to succeed, and potential applications beyond getting a job. Not that I’m recommending making economically poor choices.)

It seems better to have an attitude of collecting as many of those as possible, rather than as little as possible. So long as you’re at school anyway.

abramdemski 24 Feb 2021 16:01 UTC
5 points
in reply to: Vitor’s comment on: Kelly isn’t (just) about logarithmic utility
I wondered if someone would bring this up! Yes, some people take this as a strong argument that utilities simply have to be bounded in order to be well-defined at all. AFAIK this is just called a “bounded utility function”. Many of the standard representation theorems also imply that utility is bounded; this simply isn’t mentioned as often as other properties of utility.

However, I am not one of the people who takes this view. It’s perfectly consistent to define preferences which must be treated as unbounded utility. In doing so, we also have to specify our preferences about infinite lotteries. The divergent sum doesn’t make this impossible; instead, what it does is allow us to take many different possible values (within some consistency constraints). So for example, a lottery with 50% probability of +1 util, 25% of −3, 1/8th chance of +9, ¹⁄₁₆ −27, etc can be assigned any expected value whatsoever. Its evaluation is subjective! So in this framework, preferences encode more information than just a utility for each possible world; we can’t calculate all the expected values just from that. We also have to know how the agent subjectively values infinite lotteries. But this is fine!

How that works is a bit technical and I don’t want to get into it atm. From a mathematical perspective, it’s pretty “standard/obvious” stuff (for a graduate-level mathematician, anyway). But I don’t think many professional philosophers have picked up on this? The literature on Infinite Ethics seems mostly ignorant of it?

abramdemski 24 Feb 2021 1:22 UTC
6 points
in reply to: johnswentworth’s comment on: Kelly isn’t (just) about logarithmic utility
I agree with your prescriptive vs descriptive thing, and agree that I was basically making that mistake.
I think the correct position here is something like: expected utility maximization; and also, utility in “these cases” is going to be close to logarithmic. (IE, if you evolve trading strategies in something resembling Kelly’s conditions, you’ll end up with something resembling Kelly agents. And there’s probably some generalization of this which plausibly abstracts aspects of the human condition.)
But note how piecemeal and fragile this sounds. One layer is the relatively firm expectation-maximization layer. On top of this we add another layer (based on maximization of mode/median/quantile, so that we can ignore things not true with probability 1) which argues for some utility functions in particular.
Ole Peters is trying to re-found decision theory on the basis of this second layer alone. I think this is basically a good instinct:
1. It’s good to try to firm up this second layer, since just Kelly alone is way too special-case, and we’d like to understand the phenomenon in as much generality as possible.
2. It’s good to try and make a 1-layer system rather than a 2-layer one, to try and make our principles as unified as possible. The Kelly idea is consistent with our foundation of expectation maximization, sure, but if “realistic” agents systematically avoid some utility functions, that makes expectation maximization a worse descriptive theory. Perhaps there is a better one.
This is similar to the way Solomonoff is a two-layer system: there’s a lower layer of probability theory, and then on top of that, there’s the layer of algorithmic information theory, which tells us to prefer particular priors. In hindsight this should have been “suspicious”; logical induction merges those two layers together, giving a unified framework which gives us (approximately) probability theory and also (approximately) algorithmic information theory, tying them together with a unified bounded-loss notion. (And also implies many new principles which neither probability theory nor algorithmic information theory gave us.)
So although I agree that your descriptive lens is the better one, I think that lens has similar implications.
As for your comments about symmetry—I must admit that I tend to find symmetry arguments to be weak. Maybe you can come up with something cool, but I would tend to predict it’ll be superseded by less symmetry-based alternatives. For one thing, it tends to be a two-layered thing, with symmetry constraints added on top of more basic ideas.

abramdemski 24 Feb 2021 0:32 UTC
6 points
in reply to: johnswentworth’s comment on: Kelly isn’t (just) about logarithmic utility
Under ordinary conditions, it’s pretty safe to argue “such and such with probability 1, therefore, it’s safe to pretend such-and-such”. But this happens to be a case where doing so makes us greatly diverge from the Bayesian analysis—ignoring the “most important” worlds from a pure expectation-maximization perspective (IE the ones where we repeatedly win bets, amassing huge sums).
So I’m very against sweeping that particular part of the reasoning under the rug. It’s a reasonable argument, it’s just one that imho should come equipped with big warning lights saying “NOTE: THIS IS A SEVERELY NON-BAYESIAN STEP IN THE REASONING. DO NOT CONFUSE THIS WITH EXPECTATION MAXIMIZATION.”
Instead, you simply said:
To maximize that, we maximize expected log return each timestep, which is the Kelly rule.
which, to my eye, doesn’t provide any warning to the reader. I just really think this kind of argument should be explicit about maximizing modal/median/any-fixed-quantile rather than the more common expectation maximization. Because people should be aware if one of their ideas about rational agency is based on mode/median/quantile maximization rather than expectation maximization.

abramdemski 24 Feb 2021 0:15 UTC
2 points
in reply to: SimonM’s comment on: Kelly isn’t (just) about logarithmic utility
Yeah, in retrospect it was a bit straw of me to argue the pure bayesian perspective like I did (I think I was just interested in the pure-bayes response, not thinking hard about what I was trying to argue there).

So, similarly, I see the Peters justification of Kelly as ultimately just a fancy way of saying that taking the logarithm makes the math nice. You’re leaning on that argument to a large extent, although you also cite some other properties which I have no beef with.

I don’t really think of it as much as an “argument”. I’m not trying to “prove” Kelly criterion.

I’m surprised at that. Your post read to me as endorsing Peters’ argument. (Although you did emphasize that you were not trying to say that Kelly was the one true rule.)

Hm. I guess I should work harder on articulating what position I would argue for wrt Kelly. I basically think there exist good heuristic arguments for Kelly, but people often confuse them for more objective than they are (either in an unsophisticated way, like reading standard arguments for Kelly and thinking they’re stronger than they are, or in a sophisticated way, like Peters’ explicit attempt to rewrite the whole foundation of decision theory). Which leads me to reflexively snipe at people who appear to be arguing for Kelly, unless they clearly distinguish themselves from the wrong arguments.

I’m very interested in your potential post on corrections to Kelly.

abramdemski 23 Feb 2021 22:43 UTC
2 points
in reply to: Blackjay’s comment on: What’s your best alternate history utopia?
Yeah, I haven’t looked at any statistics, but anecdotally it seems like I can’t support the claim that better voting methods are that much better in practical terms. Many places have systems which seem better than that of the USA, without markedly better political results.

abramdemski 23 Feb 2021 22:32 UTC
4 points
in reply to: gwern’s comment on: Meetup Notes: Ole Peters on ergodicity
I like the “Bellman did it better” retort ;p
FWIW, I remain pretty firmly in the expected-utility camp; but I’m quite interested in looking for cracks around the edges, and exploring possibilities.
I agree that there’s no inherent decision-theory issue with multi-step problems (except for the intricacies of tiling issues!).
However, the behavior of Bayesian agents with utility linear in money, on the Kelly-betting-style iterated investment game, for high number of iterations, seems viscerally wrong. I can respect treating it as a decision-theoretic counterexample, and looking for decision theories which don’t “make that mistake”. I’m interested in seeing what the proposals look like.

abramdemski 23 Feb 2021 22:24 UTC
2 points
in reply to: GuySrinivasan’s comment on: Kelly isn’t (just) about logarithmic utility
Maximizing expected value is a fantastically useful concept.
Maximizing expected value at a time t after many repeated choices is a fantastically useful concept.
It turns out that maximizing expected magnitude of value at every choice is very close to the optimal way to maximize your expected value at a time t after many repeated choices. Kelly is a nice, intuitive, easy heuristic for doing so.
I’m not 100% sure what mathematical fact you are trying to refer to here, but I am worried that you are stating a falsehood.
Slightly editing stuff from another comment of mine:
In a one-step scenario, the Bayesian wants to maximize $E [u (S \cdot x)]$ where $x$ is your starting money and $S$ is a random variable for the payoff-per-dollar of your strategy. In a two-step scenario, the Bayesian wants to maximize $E [u (S_{1} \cdot S_{2} \cdot x)]$ . And so on. If $u (x) = x$ , this allows us to push the expectation inwards; $E [u (S_{1} \cdot S_{2} \cdot x)]$ $= E [S_{2} \cdot S_{1} \cdot x]$ $= E [S_{1}] \cdot E [S_{2}] \cdot x$ (the last step holds because we assume the random variables are independent). So in that case, we could just choose the best one-step strategy and apply it at each time-step.
In other words, starting with the idea of raw expectation maximization (maximizing money, so $u (x) = x$ , where $x$ is our bankroll) and adding the idea of iteration, we don’t get any closer to Kelly. Kelly isn’t an approximately good strategy for the version of the game with a lot of iterations. The very same greedy one-step strategy remains optimal forever.
But I could be misunderstanding the point you were trying to communicate.

abramdemski 23 Feb 2021 22:11 UTC
2 points
in reply to: pilord’s comment on: Kelly isn’t (just) about logarithmic utility
I’m not sure what prompted all of this effort, and I’ve rarely heard Kelly described as corresponding to log utility, only ever as an aside about mean-variance optimization. There, log utility corresponds to A=1, which is also the Kelly portfolio. That is the maximally aggressive portfolio, and most people are much, much more risk averse.
Where can I read more about this?

abramdemski 23 Feb 2021 22:09 UTC
2 points
in reply to: SimonM’s comment on: Kelly isn’t (just) about logarithmic utility
I think that (roughly) this post isn’t aimed at someone who has already decided what their utility is. Most of the examples you didn’t like / saw as non-sequitor were explicitly given to help people think about their utility.
Ah, I suspect this is a mis-reading of my intention. For a good portion of my long response, I was speaking from the perspective of a standard Bayesian. Standard Bayesians have already decided what their utility is. I didn’t intend that part as my “real response”. Indeed, I intended some of it to sound a bit absurd. However, a lotta folks round here are real fond of Bayes, so articulating “the bayesian response” seems relevant.
If you’d wanted to forestall that particular response, in writing the piece, I suppose you could have been more explicit about which arguments are Bayesian, and which are explicitly anti-Bayesian (IE Peters), and where you fall wrt accepting Bayesian / anti-Bayesian assumptions. Partly I thought you might be pretty Bayesian and would take “Bayesians wouldn’t accept this argument” pretty seriously. (Although I also had probability on you being pretty anti-Bayesian.)

abramdemski 23 Feb 2021 21:42 UTC
2 points
in reply to: SimonM’s comment on: Kelly isn’t (just) about logarithmic utility
I appreciated that you posted this, and I think your further proposed post(s) sound good, too!
To my personal taste, my main disappointment when reading the post was that you didn’t expand this section more:
Utilities and repeated bets are two sides of the same coin
I think this is fairly clear—“The time interpretation of expected utility theory” (Peters, Adamou) prove this. I don’t want to say too much more about this. This section is mostly to acknowledge that:
- Yes, Kelly is maximising log utility
- No, it doesn’t matter which way you think about this
- Yes, I do think thinking in the repeated bets framework is more useful
But that’s a perfectly reasonable choice given (a) writing time constraints, (b) time constraints of readers. You probably shouldn’t feel pressured to expand arguments out more as opposed to writing up rough thoughts with pointers to the info.
Speaking of which, I appreciated all the references you included!

abramdemski 23 Feb 2021 21:35 UTC
4 points
in reply to: johnswentworth’s comment on: Kelly isn’t (just) about logarithmic utility
The logarithm in the “maximize expected log wealth” formulation is a reminder of that. If returns are multiplicative and bets are independent, then the long run return will be the product of individual returns, and the log of long-run return will be the sum of individual log returns. That’s a sum of independent random variables, so we apply the central limit theorem, and find that long-run return is roughly e^((number of periods)*(average expected log return)). To maximize that, we maximize expected log return each timestep, which is the Kelly rule.
Wait, so, what’s actually the argument?
It looks to me like the argument is “returns are multiplicative, so the long-run behavior is log-normal. But we like normals better than log-normals, so we take a log. Now, since we took a log, when we maximize we’ll find that we’re maximizing log-wealth instead of wealth.”
But what if I like log-normals fine, so I don’t transform things to get a regular normal distribution? Then when I maximize, I’m maximizing raw money rather than log money.
I don’t see a justification of the Kelly formula here.

abramdemski 23 Feb 2021 21:28 UTC
2 points
in reply to: johnswentworth’s comment on: Kelly isn’t (just) about logarithmic utility
Well, here’s my longer response!

abramdemski 23 Feb 2021 21:14 UTC
17 points
on: Kelly isn’t (just) about logarithmic utility
OK, here is my fuller response.
First of all, I want to state that Ergodicity Economics is great and I’m glad that it exists—like I said in my other comment, it’s a true frequentist alternative to Bayesian decision theory, and it does some cool things, and I would like to see it developed further.
That said, I want to defend two claims:
- From a strict Bayesian perspective, it’s all nonsense, so those arguments in favor of Kelly are at best heuristic. A Bayesian should be much more interested in the question of whether their utility is log in money. (Note: I am not a strict bayesian, although I think strict Bayes is a pretty good starting point.)
- From a broader perspective, I think Peters’ direction is interesting, but his current methodology is not very satisfying, and doesn’t support his strong claims that EG Kelly naturally pops out when you take a time-averaging rather than ensemble-averaging perspective.
“It’s All Nonsense”
Let’s deal with the strict Bayesian perspective first.
You start your argument by discussing the “variance drag” phenomenon. You point out that the expected monetary value of an investment can be constant or even increasing, but it may be that due to variance drag, it typically (with high probability) goes down over time.
You go on to assert that “people need to account for this” and “adjust their strategy” because “variance is bad”.
To a strict Bayesian, this is a non-sequitur. If it be the case that a strict Bayesian is a money-maximizer (IE, has utility linear in money), they’ll see no problem with variance drag. You still put your money where it gets the highest expected value.
Furthermore, repeated bets don’t make any material difference to the money-maximizer. Suppose the optimal one-step investment strategy has a tare of return R. Then the best two-step strategy is to use that strategy twice to get an expected rate of return R^2. This is because the best way they can set themselves up to succeed at stage 2 (in expectation!) is to get as much money as they can at stage 1 (in expectation!). This reasoning applies no matter how many stages we string together. So to the money-maximizer, your argument about repeated bets is totally wrong.
Where a strict Bayesian would start to agree with you is if they don’t have utility linear in money, but rather, have diminishing returns.
In that case, the best strategy for single-step investing can differ from the best strategy for multi-step investing.
Say the utility function is $u (x)$ . Treat an investment strategy as a stochastic function $s (x)$ . We assume the random variable $c \cdot s (x)$ behaves the same as the random variable $s (c \cdot x)$ ; IE, returns are proportional to investment. So, actually, we can treat a strategy as a random variable $S$ which gets multiplied by our money; $s (x) = S \cdot x$ .
In a one-step scenario, the Bayesian wants to maximize $E [u (s (x))]$ where $x$ is your starting money. In a two-step scenario, the Bayesian wants to maximize $E [u (s_{2} (s_{1} (x)))]$ . And so on. The reason the strategy was so simple when $u (x) = x$ was that this allowed us to push the expectation inwards; $E [u (s_{2} (s_{1} (x)))] = E [s_{2} (s_{1} (x))]$ $= E [S_{2} \cdot S_{1} \cdot x]$ $= E [S_{1}] \cdot E [S_{2}] \cdot x$ (the last step holds because we assume the random variables are independent). So in that case, we could just choose the best individual strategy and apply it at each time-step.
In general, however, nonlinear $u$ stops us from pushing the expectations inward. So multi-step matters.
A many-step problem looks like $E [u (S_{1} \cdot S_{2} \cdot S_{3} \cdot . . . \cdot S_{n} \cdot x)]$ . The product of many independent random variables will closely approximate a log-normal distribution, by the multiplicative central limit theorem. I think this somehow implies that if the Bayesian had to select the same strategy to be used on all days, the Bayesian would be pretty happy for it to be a log-wealth maximizing strategy, but I’m not connecting the dots on the argument right now.
So anyway, under some assumptions, I think a Bayesian will behave increasingly like a log-wealth maximizer as the number of steps goes up.
But this is because the Bayesian had diminishing returns in the first place. So it’s still ultimately about the utility function.
Most attempts to justify Kelly by repeated-bet phenomena overtly or implicitly rely on an assumption that we care about typical outcomes. A strict Bayesian will always stop you there. “Hold on now. I care about average outcomes!”
(And Ergodicity Economics says: so much the worse for Bayes!)
Bayesians may also raise other concerns with Peters-style reasoning:
- Ensemble averaging is the natural response to decision-making under uncertainty; you’re averaging over different possibilities. When you try to time-average to get rid of your uncertainty, you have to ask “time average what?”—you don’t know what specific situation you’re in.
- In general, the question of how to turn your current situation into a repeated sequence for the purpose of time-averaging analysis seems under-determined (even if you are certain about your present situation). Surely Peters doesn’t want us to use actual time in the analysis; in actual time, you end up dead and lose all your money, so the time-average analysis is trivial. So Peters has to invent an imagined time dimension, in which the game goes on forever. In contrast, Bayesians can deal with time as it actually is, accounting for the actual number of iterations of the game, utility of money at death, etc.
- Even if you settle on a way to turn the situation into an iterated sequence, the necessary limit does not necessarily exist. This is also true of the possibility-average, of course (the St Petersburg Paradox being a classic example); but it seems easier to get failure in the time-avarage case, because you just need non-convergence; ie, you don’t need any unbounded stuff to happen.
It’s Not Nonsense, But It’s Not Great, Either
Each of the above three objections are common fare for the Frequentist:
- Frequentist approaches start from the objective/external perspective rather than the agent’s internal uncertainty. (For example, an “unbiased estimator” is one such that given a true value of the parameter, the estimator achieves that true value on average.) So they’re not starting from averaging over subjective possibilities. Instead, Frequentism averages over a sequence of experiments (to get a frequency!). This is like a time-average, since we have to put it in sequence. Hence, time-averages are natural to the way Frequentists think about probability.
- Even given direct access to objective truth, frequentist probabilities are still under-defined because of the reference class problem—what infinite sequence of experiments do you conceive of your experiment as part of? So the ambiguity about how to turn your situation into a sequence is natural.
- And, again, once you select a sequence, there’s no guarantee that a limit exists. Frequentism has to solve this by postulating that limits exist for the kinds of reference classes we want to talk about.
So, actually, this way of thinking about decision theory is quite appealing from a Frequentist perspective.
Remember all that stuff about how a Bayesian money-maximizer would behave? That was crazy. The Bayesian money-maximizer would, in fact, lose all its money rather quickly (with very high probability). Its in-expectation returns come from increasingly improbable universes. Would natural selection design agents like that, if it could help it?
So, the fact that Bayesian decision theory cares only about in-expectation, rather than caring about typical outcomes, does seem like a problem—for some utility functions, at least, it results in behavior which humans intuitively feel is insane and ineffective. (And philosophy is all about trying to take human intuitions seriously, even if sometimes we do decide human intuition is wrong and better discarded).
So I applaud Ole Peters’ heroic attempt to fix this problem.
However, if you dig into Peters, there’s one critical place where his method is sadly arbitrary. You say:
Compounding is multiplicative, so it becomes “natural” (in some sense) to transform everything by taking logs.
Peters makes much of this idea of what’s “natural”. He talks about additive problems vs multiplicative problems, as well as the more general case (when neither additive/multiplicative work).
However, as far as I can tell, this boils down to creatively choosing a function which makes the math work out.
I find this inexcusable, as-is.
For example, Peters makes much of the St Petersburg lottery:
A resolution of the St Petersburg paradox is presented. In contrast to the standard resolution, utility is not required. Instead, the time-average performance of the lottery is computed. The final result can be phrased mathematically identically to Daniel Bernoulli’s resolution, which uses logarithmic utility, but is derived using a conceptually different argument. The advantage of the time resolution is the elimination of arbitrary utility functions.
Daniel Bernoulli “solved” the St Petersburg paradox by suggesting that utility is log in money, which makes the expectations work out. (This is not very satisfying, because we can always transform the monetary payouts by an exponential function, making things problematic again! D Bernoulli could respond “take the logarithm twice” in such a case, but by this point, it’s clear he’s just making stuff up, so we ignore him.)
Ole Peters offers the same solution, but, he says, with an improved justification: he promises to eliminate the arbitrary choice of utility functions.
He does this by arbitrarily choosing to examine the time-average behavior in terms of growth rate, which is defined multiplicatively, rather than additively.
“Growth rate” sounds like an innocuous, objective choice of what to maximize. But really, it’s sneaking in exactly the same arbitrary decision which D Bernoulli made. Taking a ratio, rather than a difference, is just a sneaky way to take the logarithm.
So, similarly, I see the Peters justification of Kelly as ultimately just a fancy way of saying that taking the logarithm makes the math nice. You’re leaning on that argument to a large extent, although you also cite some other properties which I have no beef with.
Again, I think Ole Peters is pretty cool, and there’s interesting stuff in this general direction, even if I find that the particular methodology he’s developed doesn’t justify anything and instead engages in obfuscated question-begging of existing ideas (like Kelly, and like D Bernoulli’s proposed solution to St Petersburg).
Also, I could be misunderstanding something about Peters!

abramdemski 23 Feb 2021 18:30 UTC
4 points
on: Kelly isn’t (just) about logarithmic utility
I’ve only skimmed this post so far, so, I’ll need to read it in greater detail. However, I think this is quite related to Nassim Taleb and Ole Peters’ critique of Bayesian decision theory. I’ve left fairly extensive comments there, although I’m afraid I changed my mind a couple of times as I read deeper into the material, and there’s not a nice summary of my final assessment.
I would currently defend the following statements:
- From a Bayesian standpoint, Kelly is all about logarithmic utility, and the arguments about repeated bets don’t make very much sense.
- From a frequentist standpoint, the arguments about repeated bets make way more sense. Ole Peters is developing a true frequentist decision theory: it used to be that Bayesianism was the only game in town if you wanted your philosophy of statistics to also give you decision theory, but no longer!
- I ultimately find Peters’ decision theory unsatisfying. It promises to justify different strategies for different situations (EG logarithmic utility for Kelly-type scenarios), but really, pulls the quantity to be maximized out of thin air (IE the creativity of the practitioner). It’s slight of hand.
Anyway, I’ll read the post more closely, and go further into my critique of Peters if it seems relevant.

abramdemski 23 Feb 2021 18:13 UTC
2 points
in reply to: pilord’s comment on: Kelly isn’t (just) about logarithmic utility
I would defend the idea that Kelly is more genuinely about log utility when approached from a strict Bayesian perspective, ie, a Bayesian has little reason to buy the other arguments in favor of Kelly.

abramdemski 23 Feb 2021 17:44 UTC
4 points
in reply to: Kaj_Sotala’s comment on: Support vs Advice & Holding Off Solutions
What you’re calling a problem-oriented mode sounds like it’s closer to coaching on the “clarifying questions” dimension than the two others are, but given that it’s still aiming to eventually involve proposing solutions, it’s not totally that.
Well, to clarify, I meant that you can still eventually propose solutions because you don’t have to be stuck in the problem-oriented mode forever rather than that my definition of problem-oriented mode allows proposing solutions.
So then the modes might reflect varying points in a space with three dimensions: amount of support, amount of offered solutions, and amount of clarifying questions.
I would want to clarify what “support” is, since part of my idea here is that asking clarifying questions can itself provide some emotional support. Some concrete actions which might constitute support:
- Validating the other person’s feelings as legitimate.
  - Verifying that their response to the situation makes logical sense.
  - Verifying that what they are feeling isn’t weird (that it is “normal”)
  - Stating that their feelings are legitimate
  - Telling a similar story about yourself, to show that you have experienced similar things
- Asserting support for the person.
  - Stating that you are there for them, want to help, care about them, etc.
  - Concrete reassurances, EG “I will never ____”
  - Showing social allegiance by denigrating “the other side” (if applicable)
- Showing that you understand.
  - “That must feel ____” (correctly describing the other person’s state)
  - Talking about a similar experience you’ve had.
  - Making sympathetic satements, EG, “Yeah, that person is just awful”
Lots of these things feel like they could easily be the wrong thing, depending on the situation; this makes me think “support” is this complicated thing which can’t really be described by concrete conversational moves too well.

abramdemski(Abram Demski)

Utilities and repeated bets are two sides of the same coin

“It’s All Nonsense”

It’s Not Nonsense, But It’s Not Great, Either