The Case for Convexity

[Jesse Richardson]

Epistemic Status

This post is made as part of the ERA Cambridge Fellowship. The core idea around convexity comes from conversations with Scott Garrabrant and Alexander Gietelink Oldenziel. I have some uncertainty about the conclusions in this post, and there is some academic literature that touches on these ideas that I have not yet fully engaged with. Thanks to Tilman, Aris and Charlie for feedback.

Introduction

The Independence Axiom is a crucial building block in the theory of expected utility maximizers, and it is usually supported by a dutch book argument that shows an agent without Independence to be vulnerable to arbitrary losses. In this post I examine this argument and propose that it doesn’t hold in all relevant situations because agents can intrinsically value certainty in sequential planning.

Setup

Consider a finite space of deterministic outcomes Ω, and $L$ the space of lotteries over Ω, $L$ =ΔΩ. A lottery is a probabilistic mixture of deterministic outcomes e.g.$L=0.9A+0.1B$ for some $L\in L,A,B\in \Omega$. We have a preference relation $⪰$ over $L$where $M⪰N$ means an agent prefers $M$ to $N$ or is indifferent (and the associated relations $\succ$ for strict preference and $\sim$ for indifference).

The von Neumann-Morgenstern (VNM) Theorem says that an agent with a preference ordering over lotteries that satisfies certain axioms can be described as maximizing the expectation of a utility function $u:L\to R$ with the property that $M⪰N$ iff $u\left(M\right)\ge u\left(N\right)$. The axiom we are concerned with in this post is Independence:

Independence Axiom

For all $L,M,N\in L$ and $p\in [0,1]$:

$L⪰M$ iff $pL+(1-p)N⪰pM+(1-p)N$

This axiom gives us the following property:

Linearity in probabilities

For all $L,M,N\in L$ and $p\in [0,1]$:

If $M=pL+(1-p)N$, then

$u\left(M\right)=p\cdot u\left(L\right)+(1-p)\cdot u\left(N\right)$

Going forward, we’ll call utility functions with this property linear and we’ll use convex and concave to denote cases where $u\left(M\right)\le p\cdot u\left(L\right)+(1-p)\cdot u\left(N\right)$ or $u\left(M\right)\ge p\cdot u\left(L\right)+(1-p)\cdot u\left(N\right)$ respectively. These two cases can also be described as ‘preferring certainty’ and ‘preferring mixtures’ respectively.

Is convexity vulnerable to dutch book arguments?

Recent writing by Scott Garrabrant has questioned Independence and it has been suggested that concave utility functions should be considered rational in order to allow agents to cooperate/​merge with other agents. If we extend the space of forms that $u$ can take to concavity, another question to consider is when convexity—preferring certainty to probabilistic mixtures—can also be rational?

What follows is a rough argument that an agent with a convex utility function is not necessarily vulnerable to a dutch book. This does entail a rejection of Independence, but does not go so far as to propose an alternative axiom.

Consider the setup where we have three urns (urns being roughly analogous to lotteries): the Red Urn is filled with red balls, the Blue Urn is filled with blue balls and the Mixed Urn is filled with an even mixture of each. The game is as follows: first someone (“Sarah”) draws a ball from the Mixed Urn, without me seeing what colour it is. Second, I get to choose a ball from any urn. I get $100 if the two balls are the same colour, and $0 otherwise (think of money and utility as the same concept here, for simplicity). Now suppose I have (strictly) convex utility over the probability that the first ball is Red/​Blue—I have a preference for certainty. How can we construct a dutch book for this situation, given that I am violating Independence?

Suppose Sarah offers me a deal. In exchange for $1, she will flip a coin before drawing. If it’s heads, she’ll instead draw from the Blue Urn, if it’s tails, the Red Urn. If I am not allowed to see the results of this coin flip, I say ‘no thanks’. Drawing at random from these two urns without my knowledge is, to me, exactly the same as drawing from the Mixed Urn. But if I am allowed to see the results of the coin flip, I pay the dollar because I will be given certainty about whether the first ball is Blue or Red, which I prefer to my current uncertainty. Have I just been dutch booked? I argue not, as I can now choose my second ball with knowledge of the first, guaranteeing myself a payoff of $100, whereas previously this was ⁵⁰⁄₅₀. My preference for certainty was not arbitrary, it enabled me to pick up more money/​utility from a later choice. When I paid $1, I did not change the outcome regarding the likelihood of either ball being Red or Blue, but I gained information which was worth more than $1 to me.

The takeaway: convex utility functions seem a natural way of describing the situation where outcomes today give agents information that can help them plan for tomorrow.

Does this cover every dutch book?

Here I’ve shown a potential dutch book argument against convexity and why I think it fails, this doesn’t necessarily rule out an alternative dutch book however my (unproven) contention is that this reasoning covers all other dutch books for this scenario. The core components are the same in all cases: willingness to lose money in exchange for certainty.

Counterarguments

One might be tempted to question assigning preferences to the colour of the first ball, when my payoff is a function only of whether the two balls have the same colour or not. However this objection would assume a clean split between instrumental and terminal goals. In reality, most goals we might think of as terminal in one context are often instrumental in another. We could avoid assigning utilities/​preferences to the probability that the first ball is Red or Blue, but this standard would also force us to abandon preferences for most things in the world that we’d like to say humans have preferences for. In fact, I’d say that most outcomes being instrumental in some sense is the default setup for agents in a wide class of environments.

Another counterargument to this post could come from the comments to another post making a similar point to mine:

If you are an agent that exists in a timeline, then outcomes are world-histories. D is actually equal to (.5A’ + .5B’), where A’ is everything that will happen to you if you’re unsure what will happen to you for a period of time and then you go on a trip to Ecuador; and B’ is everything that will happen to you if you’re unsure what will happen to you for a period of time and then you get a laptop. Determining what A’ and B’ are requires predicting your future actions.

In the original setup, everything happens instantaneously, so there’s no period of uncertainty where you have to plan for two possible events.

This comment draws a distinction between A: going to Ecuador and A’: the trajectory of everything that happens if you’re uncertain and then you go to Ecuador. This seems potentially valid to me, the violation from Independence may come only from ignoring the distinction between A and A’. We get around this issue by thinking of the question of whether the first ball is Red or Blue as being decided at the start and it being a hidden state, rather than an uncertain future. This enables us to say that, when the coinflip-for-$1 deal is proposed, we can have a simple preference for Red or Blue over $0.5Red+0.5Blue$ that doesn’t need to involve trajectories, because there are no lotteries to consider between the current timestep and the timestep when the first ball is drawn. My claim: this preference does not follow Independence, yet it does seem rational.

Conclusion

If it’s true that a convex utility function $u$ is not vulnerable to dutch books, and the same goes for concavity, then there are a wide variety of forms that this function could take. Going forward, it seems relevant to ask whether (potentially) non-linear utility functions are the most natural representation for such an agent’s preferences, or whether an alternative representation might better satisfy certain desired properties. This ties in to ongoing discussions around whether other VNM axioms are also supported by dutch book arguments.

Also, I’m now wondering whether our function $u$ be both concave and convex on different parts of the simplex /​ domain? Are there any reasons that an agent with this function might be selected against? More generally, we must think like any sensible God-fearing Ed Sheeran hater and ask ourselves: are there any restrictions we can place on The Shape of (Yo)u?

[Carl Feynman]

First: the definition of convexity is the exact opposite of what other sources call convexity. What you call convexity they call concavity and vice versa. You might want to flip that before other people are confused.

Second: your use of utility functions is wrong. The problem as stated gives a utility for the two balls to match or not match. That’s a utility function over a three-dimensional space over all lotteries between (R,R), (R,B), (B,R), (B,B). And it’s symmetric. You literally don’t care what the first ball is, as long as the second ball matches it. The important thing is to get knowledge of the first ball, which restricts the problem to one edge of the feasible tetrahedron. You can’t reduce that to a utility function over lotteries over the first ball.

Third: the demonstration is unconvincing. You claim that there are no Dutch books for any utility function over lotteries, convex or concave. You construct something that is not a Dutch book for one particular problem. Not only is this not a proof (as you say) but it doesn’t even give any reason to believe the proposition. The fact that you can construct one point in a very large space which is in X, does not give any evidence for the proposition “all points are X”. At least to this mathematician.

[abhatt349]

$p\in [0,1]$

Small nitpick, but is this meant to say $p\in (0,1]$ instead? Because if $p=0$, then the axiom reduces to $L⪰M⟺N⪰N$, which seems impossible to satisfy for all $L,M,N\in L$ (for nearly all preference relations).

[martinkunev]

Arguably the notion of certainty is not applicable to the real world but only to idealized settings. This is also relevant.