VNM agents and lotteries involving an infinite number of possible outcomes

Summary: The VNM utility theorem only applies to lotteries that involve a finite number of possible outcomes. If an agent maximizes the expected value of a utility function when considering lotteries that involve a potentially infinite number of outcomes as well, then its utility function must be bounded.

Outcomes versus Lotteries

One way to formulate the VNM utility theorem is in terms of outcomes and lotteries over outcomes. That is, there is some set $O$ of outcomes, and a set $L$ of lotteries defined as $L := {n \sum i = 1 p_{i} O_{i} ∣ n \in N, p_{i} > 0, O_{i} \in O (\forall i), n \sum i = 1 p_{i} = 1}$ . In other words, the set of lotteries is the set of probability distributions over a finite number of outcomes. The finiteness is very important; we’ll get to that later. Note that for each outcome, there is a corresponding lottery that guarantees this outcome, and these “pure outcome” lotteries are a basis for $L$ .

Given that formulation, and given the VNM axioms, there exists some function $u : O \to R$ such that given any 2 lotteries $L := n \sum i = 1 p_{i} O_{L, i}$ and $M := m \sum i = 1 q_{i} O_{M, i}$ , $L ≻ M$ iff $n \sum i = 1 p_{i} u (O_{L, i}) > m \sum i = 1 q_{i} u (O_{M, i})$ .

The other formulation does not mention $O$ . Instead, there is simply a set $L$ of lotteries, such that $\forall L_{1}, L_{2}, . . ., L_{n} \in L, \forall p_{1}, p_{2}, . . ., p_{n} > 0 n \sum i = 1 p_{i} L_{i} \in L$ iff $n \sum i = 1 p_{i} = 1$ . In this formulation, there exists some function $u : L \to R$ such that if $L = n \sum i = 1 p_{i} L_{i}$ , then $u (L) = n \sum i = 1 p_{i} u (L_{i})$ (notice $n$ still must be finite) and for any 2 lotteries $L$ and $M$ , $L ≻ M$ iff $u (L) > u (M)$ .

The formulation in terms of outcomes and lotteries over outcomes is more intuitively appealing (to me, at least), since real life has outcomes and uncertainty about outcomes, so I will use it when I can, but the formulation purely in terms of lotteries, which is more similar to what von Neumann and Morgenstern did in their original paper, will be useful sometimes, so I will switch back to it intermittently.

Infinite lotteries

Myth: Given some utility function $u : O \to R$ that accurately describes a VNM-rational agent’s preferences over finite lotteries, if you expand $L$ to include lotteries with an infinite number of possible outcomes (let’s call the expanded set of lotteries $L$ ), then for any 2 lotteries $L := \infty \sum i = 1 p_{i} O_{L, i}$ and $M := \infty \sum i = 1 q_{i} O_{M, i}$ , $L ≻ M$ iff $\infty \sum i = 1 p_{i} u (O_{L, i}) > \infty \sum i = 1 q_{i} u (O_{M, i})$ .

Reality: Knowing an agent’s preferences over finite lotteries, and that the agent obeys the VNM axioms, does not tell you everything about the agent’s preferences over lotteries with an infinite number of possible outcomes. To demonstrate this, I’m going to construct a VNM-rational agent that maximizes a utility function $u : L \to R$ , where $\exists {L_{i}}_{i}^{\infty} \in L \exists {p_{i} (\geq 0)}_{i}^{\infty} \infty \sum i = 1 p_{i} = 1, u (\infty \sum i = 1 p_{i} L_{i}) \neq \infty \sum i = 1 p_{i} u (L_{i})$ . This construction relies on the axiom of choice (please let me know if you figure out whether or not it is possible to construct such an agent without the axiom of choice). I will also be assuming that $O$ is countably infinite (if $O$ is finite, such an agent is impossible, and if it is uncountable, then you can consider a countable subset).

Notice that $L$ can be seen as a subset of the real vector space $V := {\sum O \in O α_{O} O ∣ \forall O \in O α_{O} \in R, \sum O \in O | α_{O} | c o n v e r g e s}$ , with the addition and multiplication by scalar operations being exactly what you would expect ( $(\sum O \in O α_{O} O) (\sum O \in O β_{O} O) = \sum O \in O (α_{O} β_{O}) O$ , and $x (\sum O \in O α_{O} O) = \sum O \in O x α_{O} O$ ). A utility function $u$ can be seen as an element of the dual space of $V$ . The axiom of choice implies that this vector space has a basis (in this context, a basis means a set of vectors for which any finite subset is linearly independent, and every vector is a linear combination of a finite number of basis vectors). The value of $u$ on each basis element can be chosen independently, and these choices completely determine $u$ . In particular, the basis could contain every element of $O$ , and also contain $\infty \sum i = 1 2^{- i} O_{i}$ for some sequence ${O_{i} \in O}_{i}^{\infty}$ with distinct elements. Then we could have $\forall O \in O u (O) = 0$ and $u (\infty \sum i = 1 2^{- i} O_{i}) = 1$ , violating the conclusion of the myth, but this meets all the VNM axioms.

The fact that there is a real-valued function on lotteries that the agent maximizes guarantees that the completeness and transitivity axioms hold, since $u (L) > u (M)$ or $u (L) = u (M)$ or $u (L) < u (M)$ (completeness), and if $u (L) \leq u (M)$ and $u (M) \leq u (N)$ then $u (L) \leq u (N)$ (transitivity). The fact that the function is linear with respect to finite sums guarantees that the continuity and independence axioms hold, since if $u (L) < u (M) < u (N)$ then $u (\frac{u (N) - u (M)}{u (N) - u (L)} L \frac{u (M) - u (L)}{u (N) - u (L)} N) = \frac{u (N) - u (M)}{u (N) - u (L)} u (L) \frac{u (M) - u (L)}{u (N) - u (L)} u (N) = u (M)$ (continuity), and if $u (L) < u (M)$ then for any lottery $N$ and positive probability $p$ , $u (p L (1 - p) N) = p u (L) (1 - p) u (N) < p u (M) (1 - p) u (N) = u (p M (1 - p) N)$ (independence).

Extended VNM hypothesis

The VNM utility theorem does not prove that an agent meeting its axioms will maximize the expected value of a utility function when presented with infinite lotteries, but the fact that any such agent will maximize the expected value of a utility function when presented with finite lotteries certainly seems very suggestive. With that in mind, I suggest that this be called the “extended von Neumann-Morgenstern hypothesis”:

An agent, in order to be considered rational, should maximize the expected value of a utility function over outcomes when choosing between lotteries over any (possibly infinite) number of outcomes.

Bounded and unbounded utility functions

It is perfectly possible to construct VNM-rational agents with an unbounded utility function. But all such agents will inevitably violate the extended VNM hypothesis, because it is possible to create infinite lotteries with undefined expected value. For instance, the St. Petersburg paradox can be modified to refer specifically to utilities instead of money. That is, if there is no upper bound to the agent’s utility function, then there exists a sequence of outcomes ${O_{i}}_{i}^{\infty}$ such that for each $i$ , $u (O_{i}) \geq 2^{i}$ . Then the expected utility of $\infty \sum i = 1 2^{- i} O_{i}$ is $\infty \sum i = 1 2^{- i} u (O_{i}) \geq \infty \sum i = 1 2^{- i} 2^{i} = \infty \sum i = 1 1$ , which does not converge. So unbounded utility functions are not compatible with the extended VNM hypothesis.

At this point, one may feel a strong temptation to come up with some way to characterize the values of infinite sums with some ordered superset of the real numbers, so that it is possible to compare nonconvergent sums. However, by the formulation of the VNM theorem solely in terms of lotteries, the utility of any lottery, such as $u (\infty \sum i = 1 2^{- i} O_{i})$ , is a real number. So any such scheme that requires that the range of the utility function include nonreals will violate the VNM axioms. In particular, it will probably violate the archimedian axiom.

One possible response to this is to dismiss the archimedian axiom, and try to characterize agents that obey the completeness, transitivity, and independence axioms. Benja has written (section “Doing without Continuity) about this, and I find his solution fairly compelling, but it isn’t clear that it helps us deal with situations like the St. Petersburg paradox. I intend to say more about nonarchimedian preferences soon.