This is a result from the first MIRIx Cambridge workshop (coauthored with Janos and Jim).
One potential problem with bounded utility functions is: what happens when the bound is nearly reached? A bounded utility maximizer will get progressively more and more risk averse as it gets closer to its bound. We decided to investigate what risks it might fear. We used a toy model with a bounded-utility chocolate maximizer, and considered what happens to its resource allocation in the limit as resources go to infinity.
We use “chocolate maximizer″ as conceptual shorthand meaning an agent that we model as though it has a single simple value with a positive long-run marginal resource cost, but only as a simplifying assumption. This is as opposed to a paperclip maximizer, where the inappropriate simplicity is implied to be part of the world, not just part of the model.
Conceptual uncertainty
We found that if a bounded utility function approaches its bound too fast, this has surprising pathological results when mixed with logical uncertainty. Consider a bounded-utility chocolate maximizer, with philosophical uncertainty about what chocolate is. It has a central concept of chocolate C1
, and there are classes of mutated versions of the concept of chocolate at varying distances from the central concept, such that the probability that the true chocolate is in class Ci
is proportional to i−α
(i.e. following a power law).
Suppose also that utility is bounded using a sigmoid function , where x is the amount of chocolate produced. In the limit as resources go to infinity, what fraction of those resources will be spent on the central class C1
? That depends which sigmoid function is used, and in particular, how quickly it approaches the utility bound.
Example 1: exponential sigmoid
Suppose we allocate ri
resources to class Ci
, with for total resource r. Let .
Thus, the resources will be evenly distributed among all the classes as r increases. This is bad, because the resource fraction for the central class C1
goes to 0 as we increase the number of classes.
EDITED: Addendum on asymptotics
Since we have both r and n going to infinity, we can specify their relationship more precisely. We assume that n is the highest number of classes that are assigned nonnegative resources for a given value of r:
Thus, for α>2
the limit of the resource fraction for the central class C1
is finite and positive.
Conclusion
The arctan sigmoid results in a better limiting resource allocation than the exponential sigmoid, because it has heavier tails (for sufficiently large α
). Thus, it matters which bounding sigmoid function you choose.
[MIRIx Cambridge MA] Limiting resource allocation with bounded utility functions and conceptual uncertainty
This is a result from the first MIRIx Cambridge workshop (coauthored with Janos and Jim).
One potential problem with bounded utility functions is: what happens when the bound is nearly reached? A bounded utility maximizer will get progressively more and more risk averse as it gets closer to its bound. We decided to investigate what risks it might fear. We used a toy model with a bounded-utility chocolate maximizer, and considered what happens to its resource allocation in the limit as resources go to infinity.
We use “chocolate maximizer″ as conceptual shorthand meaning an agent that we model as though it has a single simple value with a positive long-run marginal resource cost, but only as a simplifying assumption. This is as opposed to a paperclip maximizer, where the inappropriate simplicity is implied to be part of the world, not just part of the model.
Conceptual uncertainty
We found that if a bounded utility function approaches its bound too fast, this has surprising pathological results when mixed with logical uncertainty. Consider a bounded-utility chocolate maximizer, with philosophical uncertainty about what chocolate is. It has a central concept of chocolate C1 , and there are classes of mutated versions of the concept of chocolate at varying distances from the central concept, such that the probability that the true chocolate is in class Ci is proportional to i−α (i.e. following a power law).
Suppose also that utility is bounded using a sigmoid function , where x is the amount of chocolate produced. In the limit as resources go to infinity, what fraction of those resources will be spent on the central class C1 ? That depends which sigmoid function is used, and in particular, how quickly it approaches the utility bound.
Example 1: exponential sigmoid
Suppose we allocate ri resources to class Ci , with for total resource r. Let .
Then the optimal resource allocation is
space;→rspace;space;=argmaxn∑i=1P(Ci)U(ri)space;space;=argmaxn∑i=1ci−α(1−e−ri)space;
Using Lagrange multipliers, we obtain for all i,
Then,
Thus, the resources will be evenly distributed among all the classes as r increases. This is bad, because the resource fraction for the central class C1 goes to 0 as we increase the number of classes.
EDITED: Addendum on asymptotics
Since we have both r and n going to infinity, we can specify their relationship more precisely. We assume that n is the highest number of classes that are assigned nonnegative resources for a given value of r:
space;logspace;nspace;≤space;rnαspace;1nn∑j=1logspace;jspace;space;=space;rnα1nlog(n!)space;≈space;rnαspace;logspace;nspace;−space;1space;space;1nO(logspace;n)space;
Thus,
so the highest class index that gets nonnegative resources satisfies
Example 2: arctan sigmoid
Let .
The optimal resource allocation is
space;→rspace;space;=argmaxn∑i=1P(Ci)U(ri)space;space;=argmaxn∑i=1ci−αarctan(ri)space;
Using Lagrange multipliers, we obtain for all i,
Then,
space;rispace;space;=√cλi−α−1≈√cλi−α/2space;rspace;space;=√cλn∑i=1i−α/2<√cλζ(α/2)space;rirspace;space;>i−α/2ζ(α/2)space;r0rspace;space;>1ζ(α/2)space;
Thus, for α>2 the limit of the resource fraction for the central class C1 is finite and positive.
Conclusion
The arctan sigmoid results in a better limiting resource allocation than the exponential sigmoid, because it has heavier tails (for sufficiently large α ). Thus, it matters which bounding sigmoid function you choose.