Anatomy of Multiversal Utility Functions: Tegmark Level IV
Outline: Constructing utility functions that can be evaluated on any possible universe is known to be a confusing problem, since it is not obvious what sort of mathematical object should be the domain and what properties should the function obey. In a sequence of posts, I intend break down the question with respect to Tegmark’s multiverse levels and explain the answer on each level, starting with level IV in the current post.
An intelligent agent is often described as an entity whose actions drive the universe towards higher expectation values of a certain function, known as the agent’s utility function. Such a description is very useful in contexts such as AGI, FAI, decision theory and more generally any abstract study of intelligence.
Applying the concept of a utility function to agents in the real worlds requires utility functions with a very broad domain. Indeed, since the agent is normally assumed to have only finite information about the universe in which it exists, it should allow for a very large variety of possible realities. If the agent is to make decisions using some sort of utility calculus, it has to be able to evaluate its utility function on each of the realities it can conceive.
Tegmark has conveniently arranged the space of possible realities (“universes”) into 4 levels, 3 of which are based on our current understanding of physics. Tegmark’s universes are usually presented as co-existing but it is also possible to think of them as the “potential” universes in which our agent can find itself. I am going to traverse Tegmark’s multiverse from top to bottom, studying the space of utility functions on each level (which, except for level IV, is always derived from the higher level). The current post addresses Tegmark level IV, leaving the lower levels for follow-ups.
Some of the ideas in this post previously appeared in a post about intelligence metrics, where I explained them much more tersely.
Tegmark Level IV
Tegmark defined this level as the collection of all mathematical models. Since it is not even remotely clear how to define such a beast, I am going to use a different space which (I claim) is conceptually very close. Namely, I am going to consider universes to be infinite binary sequences
The natural a priori probability measure on this space is the Solomonoff measure
From the point of view of Updateless Decision Theory, a priori expectation values are the only sort that matters: conditional expectation values wrt logical uncertainty replace the need to update the measure.
In order to guarantee the convergence of expectation values, we are going to assume
A Simple Example
So far, we know little about the form of the function
The glider maximizer
We wish to “release”
To accomplish this, fix a way
In other words, the “liberated”
Superficially, it seems that the construction of
Human Preferences and Dust Theory
Human preferences revolve around concepts which belong to an “innate” model of reality: a model which is either genetic or acquired by brains at early stages of childhood. This model describes the world mostly in terms of humans, their emotions and interactions (but might include other elements as well e.g. elements related to wild nature).
Therefore, utility functions which are good descriptions of human preferences (“friendly” utility functions) are probably of similar form to
Applying UDT to the
The Procrastination Paradox
Consider an agent
My point of view on the paradox is that it is the result of a topological pathology of
Buttonverse histories are naturally described as binary sequences
Consider the following sequence of buttonverse histories:
Now, with respect to the product topology on
However the utilities don’t behave correspondingly:
Therefore, it seems natural to require any utility function to be an upper semicontinuous function on X 2. I claim that this condition resolves the paradox in the precise mathematical sense considered in Yudkowsky 2013. Presenting the detailed proof would take us too far afield and is hence out of scope for this post.
Bounded utility functions typically contain some kind of temporal discount. In the Game of Life example, the discount manifests as the factor
Note that any sequence
As a result, the temporal discount function effectively undergoes convolution with the function
In other words, if a utility function
Next in sequence: The Role of Physics in UDT, Part I
1 It might seem that there are “realities” of higher set theoretic cardinality which cannot be encoded. However, if we assume our agent’s perceptions during a finite span of subjective time can be encoded as a finite number of bits, then we can safely ignore the “larger” realities. They can still exist as models the agent uses to explain its observations but it is unnecessary to assume them to exist on the “fundamental” level.
2 In particular, all computable functions are admissible since they are continuous.
3 I think that