In one model I considered I put tegmark 4 as the one weighted according to my values, and called the set of different counterfactual universes other agents might care about as tegmark 5. This was mainly for the purpose of fiction where it filled a role as a social convention among agents with very different values of this type, but it’s an interesting idea of what the concept might look like.
These could by the way not necessarily be just quantitatively different weights over the same set of universes. For example we can imagine that it turns out humans and human derived agents are solomnoff induction like and only value things describable by turing machines computing them causally, but some other things value only the outputs of continuous functions.
In one model I considered I put tegmark 4 as the one weighted according to my values, and called the set of different counterfactual universes other agents might care about as tegmark 5. This was mainly for the purpose of fiction where it filled a role as a social convention among agents with very different values of this type, but it’s an interesting idea of what the concept might look like.
These could by the way not necessarily be just quantitatively different weights over the same set of universes. For example we can imagine that it turns out humans and human derived agents are solomnoff induction like and only value things describable by turing machines computing them causally, but some other things value only the outputs of continuous functions.