Robin Hanson has suggested penalizing the prior probability of hypotheses which argue that we are in a surprisingly unique position to affect large numbers of other people who cannot symmetrically affect us. Since only one in 3^^^^3 people can be in a unique position to ordain the existence of at least 3^^^^3 other people who are not symmetrically in such a situation themselves, the prior probability would be penalized by a factor on the same order as the utility. ( http://wiki.lesswrong.com/mediawiki/index.php?title=Pascal%27s_mugging )
This seems like a hack around the problem.
What if we are told there’s an infinite number of people, so everybody could affect 3^^^^3 other people (per Hilbert’s Hotel)?
What consequences would this prior lead to—assuming that the odds of us making a successful AI are 1/some-very-large-number, because a successful AI could go on to control everything within our light cone and for the rest of history affect the lives of some-very-large-number of beings?
(For that matter, wouldn’t this solution have us bite the bullet of the Doomsday argument in general, and assume that we and our creations will expire soon because otherwise, how likely was it that we would just happen to exist near the beginning of the universe/humanity and thus be in a position to affect the yawning eons after us?)
To resurrect the Pascal’s mugging problem:
This seems like a hack around the problem.
What if we are told there’s an infinite number of people, so everybody could affect 3^^^^3 other people (per Hilbert’s Hotel)?
What consequences would this prior lead to—assuming that the odds of us making a successful AI are 1/some-very-large-number, because a successful AI could go on to control everything within our light cone and for the rest of history affect the lives of some-very-large-number of beings?
(For that matter, wouldn’t this solution have us bite the bullet of the Doomsday argument in general, and assume that we and our creations will expire soon because otherwise, how likely was it that we would just happen to exist near the beginning of the universe/humanity and thus be in a position to affect the yawning eons after us?)