I hadn’t considered the problem of finite axioms giving you unboundedly large likelihood ratios over your exact situation.
Really? People have been raising this (worlds with big payoffs and in which your observations are not correspondingly common) from the very beginning. E.g. in the comments of your original Pascal’s Mugging post in 2007, Michael Vassar raised the point:
The guy with the button could threaten to make an extra-planar factory farm containing 3^^^^^3 pigs instead of killing 3^^^^3 humans. If utilities are additive, that would be worse.
and you replied:
Congratulations, you made my brain asplode.
Wei Dai and Rolf Nelson discussed the issue further in the comments there, and from different angles. And it is the obvious pattern-completion for “this argument gives me nigh-infinite certainty given its assumptions—now do I have nigh-infinite certainty in the assumptions?” i.e. Probing the Improbable issues. This is how I explained the unbounded payoffs issue to Steven Kaas when he asked for feedback on earlier drafts of his recent post about expected value and extreme payoffs (note how he talks about our uncertainty re anthropics and the other conditions required for Hanson’s anthropic argument to go through).
Really? People have been raising this (worlds with big payoffs and in which your observations are not correspondingly common) from the very beginning. E.g. in the comments of your original Pascal’s Mugging post in 2007, Michael Vassar raised the point:
and you replied:
Wei Dai and Rolf Nelson discussed the issue further in the comments there, and from different angles. And it is the obvious pattern-completion for “this argument gives me nigh-infinite certainty given its assumptions—now do I have nigh-infinite certainty in the assumptions?” i.e. Probing the Improbable issues. This is how I explained the unbounded payoffs issue to Steven Kaas when he asked for feedback on earlier drafts of his recent post about expected value and extreme payoffs (note how he talks about our uncertainty re anthropics and the other conditions required for Hanson’s anthropic argument to go through).