Personally I like Unsong’s God, and I think His approach is better than tiling the Universe with copies of the same optimal entity (or copies of an optimal neighborhood where each being can encounter enough diversity to satisfy them in their own neighborhood).
The Unsong approach might still lead to uncomfortable outcomes with some people tortured to make other people have different positive experiences than the ones already tried (hence the solution to the Problem of evil in Unsong), but I think that with giving big enough negative utilities to suffering, the system probably wouldn’t create people with overall very net-negative lives (and maybe put suffering p-zombie robots in the world if that’s really necessary for other people having novel positive experiences). These are just my guesses and I’m not confident that we can actually make this right, as I mentioned, I wouldn’t want to create any kind of utilitarian sovereign superintelligence. But I think that the weird asymmetry baked in infra-Bayesianism that it can’t give negative utility to any event makes the whole problem significantly harder and points at a weakness of IB.
Personally I like Unsong’s God, and I think His approach is better than tiling the Universe with copies of the same optimal entity (or copies of an optimal neighborhood where each being can encounter enough diversity to satisfy them in their own neighborhood).
The Unsong approach might still lead to uncomfortable outcomes with some people tortured to make other people have different positive experiences than the ones already tried (hence the solution to the Problem of evil in Unsong), but I think that with giving big enough negative utilities to suffering, the system probably wouldn’t create people with overall very net-negative lives (and maybe put suffering p-zombie robots in the world if that’s really necessary for other people having novel positive experiences). These are just my guesses and I’m not confident that we can actually make this right, as I mentioned, I wouldn’t want to create any kind of utilitarian sovereign superintelligence. But I think that the weird asymmetry baked in infra-Bayesianism that it can’t give negative utility to any event makes the whole problem significantly harder and points at a weakness of IB.