If future AIs can do such things, they can also probably run “rescue sims” that save you from awful things happening in your life. If you have ever experienced awful suffering that shouldn’t look OK to an AI running CEV, this indicates future AIs likely won’t be saving people as you suggest. Note that observing other people’s awful suffering shouldn’t count as evidence to you, because you have no access to their subjective anticipation (the branches of other people you see may all have very low probability). Freaky.
A “rescue sim” could possibly branch a person before they have some horrible experience, which would result in a copy that never had that experience, but instance that does have the experience would still exist.
If future AIs can do such things, they can also probably run “rescue sims” that save you from awful things happening in your life. If you have ever experienced awful suffering that shouldn’t look OK to an AI running CEV, this indicates future AIs likely won’t be saving people as you suggest. Note that observing other people’s awful suffering shouldn’t count as evidence to you, because you have no access to their subjective anticipation (the branches of other people you see may all have very low probability). Freaky.
A “rescue sim” could possibly branch a person before they have some horrible experience, which would result in a copy that never had that experience, but instance that does have the experience would still exist.