It’s kinda messy how to think about non-consensual uploads that the person is totally fine with after some reflection (especially if the reason the AI did the upload is because it know the person would be fine with this on reflection). I also don’t think considering uploads without informed consent to be death makes a big difference to my numbers.
I think this sort of thing is also why I don’t think the “AI may be very slightly nice” is likely to result in something that’s clearly “not death”. It seems really unlikely to me that very slightly nice things would go the “preserve parochial humans exactly as is”, it’s just such a narrow target to hit even within niceness.
I don’t really agree? It seems like “don’t upload people without their consent when they consider this to be death unless there are no better options” is pretty natural and the increase in resources for keeping people physically alive is pretty small.
(FYI I think I am sold by your other reply that the main question is “how much is it slowed down initially, and how much does it value the resources that it loses by doing so?”. I agree leaving us with one star isn’t that big a deal, modulo making sure we can’t somehow mess up the rest of it’s plans, which doesn’t seem that hard)
It’s kinda messy how to think about non-consensual uploads that the person is totally fine with after some reflection (especially if the reason the AI did the upload is because it know the person would be fine with this on reflection). I also don’t think considering uploads without informed consent to be death makes a big difference to my numbers.
I don’t really agree? It seems like “don’t upload people without their consent when they consider this to be death unless there are no better options” is pretty natural and the increase in resources for keeping people physically alive is pretty small.
(FYI I think I am sold by your other reply that the main question is “how much is it slowed down initially, and how much does it value the resources that it loses by doing so?”. I agree leaving us with one star isn’t that big a deal, modulo making sure we can’t somehow mess up the rest of it’s plans, which doesn’t seem that hard)