If an AI research laboratory poised to create a superintelligence could at the same cost carry out something like CEV, MR, or a ‘CEV’ of their own group, under what circumstances should they choose each?
MR seems correct if we have evidence that points to CEV containing unacceptable amounts of unavoidable suffering because we are just unlucky and our true values suck.
Limited CEV seems correct when we have evidence that some class of agents don’t have moral worth and shouldn’t have their preferences taken into account. Most conceptions of CEV have some sort of limitation that isn’t obviously a correct carving of reality at the joints. Choosing between “all humans CEV” and “all humans + extrapolation of uplifted minds” and “CEV of various future minds that we would value” is non-obvious for example.
Choosing between “all humans CEV” and “all humans + extrapolation of uplifted minds” and “CEV of various future minds that we would value” is non-obvious for example.
If an AI research laboratory poised to create a superintelligence could at the same cost carry out something like CEV, MR, or a ‘CEV’ of their own group, under what circumstances should they choose each?
MR seems correct if we have evidence that points to CEV containing unacceptable amounts of unavoidable suffering because we are just unlucky and our true values suck.
Limited CEV seems correct when we have evidence that some class of agents don’t have moral worth and shouldn’t have their preferences taken into account. Most conceptions of CEV have some sort of limitation that isn’t obviously a correct carving of reality at the joints. Choosing between “all humans CEV” and “all humans + extrapolation of uplifted minds” and “CEV of various future minds that we would value” is non-obvious for example.
Take an average.
Weighting the uncertainty and value of each CEV is non-obvious, so you’ve pushed the uncertainty up one level.
True, but it seems easier to compromise on.