Furthermore, desirability of the CEV ouput to the average human in existence today should be weighed against the desires of (for example) sentient human uploads created in a post-singularity scenario.
No, it really shouldn’t. CEV is inclusive. That is, if we care about post human sentient uploads then CEV accounts for that better than we can. That’s the whole point. (This holds unless ‘desirability’ is defined to mean some other arbitrary thing independent of volition of the type we are talking about.)
That depends who “we” is referring to. What I meant to say is that if the FAI programmers and other intellectuals believe that extrapolated humans in general might not care about harming uploads (future humans with no voice in that CEV) - whereas a selective CEV of intellectuals is expected to do so—then they should consider this when deciding whether to allow all of humanity to have a say in the CEV, rather than a subset of minds whom they consider to be safer in that regard.
So even if the implementation of a universal CEV as initial dynamic can be expected to reflect the desires of humanity en masse better than a selective initial CEV, this doesn’t define the total moral space that should be of concern to those responsible for and having the power to influence that implementation.
No, it really shouldn’t. CEV is inclusive. That is, if we care about post human sentient uploads then CEV accounts for that better than we can. That’s the whole point. (This holds unless ‘desirability’ is defined to mean some other arbitrary thing independent of volition of the type we are talking about.)
That depends who “we” is referring to. What I meant to say is that if the FAI programmers and other intellectuals believe that extrapolated humans in general might not care about harming uploads (future humans with no voice in that CEV) - whereas a selective CEV of intellectuals is expected to do so—then they should consider this when deciding whether to allow all of humanity to have a say in the CEV, rather than a subset of minds whom they consider to be safer in that regard.
So even if the implementation of a universal CEV as initial dynamic can be expected to reflect the desires of humanity en masse better than a selective initial CEV, this doesn’t define the total moral space that should be of concern to those responsible for and having the power to influence that implementation.