What you’re looking for is a way to construe the extrapolated volition that washes out superstition and dementation.
you could do that. But if you want a clean shirt out of the washing machine, you don’t add in a diaper with poo on it and then look for a really good laundry detergent to “wash it out”.
My feeling with the CEV of humanity is that if it is highly insensitive to the set of people you extrapolate, then you lose nothing by extrapolating fewer people. On the other hand, if including more people does change the answer in a direction that you regard as bad, then you gain by excluding people with values dissimilar from yours.
Furthermore, excluding people from the CEV process doesn’t mean disenfranchising them—it just means enfranchising them according to what your values count as enfranchisement.
Most people in the world don’t hold our values(1). Read, e.g. Haidt on Culturally determined values. Human values are universal in form but local in content. Our should function is parochial.
(1 - note—this doesn’t mean that they will be different after extrapolation. f(x) can equal f(y) for x!=y. But it does mean that they might, which is enough to give you an incentive not to include them)
if you want a clean shirt out of the washing machine, you don’t add in a diaper with poo on it and then look for a really good laundry detergent to “wash it out”.
I want to claim that a Friendly initial dynamic should be more analogous to a biosphere-with-a-textile-industry-in-it machine than to a washing machine. How do we get clean shirts at all, in a world with dirty diapers?
But then, it’s a strained analogy; it’s not like we’ve ever had a problem of garments claiming control over the biosphere and over other garments’ cleanliness before.
you could do that. But if you want a clean shirt out of the washing machine, you don’t add in a diaper with poo on it and then look for a really good laundry detergent to “wash it out”.
My feeling with the CEV of humanity is that if it is highly insensitive to the set of people you extrapolate, then you lose nothing by extrapolating fewer people. On the other hand, if including more people does change the answer in a direction that you regard as bad, then you gain by excluding people with values dissimilar from yours.
Furthermore, excluding people from the CEV process doesn’t mean disenfranchising them—it just means enfranchising them according to what your values count as enfranchisement.
Most people in the world don’t hold our values(1). Read, e.g. Haidt on Culturally determined values. Human values are universal in form but local in content. Our should function is parochial.
(1 - note—this doesn’t mean that they will be different after extrapolation. f(x) can equal f(y) for x!=y. But it does mean that they might, which is enough to give you an incentive not to include them)
I want to claim that a Friendly initial dynamic should be more analogous to a biosphere-with-a-textile-industry-in-it machine than to a washing machine. How do we get clean shirts at all, in a world with dirty diapers?
But then, it’s a strained analogy; it’s not like we’ve ever had a problem of garments claiming control over the biosphere and over other garments’ cleanliness before.