When reading the post, I immediately pictured a white guy in the 18th century saying something like “Of course we shouldn’t give blacks rights based on the reasons you suggested, that would imply that the next thing would be giving rights to women!” (and the rejection of the latter being motivated by a false map of the world).
I see no reason why Armstrong-1 needs to like or at least accept the conclusions of Armstrong-n, assuming he is not yet aware of all the inferential steps. The premise of CEV, as I understand it, is that each step by itself is sound and in accordance with the terminal value(s) of Armstrong-1. Whether this works and whether we can check if it worked is a different question.
Even if you think changing one’s stated (as opposed to terminal) utility function is always done based on comparing outcomes for intuitiveness and consistency, a position I’m sympathetic towards, it would be irrational for Armstrong-1 to reject Armstrong-n’s values just because they lead to very counterintuitive conclusions. The reason being that Armstrong-1 may not be aware, due to ignorance or mistakes in reasoning, that his own values imply conclusions that are even more absurd.
In general, I think people on LW are way too quick to declare something to be their terminal value (which is of course somewhat of a self-fulfilling prophecy).
Of course the sequence presented seems ridiculous, and it were the actual output of CEV, I would be virtually certain that something went wrong. However, that is based on the reasoning outlined at each step, not due to the final conclusion. All I’m saying is that Armstrong-1 has no vote on the final output before actually having gone through all the arguments. Not even if the outcome would be something counterintuitive like negative utilitarianism. (BTW, I have yet to hear a take on population ethics that doesn’t include conclusions that are highly counterintuitive!)
(BTW, I have yet to hear a take on population ethics that doesn’t include conclusions that are highly counterintuitive!)
You will never hear of such a take, since it’s been shown that all population theories will violate at least one highly intuitive criterion of adequacy. See Blackorby, Bossert & Donaldson (2003) and Arrhenius (2000). Unfortunately few people in this community seem to be aware of these results.
When reading the post, I immediately pictured a white guy in the 18th century saying something like “Of course we shouldn’t give blacks rights based on the reasons you suggested, that would imply that the next thing would be giving rights to women!” (and the rejection of the latter being motivated by a false map of the world).
I see no reason why Armstrong-1 needs to like or at least accept the conclusions of Armstrong-n, assuming he is not yet aware of all the inferential steps. The premise of CEV, as I understand it, is that each step by itself is sound and in accordance with the terminal value(s) of Armstrong-1. Whether this works and whether we can check if it worked is a different question.
Even if you think changing one’s stated (as opposed to terminal) utility function is always done based on comparing outcomes for intuitiveness and consistency, a position I’m sympathetic towards, it would be irrational for Armstrong-1 to reject Armstrong-n’s values just because they lead to very counterintuitive conclusions. The reason being that Armstrong-1 may not be aware, due to ignorance or mistakes in reasoning, that his own values imply conclusions that are even more absurd.
In general, I think people on LW are way too quick to declare something to be their terminal value (which is of course somewhat of a self-fulfilling prophecy).
Of course the sequence presented seems ridiculous, and it were the actual output of CEV, I would be virtually certain that something went wrong. However, that is based on the reasoning outlined at each step, not due to the final conclusion. All I’m saying is that Armstrong-1 has no vote on the final output before actually having gone through all the arguments. Not even if the outcome would be something counterintuitive like negative utilitarianism. (BTW, I have yet to hear a take on population ethics that doesn’t include conclusions that are highly counterintuitive!)
You will never hear of such a take, since it’s been shown that all population theories will violate at least one highly intuitive criterion of adequacy. See Blackorby, Bossert & Donaldson (2003) and Arrhenius (2000). Unfortunately few people in this community seem to be aware of these results.