They create staples, too. Do you think humanity’s CEV will maximize staples? The point of my argument is that those things are inconsistent. You can only maximize one thing, and there is no human reason for that to be paperclips.
At humans’ current stage of civilization and general reflective coherence, their terminal values are still deeply intertwined with their instrumental values, and the political-orientedness of their cognitive architecture doesn’t help. So I would say that instrumental values do indeed matter in this case.
They create staples, too. Do you think humanity’s CEV will maximize staples? The point of my argument is that those things are inconsistent. You can only maximize one thing, and there is no human reason for that to be paperclips.
All true (up to “there is no human reason...”). But can we at least agree that you went too far in saying that humans “don’t care” about paperclips?
I care about paperclips!
Good! Just remember, you still have room for improvement. ⊂≣⊇
No, I meant they don’t care about them as a terminal value, which is all that matters for this discussion.
At humans’ current stage of civilization and general reflective coherence, their terminal values are still deeply intertwined with their instrumental values, and the political-orientedness of their cognitive architecture doesn’t help. So I would say that instrumental values do indeed matter in this case.