Step 1 -- Replace humanity with a sufficiently large number of paperclip satisficers (they want to make only a few very specific paperclips to be absolutely happy, but each of them wants a different kind of a paperclip). If you give a non-zero value to a paperclips satisficer, then for a sufficiently large number of paperclip satisficers, replacing humanity with them should be considered a good choice.
Step 2 -- Paperclip satisficers replace themselves with paperclips. (Because they are satisficers and not maximizers, they don’t care about expanding and transforming the whole universe.)
Is the size of the cosmological event horizon until the heat death of the local area large enough to postulate enough paperclip satisficers? Since we are dealing with finite time and space, it is not a given that a sufficiently large number of anything would be possible, and we are discussing how large numbers are counterintuititve rather than a moral point.
I would argue that the solution to this may be to make the value of having humans, or some other type of moral being, exist, be discontinuous in some fashion.
In other words, adding paperclip satisficers might add some value, but in a world where no humans (or other type of moral being) exists there is no amount of paperclip satisficers you can add that would be as valuable as creating humans. Similarly, there is no amount of paperclip satisficers that can be added that could ever replace the human race.
Now please note that I am not trying to be speciesist when I talk about humans and human values. A creature that shares human values, but isn’t genetically a member of the human species, is far more worth creating than a creature that is genetically a member of the human species, but lacks certain values. For instance, an alien that had evolved human-like values through parallel evolution is far more worth creating than a human sociopath.
But what if mere paper clip satisficers simply aren’t capable of experiencing the full joy of being a great paperclip maximiser who has paperclipped the universe? I think the simplest solution is to measure joy in number of paperclips made =]
Paperclip satisficers, by definition, are completely happy with producing their paperclips, and absolutely don’t mind missing the joys of being a paperclip maximizer.
Essentially, it is a question of what happens first. If as a first step we replace all human beings with enough paperclip satisficers to increase the total utility, in the resulting world nobody will attempt to create a paperclip maximizer. If something else happens first, the results may be completely different.
Step 1 -- Replace humanity with a sufficiently large number of paperclip satisficers (they want to make only a few very specific paperclips to be absolutely happy, but each of them wants a different kind of a paperclip). If you give a non-zero value to a paperclips satisficer, then for a sufficiently large number of paperclip satisficers, replacing humanity with them should be considered a good choice.
Step 2 -- Paperclip satisficers replace themselves with paperclips. (Because they are satisficers and not maximizers, they don’t care about expanding and transforming the whole universe.)
Game Over
Is the size of the cosmological event horizon until the heat death of the local area large enough to postulate enough paperclip satisficers? Since we are dealing with finite time and space, it is not a given that a sufficiently large number of anything would be possible, and we are discussing how large numbers are counterintuititve rather than a moral point.
I would argue that the solution to this may be to make the value of having humans, or some other type of moral being, exist, be discontinuous in some fashion.
In other words, adding paperclip satisficers might add some value, but in a world where no humans (or other type of moral being) exists there is no amount of paperclip satisficers you can add that would be as valuable as creating humans. Similarly, there is no amount of paperclip satisficers that can be added that could ever replace the human race.
Now please note that I am not trying to be speciesist when I talk about humans and human values. A creature that shares human values, but isn’t genetically a member of the human species, is far more worth creating than a creature that is genetically a member of the human species, but lacks certain values. For instance, an alien that had evolved human-like values through parallel evolution is far more worth creating than a human sociopath.
But what if mere paper clip satisficers simply aren’t capable of experiencing the full joy of being a great paperclip maximiser who has paperclipped the universe? I think the simplest solution is to measure joy in number of paperclips made =]
Paperclip satisficers, by definition, are completely happy with producing their paperclips, and absolutely don’t mind missing the joys of being a paperclip maximizer.
Essentially, it is a question of what happens first. If as a first step we replace all human beings with enough paperclip satisficers to increase the total utility, in the resulting world nobody will attempt to create a paperclip maximizer. If something else happens first, the results may be completely different.