Similarly to “Zombies: The Movie”, this was very entertaining, but I don’t think I’ve learned anything new from it.
Z. M. Davis wrote:
Also, even if there are no moral facts, don’t you think the fact that no existing person would prefer a universe filled with paperclips …
Have you performed a comprehensive survey to establish this? Asserting “No existing person” in a civilization of 6.5e9 people amounts to assigning a probability of less than 1.54e-10 that a randomly chosen person would prefer a universe filled with paperclips. This is an extremely strong claim to make!
For example, note that the set of people alive includes a significant number of people who are certifiably insane, and in all probability others who, while reasonably sane, have gotten very fed up with various forms of torture inflicted on them over the last few days and might be willing to neglect collateral damage if they could make it stop.
If such a survey were performed, and the results were actually what you claim, I would assign a higher probability to the possibility of a nefarious anti-paperclip conspiracy having infiltrated the survey effort than to the possibility of the results being correct.
Unanimous agreement of our entire species is also a much stronger claim than you need to make for your argument.
I agree with your first statement: it is enjoyable to read fiction that can convincingly propose a difficult idea (here, that my fundamental assumptions are false), but the point behind (maybe about what is matter?) is too obfuscated to be applicable in another context, and so I do not grow as a rationalist, only as a sci-fi reader.
For your pro-paperclip argument, the probability is wrong: it assumes that people’s beliefs are independent, which is a dubious model. Perhaps a way to salvage your reasoning would be to consider the amount of classes of people who have the same opinion on paperclipping the universe for the same reason. I’d expect two such classes to have somewhat independent beliefs (as in, an argument that convinces one would not necessarily convince another). There aren’t a million ways to justify or refuse paperclipping the universe, though.
Using the honoured tradition of pulling numbers out of my arse, I’ll assume there are a thousand such classes. It does not seem that extreme to assign a probability less than e-4 that a random class would embrace paperclipping. It’s not outright out of my range of intuitively considerable probabilities the way e-10 is.
But regardless, Z. M. Davis’s argument does not hinge on unanimous agreement, it’s easy to see how even partial agreement can still be reworked into an argument Z. M. Davis would probably agree with: ”A certain amount of people agree that we should not paperclip the universe (for reasons unknown or too complicated to understand or that I just don’t want to disclose(*)); in the absence of information about the preferences of the rest of the people and in the absence of an object-level reason to paperclip the universe or not, that is evidence we should not paperclip the universe (exactly how much evidence depends on how confident you are about your own ability to discern the answer to this question relative to a random person’s ability, and how you adjust the information just given to take biases into account). Thanks to conservation of expected evidence, this means that even in the case where we are aware of other arguments, this one still holds some weight, if a reduced one.”
Similarly to “Zombies: The Movie”, this was very entertaining, but I don’t think I’ve learned anything new from it.
Z. M. Davis wrote:
Have you performed a comprehensive survey to establish this? Asserting “No existing person” in a civilization of 6.5e9 people amounts to assigning a probability of less than 1.54e-10 that a randomly chosen person would prefer a universe filled with paperclips. This is an extremely strong claim to make!For example, note that the set of people alive includes a significant number of people who are certifiably insane, and in all probability others who, while reasonably sane, have gotten very fed up with various forms of torture inflicted on them over the last few days and might be willing to neglect collateral damage if they could make it stop.
If such a survey were performed, and the results were actually what you claim, I would assign a higher probability to the possibility of a nefarious anti-paperclip conspiracy having infiltrated the survey effort than to the possibility of the results being correct.
Unanimous agreement of our entire species is also a much stronger claim than you need to make for your argument.
I agree with your first statement: it is enjoyable to read fiction that can convincingly propose a difficult idea (here, that my fundamental assumptions are false), but the point behind (maybe about what is matter?) is too obfuscated to be applicable in another context, and so I do not grow as a rationalist, only as a sci-fi reader.
For your pro-paperclip argument, the probability is wrong: it assumes that people’s beliefs are independent, which is a dubious model.
Perhaps a way to salvage your reasoning would be to consider the amount of classes of people who have the same opinion on paperclipping the universe for the same reason. I’d expect two such classes to have somewhat independent beliefs (as in, an argument that convinces one would not necessarily convince another). There aren’t a million ways to justify or refuse paperclipping the universe, though.
Using the honoured tradition of pulling numbers out of my arse, I’ll assume there are a thousand such classes. It does not seem that extreme to assign a probability less than e-4 that a random class would embrace paperclipping. It’s not outright out of my range of intuitively considerable probabilities the way e-10 is.
But regardless, Z. M. Davis’s argument does not hinge on unanimous agreement, it’s easy to see how even partial agreement can still be reworked into an argument Z. M. Davis would probably agree with:
”A certain amount of people agree that we should not paperclip the universe (for reasons unknown or too complicated to understand or that I just don’t want to disclose(*)); in the absence of information about the preferences of the rest of the people and in the absence of an object-level reason to paperclip the universe or not, that is evidence we should not paperclip the universe (exactly how much evidence depends on how confident you are about your own ability to discern the answer to this question relative to a random person’s ability, and how you adjust the information just given to take biases into account).
Thanks to conservation of expected evidence, this means that even in the case where we are aware of other arguments, this one still holds some weight, if a reduced one.”
Remember you have to face not the argument given by Z. M. Davis, but the best one that he could have come up with if you are to show his reasoning is not sound.