Psy-Kosh: I understand the metarationality arguments; my point is that we didn’t defect in a prisoner’s dilemma. PD requires C/C to be preferable to D/D; but if destroying Huygens is defecting for humans, that can only be the case (under the story’s values) if cooperating for Superhappies involves modifying themselves and/or giving us their tech without us being modified. I don’t think that was ever on the table. (BTW, I liked your explanation of why the deal isn’t so bad.)
Simon: Eliezer tries to derive his morality from human values… Common mistake; see No License to Be Human.
Thom: What do you mean by “naturalistic fallacy fallacy”? Google reveals several usages, none of which seem to fit. Also, regardless of Simon’s actual values, it seemed to me he treated the statements “I buy ice cream instead of helping starving children” and “I value ice cream over helping starving children” as identical; this is a fallacy that I happen to find particularly annoying.
Psy-Kosh: I understand the metarationality arguments; my point is that we didn’t defect in a prisoner’s dilemma. PD requires C/C to be preferable to D/D; but if destroying Huygens is defecting for humans, that can only be the case (under the story’s values) if cooperating for Superhappies involves modifying themselves and/or giving us their tech without us being modified. I don’t think that was ever on the table. (BTW, I liked your explanation of why the deal isn’t so bad.)
Simon: Eliezer tries to derive his morality from human values… Common mistake; see No License to Be Human.
Thom: What do you mean by “naturalistic fallacy fallacy”? Google reveals several usages, none of which seem to fit. Also, regardless of Simon’s actual values, it seemed to me he treated the statements “I buy ice cream instead of helping starving children” and “I value ice cream over helping starving children” as identical; this is a fallacy that I happen to find particularly annoying.