Good essay (perhaps wasted on TVTropes?). While I agree on the point being made, I don’t think it’s the critical one.
My own opinion is that Eliezer’s preference for the True Ending does not simply proceed from his conflation of the True Ending with wireheading, but straight from his individually absolutist approach to morality. The effect is so jarring* because the overarching theme of TWC is to make the reader think about the nature of morality, introducing scenarios designed to knock the feet out from under its most common naive conceptions… and then the author goes and endorses a choice (if you can even call it that) that doggedly refuses to make any use of all that analysis of morality. It inherits the failure-to-dissect that closes the Metaethics sequence.
(* At least, I suspect, to the likes of us. One reader with whom I shared it commented that the True Ending was by far his favourite part, because it felt ‘like dropping back to the ground’.)
That the ‘True’ choice would have been reached by most people without analysis doesn’t mean it’s not also the choice that would be reached by analysis.
Whether the choice made by the crew in the True Ending was the correct one for most humans, I don’t know, because I don’t think Eliezer gave us enough information about what humanity would be transformed into by the Superhappies. Getting rid of ‘physical’ pain is mostly good, but some of the changes (such as getting rid of ‘embarrassment’) sounded pretty bad to me.
Regardless, I certainly didn’t observe any failure-to-dissect, as you put it.
Just what do you think the TVTropes demographic is? I mean, do you believe that the average troper is bound to be incapable of understanding the piece, appreciating it and taking something from it?
(note that I don’t identify as a “troper” at all, although I do waste a lot of time there)
I think the comparison to wireheading falls flat, because the Superhappies don’t seem like wireheads. They have desires regarding the external world, and seek to fulfill them, they experience sorrow, and demonstrate acute empathy. They have every appearance of a complex emotional life. Getting rid of bodily pain, embarrassment and romantic difficulties might sound like it would make our lives less interesting, complex and fulfilling, but in-story, we can just look at the Superhappies, and say “they look pretty well off.”
That’s actually my point—if you look only at the behavior of the humans, it sure looks like they’re saying, “no wireheading for us!”, even though the alternative they are rejecting isn’t.
You are taking wireheading too literally. Wireheaded animals are not actually having fun, but in most discussions of wireheading, we are taking about potential future technologies, and the least convenient possible world usually contains wireheads that are really happy.
Tangential note: I see a potential for readers of your essay to get a misleading picture of what wireheading actually is in reality. Maybe you could add a footnote with some more info?
I think the main reason the “True Ending” is jarring to us is that the future humans have quite different values from early-21st-century humans—they rape each other and intentionally put their children through painful experiences (maybe they took a wrong turn somewhere along the line, as one of the characters suggests). Our own values are probably closer to the Superhappies than the Future Humans, so naturally we prefer the happy ending.
they rape each other and intentionally put their children through painful experiences
This is just plain wrong on both counts. The non-consensual initiation of sex becoming equated to non-consensual initiation of conversation is very reasonably explained and investigated elsewhere on LW. (I still suspect that it would work a little poorly, and could be a naive policy.) Humans are probably more careful then now about letting their children bump their knees in all sorts of ways. I imagine their upbringing to be very awesome indeed, more fun and less painful then today, and totally worth it.
That is something that bothers a number of readers (and I disagree about the “intentionally”—I believe the story stated that they didn’t completely dull the pain, not that they caused it), but I don’t think it’s the dominant factor behind the disagreement.
Good essay (perhaps wasted on TVTropes?). While I agree on the point being made, I don’t think it’s the critical one.
My own opinion is that Eliezer’s preference for the True Ending does not simply proceed from his conflation of the True Ending with wireheading, but straight from his individually absolutist approach to morality. The effect is so jarring* because the overarching theme of TWC is to make the reader think about the nature of morality, introducing scenarios designed to knock the feet out from under its most common naive conceptions… and then the author goes and endorses a choice (if you can even call it that) that doggedly refuses to make any use of all that analysis of morality. It inherits the failure-to-dissect that closes the Metaethics sequence.
(* At least, I suspect, to the likes of us. One reader with whom I shared it commented that the True Ending was by far his favourite part, because it felt ‘like dropping back to the ground’.)
That is another good point—Eliezer’s metaethics were certainly a sticking point when I was discussing the idea of FAI with a philosopher.
That the ‘True’ choice would have been reached by most people without analysis doesn’t mean it’s not also the choice that would be reached by analysis.
Whether the choice made by the crew in the True Ending was the correct one for most humans, I don’t know, because I don’t think Eliezer gave us enough information about what humanity would be transformed into by the Superhappies. Getting rid of ‘physical’ pain is mostly good, but some of the changes (such as getting rid of ‘embarrassment’) sounded pretty bad to me.
Regardless, I certainly didn’t observe any failure-to-dissect, as you put it.
Just what do you think the TVTropes demographic is? I mean, do you believe that the average troper is bound to be incapable of understanding the piece, appreciating it and taking something from it?
(note that I don’t identify as a “troper” at all, although I do waste a lot of time there)
I think the comparison to wireheading falls flat, because the Superhappies don’t seem like wireheads. They have desires regarding the external world, and seek to fulfill them, they experience sorrow, and demonstrate acute empathy. They have every appearance of a complex emotional life. Getting rid of bodily pain, embarrassment and romantic difficulties might sound like it would make our lives less interesting, complex and fulfilling, but in-story, we can just look at the Superhappies, and say “they look pretty well off.”
On a side note, it’s not clear that wireheads are actually happy.
That’s actually my point—if you look only at the behavior of the humans, it sure looks like they’re saying, “no wireheading for us!”, even though the alternative they are rejecting isn’t.
You are taking wireheading too literally. Wireheaded animals are not actually having fun, but in most discussions of wireheading, we are taking about potential future technologies, and the least convenient possible world usually contains wireheads that are really happy.
Which is why it’s a side note, rather than the main point I intended to bring to the discussion.
Tangential note: I see a potential for readers of your essay to get a misleading picture of what wireheading actually is in reality. Maybe you could add a footnote with some more info?
Added.
This is a fine essay and it probably helped quite a few people resolve their inner turmoil after the choice.
I think the main reason the “True Ending” is jarring to us is that the future humans have quite different values from early-21st-century humans—they rape each other and intentionally put their children through painful experiences (maybe they took a wrong turn somewhere along the line, as one of the characters suggests). Our own values are probably closer to the Superhappies than the Future Humans, so naturally we prefer the happy ending.
This is just plain wrong on both counts. The non-consensual initiation of sex becoming equated to non-consensual initiation of conversation is very reasonably explained and investigated elsewhere on LW. (I still suspect that it would work a little poorly, and could be a naive policy.) Humans are probably more careful then now about letting their children bump their knees in all sorts of ways. I imagine their upbringing to be very awesome indeed, more fun and less painful then today, and totally worth it.
Where?
What do you base this on?
See the links in my post and the post itself.
Intuition, and my general estimate about that society being more loving in all personal contexts than ours.
That is something that bothers a number of readers (and I disagree about the “intentionally”—I believe the story stated that they didn’t completely dull the pain, not that they caused it), but I don’t think it’s the dominant factor behind the disagreement.