[SEQ RERUN] Amputation of Destiny

MinibearRex16 Jan 2013 6:02 UTC
4 points
3 commentsLW linkArchive

Today’s post, Amputation of Destiny was originally published on 29 December. A summary (taken from the LW wiki):

C. S. Lewis’s Narnia has a problem, and that problem is the super-lion Aslan—who demotes the four human children from the status of main characters, to mere hangers-on while Aslan does all the work. Iain Banks’s Culture novels have a similar problem; the humans are mere hangers-on of the superintelligent Minds. We already have strong ethical reasons to prefer to create nonsentient AIs rather than sentient AIs, at least at first. But we may also prefer in just a fun-theoretic sense that we not be overshadowed by hugely more powerful entities occupying a level playing field with us. Entities with human emotional makeups should not be competing on a level playing field with superintelligences—either keep the superintelligences off the playing field, or design the smaller (human-level) minds with a different emotional makeup that doesn’t mind being overshadowed.


Discuss the post here (rather than in the comments to the original post).

This post is part of the Rerunning the Sequences series, where we’ll be going through Eliezer Yudkowsky’s old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Can’t Unbirth a Child, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.

Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day’s sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.

MinibearRex16 Jan 2013 6:02 UTC
4 points
3 commentsLW linkArchive