Seconding Kaj_Sotala’s question. Is there a good argument why self-improvement doesn’t have diverging paths due to small differences in starting conditions?
Dunno. CEV actually contains the phrase, “and had grown up farther together,” which the above leaves out. But I feel a little puzzled about the exact phrasing, which does not make “were more the people we wished we were” conditional on this other part—I thought the main point was that people “alone in a padded cell,” as Eliezer puts it there, can “wish they were” all sorts of Unfriendly entities.
Seconding Kaj_Sotala’s question. Is there a good argument why self-improvement doesn’t have diverging paths due to small differences in starting conditions?
Dunno. CEV actually contains the phrase, “and had grown up farther together,” which the above leaves out. But I feel a little puzzled about the exact phrasing, which does not make “were more the people we wished we were” conditional on this other part—I thought the main point was that people “alone in a padded cell,” as Eliezer puts it there, can “wish they were” all sorts of Unfriendly entities.