May I suggest that Plato’s words carry some different and non-obvious sensibility, that has little to do with the outside vs inside, if we take the original text and the circumstances into account? For, in that age, people had fewer reasons to believe the physicality of the individual. They saw dead people remain dead, but that’s pretty much all of it. And they had more motives to believe in the soul, because there’s no scientific transhumanism, and religion was their only hope of personal immortality. So the introspecting self may feel that just as it has slept and awaken, and remained itself, so it is possible to survive an abscence of consciousness, and death must be temporary. Not because they superficially looked and sounded alike, but because of the common factor of lack of consciousness, which seemed much less distinguishable than they do now in light of neuroscience. It’s not outside vs inside, it might as well have been Phaecrinon thinking “the two pairs are structually different” and Plato thinking “they are equivalent and symmetrical”.
Plato has dismissed his share of strawmen opponents, and I have no problem with adapting his words, but I feel confused by this post’s focus on the principles of thinking when this more obvious reaction to the analogy comes to mind. How about choosing a purer example next time?
IMO a fun project (for those like me who like this but are clearly not smart enough to be part of a Singularity developing team): create an object-based environment with maybe rule-based reproductive agents, with customizable explicit world rules (as in a computer game, not as in physics) and let them evolve. Maybe users across the world can add new magical artifacts and watch the creatures fail hilariously...
On a more related note, the post sounds ominous for any hope of a general AI. There may be no clear distinction between protein computer and just protein, between learning and blindly acting. If we and our desires are in such a position, aren’t any AI we can make indirectly also blind? Or as I understand, Eliezer seems to think that we can (or had better) bootstrap, both for intelligence/computation and morality. For him, this bootstrapping, this understanding/theory of the generality of our own intelligence (as step one of the bootstrapping) seems to be the Central Truth of Life (tm). Maybe he’s right, but for me with less insight into intelligence, that’s not self-evident. And he didn’t explain this crucial point anywhere clearly, only advocated it. Come on, it’s not as if enough people can be a seed AI programmer even armed with that Central Truth. (But who knows.)