To me, the most likely candidate for the role of an allegory of AI in MoR currently seems to be the Source of [Atlantean] Magic, assuming that Harry’s speculation wasn’t completely off the mark.
(My Wild Mass Guess on the matter would be that Harry will eventually discover that the Source was the Atlanteans’ equivalent of a moderately unFriendly AI, which didn’t destroy the world but did wipe away Atlantis. Eliezer will then be sorely tempted to go on an all-out Author Filibuster (TVTropes) on the subject, but will manage to restrain himself to a couple of paragraphs in the Author’s Notes.)
I’ll be disappointed if Harry doesn’t turn out to have been completely off the mark there.
I mean, the process he went through was roughly “Hey, look at this thing I don’t understand. It doesn’t behave at all the way I would expect it to. Um… maybe X is an explanation!” with no particular justification for privileging X over the uncountable number of hypotheses he could have come up with instead.
Worse, the hypothesis he came up with was pretty much unfalsifiable, and doesn’t make any particular predictions. It doesn’t “pay rent,” to quote an early OB post.
The moralist in me does not want Harry “rewarded” with a correct answer for reasoning that way.
That having been said, I grant that if I knew the SO[A]M existed, I would conclude that Harry was somehow being influenced by its existence such that the theory of SO[A]M was more available, which doesn’t require any additional assumptions given that it is necessarily responsive to wizards’ thoughts to begin with. (That is, I don’t want to repeat the reasoning error I made elsewhere regarding Quirrell being polyjuiced.)
To me, the most likely candidate for the role of an allegory of AI in MoR currently seems to be the Source of [Atlantean] Magic, assuming that Harry’s speculation wasn’t completely off the mark.
(My Wild Mass Guess on the matter would be that Harry will eventually discover that the Source was the Atlanteans’ equivalent of a moderately unFriendly AI, which didn’t destroy the world but did wipe away Atlantis. Eliezer will then be sorely tempted to go on an all-out Author Filibuster (TVTropes) on the subject, but will manage to restrain himself to a couple of paragraphs in the Author’s Notes.)
I’ll be disappointed if Harry doesn’t turn out to have been completely off the mark there.
I mean, the process he went through was roughly “Hey, look at this thing I don’t understand. It doesn’t behave at all the way I would expect it to. Um… maybe X is an explanation!” with no particular justification for privileging X over the uncountable number of hypotheses he could have come up with instead.
Worse, the hypothesis he came up with was pretty much unfalsifiable, and doesn’t make any particular predictions. It doesn’t “pay rent,” to quote an early OB post.
The moralist in me does not want Harry “rewarded” with a correct answer for reasoning that way.
That having been said, I grant that if I knew the SO[A]M existed, I would conclude that Harry was somehow being influenced by its existence such that the theory of SO[A]M was more available, which doesn’t require any additional assumptions given that it is necessarily responsive to wizards’ thoughts to begin with. (That is, I don’t want to repeat the reasoning error I made elsewhere regarding Quirrell being polyjuiced.)
But right now I don’t know that.