What seems off to me is the idea that the ‘player’ is some sort of super-powerful incomprehensible lovecraftian optimizer. I think it’s more apt to think of it as like a monkey, but a monkey which happens to share your body and have write access to the deepest patterns of your thought and feeling(see Steven Byrnes’ posts for the best existing articulation of this view). It’s just a monkey, its desires aren’t totally alien and I think it’s quite possible for one’s conscious mind to develop a reasonably good idea of what it wants. That the OP prefers to push the ‘alien/lovecraftian’ framing is interesting and perhaps indicates that they find what their monkey (and/or other peoples’ monkeys) wants repulsive in some way.
I changed my mind about this, I actually think “lovecraftian horror” might be somewhat better than “monkey” as a mental image, but maybe “(non-socially-constructed)-Goodness-Maximizing AGI” or “void from which things spontaneously arise” or “the voice of God” could be even better?
IMO, I don’t particularly like this framing of the player as a (non-socially-constructed)-Goodness-Maximizing AGI”, if only because I view Goodness maximizing as undefined without an arbitrary choice of values, and the implied value misalignment is not what I’d call a goodness maximizer (for the reason that it’s simply unaligned to what the character values).
I like the voice of God/Lovecraftian horror metaphors better.
Goodness maximizing as undefined without an arbitrary choice of values
By “(non-socially-constructed) Goodness” I mean the goodness of a state of affairs as it actually seems to that particular person really-deep-down. Which can have both selfish—perhaps “arbitrary” from a certain perspective—and non-selfish components.
But I still have a problem with the post’s tone because if you really internalized that “you” are the player, then your reaction to the informational content should be like “I’m a beyond-‘human’ uncontrollable force, BOOYEAH!!”, not “I’m a beyond-human uncontrollable force, ewww tentacles😣”
What seems off to me is the idea that the ‘player’ is some sort of super-powerful incomprehensible lovecraftian optimizer. I think it’s more apt to think of it as like a monkey, but a monkey which happens to share your body and have write access to the deepest patterns of your thought and feeling(see Steven Byrnes’ posts for the best existing articulation of this view). It’s just a monkey, its desires aren’t totally alien and I think it’s quite possible for one’s conscious mind to develop a reasonably good idea of what it wants. That the OP prefers to push the ‘alien/lovecraftian’ framing is interesting and perhaps indicates that they find what their monkey (and/or other peoples’ monkeys) wants repulsive in some way.
I changed my mind about this, I actually think “lovecraftian horror” might be somewhat better than “monkey” as a mental image, but maybe “(non-socially-constructed)-Goodness-Maximizing AGI” or “void from which things spontaneously arise” or “the voice of God” could be even better?
IMO, I don’t particularly like this framing of the player as a (non-socially-constructed)-Goodness-Maximizing AGI”, if only because I view Goodness maximizing as undefined without an arbitrary choice of values, and the implied value misalignment is not what I’d call a goodness maximizer (for the reason that it’s simply unaligned to what the character values).
I like the voice of God/Lovecraftian horror metaphors better.
By “(non-socially-constructed) Goodness” I mean the goodness of a state of affairs as it actually seems to that particular person really-deep-down. Which can have both selfish—perhaps “arbitrary” from a certain perspective—and non-selfish components.
Maybe the crux is I’m more skeptical of the “particular person really deep down” part of ethics.
But I still have a problem with the post’s tone because if you really internalized that “you” are the player, then your reaction to the informational content should be like “I’m a beyond-‘human’ uncontrollable force, BOOYEAH!!”, not “I’m a beyond-human uncontrollable force, ewww tentacles😣”