I changed my mind about this, I actually think “lovecraftian horror” might be somewhat better than “monkey” as a mental image, but maybe “(non-socially-constructed)-Goodness-Maximizing AGI” or “void from which things spontaneously arise” or “the voice of God” could be even better?
IMO, I don’t particularly like this framing of the player as a (non-socially-constructed)-Goodness-Maximizing AGI”, if only because I view Goodness maximizing as undefined without an arbitrary choice of values, and the implied value misalignment is not what I’d call a goodness maximizer (for the reason that it’s simply unaligned to what the character values).
I like the voice of God/Lovecraftian horror metaphors better.
Goodness maximizing as undefined without an arbitrary choice of values
By “(non-socially-constructed) Goodness” I mean the goodness of a state of affairs as it actually seems to that particular person really-deep-down. Which can have both selfish—perhaps “arbitrary” from a certain perspective—and non-selfish components.
But I still have a problem with the post’s tone because if you really internalized that “you” are the player, then your reaction to the informational content should be like “I’m a beyond-‘human’ uncontrollable force, BOOYEAH!!”, not “I’m a beyond-human uncontrollable force, ewww tentacles😣”
I changed my mind about this, I actually think “lovecraftian horror” might be somewhat better than “monkey” as a mental image, but maybe “(non-socially-constructed)-Goodness-Maximizing AGI” or “void from which things spontaneously arise” or “the voice of God” could be even better?
IMO, I don’t particularly like this framing of the player as a (non-socially-constructed)-Goodness-Maximizing AGI”, if only because I view Goodness maximizing as undefined without an arbitrary choice of values, and the implied value misalignment is not what I’d call a goodness maximizer (for the reason that it’s simply unaligned to what the character values).
I like the voice of God/Lovecraftian horror metaphors better.
By “(non-socially-constructed) Goodness” I mean the goodness of a state of affairs as it actually seems to that particular person really-deep-down. Which can have both selfish—perhaps “arbitrary” from a certain perspective—and non-selfish components.
Maybe the crux is I’m more skeptical of the “particular person really deep down” part of ethics.
But I still have a problem with the post’s tone because if you really internalized that “you” are the player, then your reaction to the informational content should be like “I’m a beyond-‘human’ uncontrollable force, BOOYEAH!!”, not “I’m a beyond-human uncontrollable force, ewww tentacles😣”