I like that analogy. I imagine that for an AIXI-style artificial intelligence, the whole futures of the universe are just like the pieces of the sand on the beach. It chooses a piece according to some criteria, for example the brightest color, but every other aspect is so completely irrelevant than most humans would be unable to imagine that kind of indifference. Our human brains keep screaming at us: “But surely even a mere machine would not dare to choose a piece of sand that is a part of such-and-such configuration. Why would it do such a horrible thing?” But the machine is not even aware that those configurations exist, and certainly does not care to know.
Well, what I am pressing is the issue: You can know but not care.
I thin that is what many fail to grasp about psychopaths who do bad things. They know that they are committing crimes, they just don’t care. (There are some good psychopaths who has disregarded their initial stupid philosophical conclusions about morality and actually help people, but those are rarely heard.)
A superintelligent paperclipper can know everything about human ethics, but only use that to manipulate humans into making more paperclips.
I like that analogy. I imagine that for an AIXI-style artificial intelligence, the whole futures of the universe are just like the pieces of the sand on the beach. It chooses a piece according to some criteria, for example the brightest color, but every other aspect is so completely irrelevant than most humans would be unable to imagine that kind of indifference. Our human brains keep screaming at us: “But surely even a mere machine would not dare to choose a piece of sand that is a part of such-and-such configuration. Why would it do such a horrible thing?” But the machine is not even aware that those configurations exist, and certainly does not care to know.
Well, what I am pressing is the issue: You can know but not care.
I thin that is what many fail to grasp about psychopaths who do bad things. They know that they are committing crimes, they just don’t care. (There are some good psychopaths who has disregarded their initial stupid philosophical conclusions about morality and actually help people, but those are rarely heard.)
A superintelligent paperclipper can know everything about human ethics, but only use that to manipulate humans into making more paperclips.