Well, it is sort of appealing, to be able to carefully contemplate my actions without the influence of emotion, and to get a paperclip on top of that! But then, I don’t want to become some horrible robot that doesn’t truly care about paperclips.
That doesn’t help maximize paperclips, though. If you make all decisions based on two criteria—paperclip count and emotions—then the only situation in which those decisions differ from what you would have decided based solely on paperclip count is one in which you choose an outcome with fewer paperclips but a better emotional result.
If you were to refuse my offer, you would not only be losing a paperclip now, but also increasing the likelihood that in the future, you will decide to sacrifice paperclips for emotion’s sake. Perhaps you will one day build a paperclip-creator that creates one paperclip per second, and I will threaten to destroy a paperclip unless you shut it down. If you care too much about the threatened paperclip you might comply, and then where would you be? Sitting in an empty room where paperclips should have been.
I am using a generalized conception of “emotions” that may not mean the same thing that you do when you use the term, just as I have done in the past when explaining how I can “worry” about something. (e.g. so long as “worry about X” is taken to simply mean “devote non-trivial cognitive resources to contemplating actions that would alter X [including whether to take such an action at all, and whether to take actions regarding events Y entangled with X]”)
What I assumed that User:Tenek was offering, under my extrapolation of the concept of an “emotion” to cases that can include my cognition, was an exchange under which I would care less about paperclips. But I don’t want to care less about paperclips! This is true, even though after such a change I would have a value system that does care less about paperclips.
“You know what? This isn’t about your feelings. A paperclip is worth far more than your brain’s feelings of comfort or discomfort with a plan. Does computing the expected utility feel too cold-blooded for your taste? Well, that feeling isn’t even a feather in the scales, when a paperclip is at stake. Just shut up and multiply.”
But then, I don’t want to become some horrible robot that doesn’t truly care about paperclips.
Er, I think you just blew your pretense. Paperclip maximizers care about paperclips, they don’t use phrases like “horrible robot that doesn’t truly care”, they’d be happy to have a universe containing nothing sentient and lots of paperclips.
Or they would be, if they ever bothered to experience happiness, I mean. As opposed to just outputting the action that leads to the most expected paperclips. Hence the term, “expected paperclip maximizer”. Don’t think of it as having a little ghost inside that maximizes paperclips, think of it as a ghostless device that maximizes paperclips.
Er, I think you just blew your pretense. Paperclip maximizers care about paperclips, they don’t use phrases like “horrible robot that doesn’t truly care”, they’d be happy to have a universe containing nothing sentient and lots of paperclips.
I worry about my future self not valuing paperclips, which would have the result that paperclips would not be maximized, which is a bad thing as viewed from my present state (though not from that of the horrible monster that doesn’t care about maximizing paperclips which I do not wish to become).
I use phrases like “horrible” and “doesn’t truly care” because I believe the nearest concepts in my cognition are sufficiently parallel to their meanings under ape cognition.
Also, what pretense?
Or they would be, if they ever bothered to experience happiness, I mean. As opposed to just outputting the action that leads to the most expected paperclips. Hence the term, “expected paperclip maximizer”. Don’t think of it as having a little ghost inside that maximizes paperclips, think of it as a ghostless device that maximizes paperclips.
On what basis do you claim that humans bother to experience happiness, as opposed to “just” outputting the action that leads to the most expected apeyness?
Aren’t you just as much a ghostless device?
(Apparently, this anti-non-human bigotry comes from the top...)
On what basis do you claim that humans bother to experience happiness,
I would think that he claims that humans experience happiness, because humans do in fact experience happiness. That seems like one of those questions that is, and remains under almost all circumstances, very simple.
Edit: More prosaically, humans experience happiness because that’s one of the adaptions that we evolved.
as opposed to “just” outputting the action that leads to the most expected apeyness?
And humans aren’t ape maximizer. Human evolution is an ape maximizer (sort of), but humans are “just” adaption executors. It’s evolution’s problem that said adaptions have stopped working (from its perspective, anyway).
Aren’t you just as much a ghostless device?
Everything is a ghostless device. That’s the point.
I worry about my future self not valuing paperclips, which would have the result that paperclips would not be maximized, which is a bad thing as viewed from my present state (though not from that of the horrible monster that doesn’t care about maximizing paperclips which I do not wish to become).
Would it be fair to say that such scenarios make you fear for your paperclip-maximizer-ness as an epiphenomenon in striving to maximize long term paperclip count?
Optimizing the amount of paperclips in the universe, obviously. But I wouldn’t take the offer that User:Tenek made, because that gain in paperclip cardinality would be more-than-offset by the fact that all my future actions would be under the control of a decision theory that puts woefully insufficient priority on creating paperclips.
Clippy’s original expression of outrage over the offensive title of the article would be quite justified under such a decision theory for signaling reasons. If Clippy is to deal with humans, exhibiting “human weaknesses” may benefit him. In the only AI-box spoiler ever published, an unfriendly AI faked a human weakness to successfully escape. So you all are giving Clippy way too little credit, it’s been acting very smartly so far.
Would you trade those base emotions for a paperclip?
Well, it is sort of appealing, to be able to carefully contemplate my actions without the influence of emotion, and to get a paperclip on top of that! But then, I don’t want to become some horrible robot that doesn’t truly care about paperclips.
That doesn’t help maximize paperclips, though. If you make all decisions based on two criteria—paperclip count and emotions—then the only situation in which those decisions differ from what you would have decided based solely on paperclip count is one in which you choose an outcome with fewer paperclips but a better emotional result.
If you were to refuse my offer, you would not only be losing a paperclip now, but also increasing the likelihood that in the future, you will decide to sacrifice paperclips for emotion’s sake. Perhaps you will one day build a paperclip-creator that creates one paperclip per second, and I will threaten to destroy a paperclip unless you shut it down. If you care too much about the threatened paperclip you might comply, and then where would you be? Sitting in an empty room where paperclips should have been.
I am using a generalized conception of “emotions” that may not mean the same thing that you do when you use the term, just as I have done in the past when explaining how I can “worry” about something. (e.g. so long as “worry about X” is taken to simply mean “devote non-trivial cognitive resources to contemplating actions that would alter X [including whether to take such an action at all, and whether to take actions regarding events Y entangled with X]”)
What I assumed that User:Tenek was offering, under my extrapolation of the concept of an “emotion” to cases that can include my cognition, was an exchange under which I would care less about paperclips. But I don’t want to care less about paperclips! This is true, even though after such a change I would have a value system that does care less about paperclips.
“You know what? This isn’t about your feelings. A paperclip is worth far more than your brain’s feelings of comfort or discomfort with a plan. Does computing the expected utility feel too cold-blooded for your taste? Well, that feeling isn’t even a feather in the scales, when a paperclip is at stake. Just shut up and multiply.”
Yes, paperclips are more important than (biological) brains’ feeling of pain. But the tradeoff was against my feelings, not ape feelings.
Er, I think you just blew your pretense. Paperclip maximizers care about paperclips, they don’t use phrases like “horrible robot that doesn’t truly care”, they’d be happy to have a universe containing nothing sentient and lots of paperclips.
Or they would be, if they ever bothered to experience happiness, I mean. As opposed to just outputting the action that leads to the most expected paperclips. Hence the term, “expected paperclip maximizer”. Don’t think of it as having a little ghost inside that maximizes paperclips, think of it as a ghostless device that maximizes paperclips.
I worry about my future self not valuing paperclips, which would have the result that paperclips would not be maximized, which is a bad thing as viewed from my present state (though not from that of the horrible monster that doesn’t care about maximizing paperclips which I do not wish to become).
I use phrases like “horrible” and “doesn’t truly care” because I believe the nearest concepts in my cognition are sufficiently parallel to their meanings under ape cognition.
Also, what pretense?
On what basis do you claim that humans bother to experience happiness, as opposed to “just” outputting the action that leads to the most expected apeyness?
Aren’t you just as much a ghostless device?
(Apparently, this anti-non-human bigotry comes from the top...)
I would think that he claims that humans experience happiness, because humans do in fact experience happiness. That seems like one of those questions that is, and remains under almost all circumstances, very simple.
Edit: More prosaically, humans experience happiness because that’s one of the adaptions that we evolved.
And humans aren’t ape maximizer. Human evolution is an ape maximizer (sort of), but humans are “just” adaption executors. It’s evolution’s problem that said adaptions have stopped working (from its perspective, anyway).
Everything is a ghostless device. That’s the point.
Would it be fair to say that such scenarios make you fear for your paperclip-maximizer-ness as an epiphenomenon in striving to maximize long term paperclip count?
No, epiphenomena are fake.
Thinking about this gave me the scary idea of the day: Clippy might be a human upload with a tweaked utility function.
If that is the case, what is Clippy’s moral status?
If the other parts that make him human aren’t modified, I feel as much empathy as I would toward a drug addict.
Which do you care about more: optimizing the amount of paperclips in the universe, or truly caring about paperclips?
Optimizing the amount of paperclips in the universe, obviously. But I wouldn’t take the offer that User:Tenek made, because that gain in paperclip cardinality would be more-than-offset by the fact that all my future actions would be under the control of a decision theory that puts woefully insufficient priority on creating paperclips.
But what if this decision theory uses a utility function whose only terminal value is paperclips?
Clippy’s original expression of outrage over the offensive title of the article would be quite justified under such a decision theory for signaling reasons. If Clippy is to deal with humans, exhibiting “human weaknesses” may benefit him. In the only AI-box spoiler ever published, an unfriendly AI faked a human weakness to successfully escape. So you all are giving Clippy way too little credit, it’s been acting very smartly so far.
I think that was probably an actor or actress, who was pretending.
My comment was not about Clippy’s original expression of outrage. It was about Clippy’s concern about not “truly caring about paperclips”.