They think of these as defective product situations, and they see discussion of “rogue” AI as an attempt by the companies to divert blame (and perhaps legal liability) away from themselves as if Ford made a car with faulty brakes and then tried to blame this on “rogue cars.”
B:
Students found the paperclip maximizer an interesting parable in this regard, but the general consensus was that a highly capable system is capable of understanding your intent, and so it would — like anyone with common sense — understand that extracting the iron from your hemoglobin to make more paperclips is not what you asked. Thus, an AI will not go wrong in this way.
Why do they think a paperclip maximizer cannot occur in the form of a defective product?
Point A was a basically moral point not a factual one (i.e., we should not let OpenAI get away with claiming that ChatGPT’s actions are not its responsibility). So, it’s not necessarily in tension with B. It’s just talking about something else.
Could a paperclip maximizer occur as a defective product? The basic attitude here implies “no”—the thinking is that something smart enough to design a system to harvest your hemoglobin can’t be dumb enough to think that’s what you were asking when you said “maximize paperclips.”
A question I didn’t raise at any point was “if you set out to build a malicious AI designed to go out and harvest hemoglobin for paperclips, could you do it?” I think at least some of them would say “no” without venturing a guess on the proportion. A lot of my students (and a lot of people generally) apply a kind of intuitive moral realism under which there are objective standards or right and wrong that would be known, and followed, by a sufficiently intelligent AI.
Thank you for the detailed examples.
I find this dichotomy curious:
A:
B:
Why do they think a paperclip maximizer cannot occur in the form of a defective product?
Point A was a basically moral point not a factual one (i.e., we should not let OpenAI get away with claiming that ChatGPT’s actions are not its responsibility). So, it’s not necessarily in tension with B. It’s just talking about something else.
Could a paperclip maximizer occur as a defective product? The basic attitude here implies “no”—the thinking is that something smart enough to design a system to harvest your hemoglobin can’t be dumb enough to think that’s what you were asking when you said “maximize paperclips.”
A question I didn’t raise at any point was “if you set out to build a malicious AI designed to go out and harvest hemoglobin for paperclips, could you do it?” I think at least some of them would say “no” without venturing a guess on the proportion. A lot of my students (and a lot of people generally) apply a kind of intuitive moral realism under which there are objective standards or right and wrong that would be known, and followed, by a sufficiently intelligent AI.