@ Prase: “If people can create an AI which says “murder is wrong”, they can surely create also an AI which tells the contrary, and the latter would be no less intelligent than the former.”
I disagree with this message. Of course, what you have said isn’t sufficiently precise to be either correct or incorrect—words like “murder”, “intelligent” are very much in need of defining precisely. But I think that we should stop thinking of ethics and intelligence as being somehow orthogonal. I think that ethics, in its various forms, constitutes a way of organizing a system of agents so that the overall level of performance and intelligence of the system is optimal. Laws exist so that society functions correctly.
An AI that randomly murdered people would not benefit from having those people around, so it would not be as intelligent/successful as a similar system which didn’t murder.
An AI that got rid of humanity “just because we are atoms that can be usefully re-arranged into something else” would probably be violating one of the four basic AI drives: namely the creativity drive.
@ Prase: “If people can create an AI which says “murder is wrong”, they can surely create also an AI which tells the contrary, and the latter would be no less intelligent than the former.”
I disagree with this message. Of course, what you have said isn’t sufficiently precise to be either correct or incorrect—words like “murder”, “intelligent” are very much in need of defining precisely. But I think that we should stop thinking of ethics and intelligence as being somehow orthogonal. I think that ethics, in its various forms, constitutes a way of organizing a system of agents so that the overall level of performance and intelligence of the system is optimal. Laws exist so that society functions correctly.
An AI that randomly murdered people would not benefit from having those people around, so it would not be as intelligent/successful as a similar system which didn’t murder.
An AI that got rid of humanity “just because we are atoms that can be usefully re-arranged into something else” would probably be violating one of the four basic AI drives: namely the creativity drive.