In my experience, these attitudes pertain to what should be legal or illegal—what we should impose on others—rather than what we should program into a SAI. In the case of laws, we are uncertain regarding the correct choice in certain situations, so it would be irrational to impose our choice on others (for example.) Similarly, we would not program an AI with our specific opinions on various topics, because even a single mistake could lead to disaster.
In my experience, these attitudes pertain to what should be legal or illegal—what we should impose on others—rather than what we should program into a SAI. In the case of laws, we are uncertain regarding the correct choice in certain situations, so it would be irrational to impose our choice on others (for example.) Similarly, we would not program an AI with our specific opinions on various topics, because even a single mistake could lead to disaster.