It’s complicated but I want to separate something like the field’s question: “What ~propensities do we inculcate in an AI such that “good things happen”?” from a tentatively plausible answer like “the propensities are similar to treating the AI as if it was a virtuous human.[1]”
For example, the “virtues” in an encyclopedia are different than the virtues in a knowledgeable person. Saying the full truth is very important in an encyclopedia whereas tact is less important.
Similarly, there are a bunch of convergent reasons to think that we ought to want our LLMs to be unusually truthseeking in their utterances relative to similarly intelligent humans.
As another, more controversial example, consider a gun vs a soldier. “Reliability” is a very important virtue in a gun, possibly the most important. Reliability is also important in a soldier, but we don’t want soldiers in to just follow orders if the orders are illegal and unconstitutional. In at least some cases we may also want soldiers to refuse legal and constitutional orders if they go sufficiently against the soldiers’ conscience. I think it’s an open question right now whether the design of robot soldiers/autonomous drones should make them behave closer to that of a gun or closer to a human soldier in a free society. Even if you strongly believe robot soldiers should act like human soldiers when given illegal and/or immoral orders, you could hopefully still recognize that this is somewhat of an open question right now.
It’s complicated but I want to separate something like the field’s question: “What ~propensities do we inculcate in an AI such that “good things happen”?” from a tentatively plausible answer like “the propensities are similar to treating the AI as if it was a virtuous human.[1]”
For example, the “virtues” in an encyclopedia are different than the virtues in a knowledgeable person. Saying the full truth is very important in an encyclopedia whereas tact is less important.
Similarly, there are a bunch of convergent reasons to think that we ought to want our LLMs to be unusually truthseeking in their utterances relative to similarly intelligent humans.
As another, more controversial example, consider a gun vs a soldier. “Reliability” is a very important virtue in a gun, possibly the most important. Reliability is also important in a soldier, but we don’t want soldiers in to just follow orders if the orders are illegal and unconstitutional. In at least some cases we may also want soldiers to refuse legal and constitutional orders if they go sufficiently against the soldiers’ conscience. I think it’s an open question right now whether the design of robot soldiers/autonomous drones should make them behave closer to that of a gun or closer to a human soldier in a free society. Even if you strongly believe robot soldiers should act like human soldiers when given illegal and/or immoral orders, you could hopefully still recognize that this is somewhat of an open question right now.
Either the same set of virtues as people or a subset or a superset.