Driving is an area of life where millions of “ordinary” humans (non-specialists) make life-critical and therefore morally-significant judgments every day. When we drive, we are taking our lives and those of others in our hands. Many of us would wish to be better drivers than we are: not only more skilled, but better in ways that could be described as “virtue”: less prone to road rage, negligence, driving while impaired, and other faults. Robots don’t get angry, they don’t get distracted, and they don’t get drunk or tired. Since bad driving kills people, we can reasonably say that robot driving is (or can become) morally superior to human driving — in a plain consequentialist sense.
This seems like a natural analogy for CEV in superhuman systems. We do not want a robot driver to drive just like a human. We want a robot driver to drive as a human would drive if that human were faster-thinking, calmer, clearer-minded, more focused; had sharper eyes, better knowledge of the roads and hazards, better ability to cooperate with other drivers. We want a robot to optimize a utility function derived closely from ours — crudely, “get me to my destination and don’t kill anyone or cause any damage on the way” — and to do so better than we can.
It is only within a limited domain that the robot car is a superhuman decision-maker; but that limited domain is one that pretty much every adult is acquainted with. When robot cars become commonplace, every human driver will — every day — be interacting with limited-domain superhuman, non-conscious, non-recursively-optimizing artificial decision agents implementing a form of extrapolated volition and making morally significant, life-critical choices.
People might notice that the robots are nicer than humans to share the road with. They don’t cut you off. They let you merge. They stop for Grandma entering the crosswalk. They don’t run bikers off the road by not seeing them. They don’t drive really slow in the ultra-fast lane while people behind them are going insane — they’re not assholes.
We should expect this will dramatically increase visibility of AI ethics as a field.
It’s a very good example. It also illustrates how hard is to specify a useful utility function for an AGI: “get me to my destination and don’t kill anyone or cause any damage on the way” can lead to a number of non-obvious unintended consequences, compared to the CEV version “drive as a human would drive if that human were faster-thinking, calmer, clearer-minded, more focused; had sharper eyes, better knowledge of the roads and hazards, better ability to cooperate with other drivers”.
None of this is news to me, but it’s certainly nice to see the link being made between AI Driving and ethics in a positive light. Most people only jump to the part about “If an AI car kills someone, whose life do we ruin as vengeful punishment?”
Robot cars may already be better drivers than humans. And if not, they’re clearly on their way to become so.
Driving is an area of life where millions of “ordinary” humans (non-specialists) make life-critical and therefore morally-significant judgments every day. When we drive, we are taking our lives and those of others in our hands. Many of us would wish to be better drivers than we are: not only more skilled, but better in ways that could be described as “virtue”: less prone to road rage, negligence, driving while impaired, and other faults. Robots don’t get angry, they don’t get distracted, and they don’t get drunk or tired. Since bad driving kills people, we can reasonably say that robot driving is (or can become) morally superior to human driving — in a plain consequentialist sense.
This seems like a natural analogy for CEV in superhuman systems. We do not want a robot driver to drive just like a human. We want a robot driver to drive as a human would drive if that human were faster-thinking, calmer, clearer-minded, more focused; had sharper eyes, better knowledge of the roads and hazards, better ability to cooperate with other drivers. We want a robot to optimize a utility function derived closely from ours — crudely, “get me to my destination and don’t kill anyone or cause any damage on the way” — and to do so better than we can.
It is only within a limited domain that the robot car is a superhuman decision-maker; but that limited domain is one that pretty much every adult is acquainted with. When robot cars become commonplace, every human driver will — every day — be interacting with limited-domain superhuman, non-conscious, non-recursively-optimizing artificial decision agents implementing a form of extrapolated volition and making morally significant, life-critical choices.
People might notice that the robots are nicer than humans to share the road with. They don’t cut you off. They let you merge. They stop for Grandma entering the crosswalk. They don’t run bikers off the road by not seeing them. They don’t drive really slow in the ultra-fast lane while people behind them are going insane — they’re not assholes.
We should expect this will dramatically increase visibility of AI ethics as a field.
It’s a very good example. It also illustrates how hard is to specify a useful utility function for an AGI: “get me to my destination and don’t kill anyone or cause any damage on the way” can lead to a number of non-obvious unintended consequences, compared to the CEV version “drive as a human would drive if that human were faster-thinking, calmer, clearer-minded, more focused; had sharper eyes, better knowledge of the roads and hazards, better ability to cooperate with other drivers”.
None of this is news to me, but it’s certainly nice to see the link being made between AI Driving and ethics in a positive light. Most people only jump to the part about “If an AI car kills someone, whose life do we ruin as vengeful punishment?”
Thanks.