I worry very little about AI harming us by accident. I think that’s a much lower class of unsafe than an intelligent being adopting the goal of harming you. And I think we’re are busily working toward building AGI that might do just that. So I don’t worry about other classes of unsafe things much at all.
Tons of things like cars and the Facebook engagement algorithm are unsafe by changing human behavior in ways that directly and indirectly cause physical and psychological harm.
Optimized to cause harm is another level of unsafe. An engineered or natural virus are optimized to infect you, but the engineered one is optimized for harm and so probably much more dangerous.
The other crucial aspect is goal-directedness. Something that itself wants to harm you is much more dangerous. It will maneuver around unexpected obstacles to harm you, as best it can according to its intelligence or competence in that area.
That’s why the most unsafe class is highly intelligent things that want to harm you because they have goals that conflict with yours. If instrumental convergence has made someone.or something adopt an instrumental goals of getting you out of its way, that’s far more dangerous than something that was merely optimized by an outside force to harm you (unless that outside force has the intelligence to outthink you I suppose).
I don’t worry about accidental or emergent dangers because we’re building far more dangerous things on purpose, that may harm us on purpose.
I worry very little about AI harming us by accident. I think that’s a much lower class of unsafe than an intelligent being adopting the goal of harming you. And I think we’re are busily working toward building AGI that might do just that. So I don’t worry about other classes of unsafe things much at all.
Tons of things like cars and the Facebook engagement algorithm are unsafe by changing human behavior in ways that directly and indirectly cause physical and psychological harm.
Optimized to cause harm is another level of unsafe. An engineered or natural virus are optimized to infect you, but the engineered one is optimized for harm and so probably much more dangerous.
The other crucial aspect is goal-directedness. Something that itself wants to harm you is much more dangerous. It will maneuver around unexpected obstacles to harm you, as best it can according to its intelligence or competence in that area.
That’s why the most unsafe class is highly intelligent things that want to harm you because they have goals that conflict with yours. If instrumental convergence has made someone.or something adopt an instrumental goals of getting you out of its way, that’s far more dangerous than something that was merely optimized by an outside force to harm you (unless that outside force has the intelligence to outthink you I suppose).
I don’t worry about accidental or emergent dangers because we’re building far more dangerous things on purpose, that may harm us on purpose.