Because we evolved to care about things like fairness in an environment where everyone knew each other, and if you cheated someone, everyone else in the village knew it. And, modern humans still employ their evolved instincts. Therefore, agents who lack moral concerns can exploit the fact that humans are using intuitions that were optimized to work in a different situation. For instance, they can avoid doing things so heinous that society as a whole tries to hunt them down, and once they have exploited someone, they can just move.
If there’s a long-term effective strategy for cheating—one that doesn’t involve the cheater being detected and punished—why isn’t everone using it?
Because we evolved to care about things like fairness in an environment where everyone knew each other, and if you cheated someone, everyone else in the village knew it. And, modern humans still employ their evolved instincts. Therefore, agents who lack moral concerns can exploit the fact that humans are using intuitions that were optimized to work in a different situation. For instance, they can avoid doing things so heinous that society as a whole tries to hunt them down, and once they have exploited someone, they can just move.