A rogue paperclipper in a mostly Friendly world can probably only be stopped by racial prejudice—to a rational creature, it’s always easier to feed him your neighbor than it is to fight him.
A couple of problems with this statement, as I see it:
The word “only”. Forget five minutes— think for five seconds about Third Alternatives. At the very least, wouldn’t an emotion for human-favoritism serve the goal better than an emotion for race-favoritism? Then everyone could cooperate more fully, not just each race by itself.
You could be using “racial prejudice” to mean “species prejudice” or something even wider, but that’s not what the question’s about. Your argument gives no reason for maintaining the current brain architecture, which creates these divisions of allegiance within the normal human race.
Rational agents are doomed to fail because they won’t cooperate enough? I stand with Eliezer: rational agents should WIN. If the inevitable result of noncooperation is eventual destruction, genuinely rational agents WILL find ways to cooperate; the Prisoner’s Dilemma doesn’t operate within every conceivable cooperative enterprise.
A rogue paperclipper in a mostly Friendly world can probably only be stopped by racial prejudice—to a rational creature, it’s always easier to feed him your neighbor than it is to fight him.
A couple of problems with this statement, as I see it:
The word “only”. Forget five minutes— think for five seconds about Third Alternatives. At the very least, wouldn’t an emotion for human-favoritism serve the goal better than an emotion for race-favoritism? Then everyone could cooperate more fully, not just each race by itself.
You could be using “racial prejudice” to mean “species prejudice” or something even wider, but that’s not what the question’s about. Your argument gives no reason for maintaining the current brain architecture, which creates these divisions of allegiance within the normal human race.
Rational agents are doomed to fail because they won’t cooperate enough? I stand with Eliezer: rational agents should WIN. If the inevitable result of noncooperation is eventual destruction, genuinely rational agents WILL find ways to cooperate; the Prisoner’s Dilemma doesn’t operate within every conceivable cooperative enterprise.