A core part of Paul’s arguments is that having 1/million of your values towards humans only applies a minute amount of selection pressure against you. It could be that coordinating causes less kindness because without coordination it’s more likely some fraction of agents have small vestigial values that never got selected against or intentionally removed
I think this is not particularly relevant if entities are deliberately adopting competitive personas in order to win contests. It might take a lot of mutation to drop the 1/million penalty, but it probably doesn’t take a lot of cognition for an agent that believes a meme like “winning isn’t everything, it’s the only thing.”
A core part of Paul’s arguments is that having 1/million of your values towards humans only applies a minute amount of selection pressure against you. It could be that coordinating causes less kindness because without coordination it’s more likely some fraction of agents have small vestigial values that never got selected against or intentionally removed
I think this is not particularly relevant if entities are deliberately adopting competitive personas in order to win contests. It might take a lot of mutation to drop the 1/million penalty, but it probably doesn’t take a lot of cognition for an agent that believes a meme like “winning isn’t everything, it’s the only thing.”