The reason that property rights and systems for enforcing them exist is that instrumental drives to steal, murder, etc are extremely negative sum, and so having a system that prevents that (money, laws, law enforcement) is really a super-convergent feature of reality.
There is the possibility that such an AI-world would adopt a system of property rights that excludes humans and treats us the way we treat farm animals; this is a possible equilibrium but I think it is very hard to incrementally get from where we are now to that; it seems more likely that the AI-world’s system of property rights would try to be maximally inclusive to maximize adoption—like Bitcoin. But once we are discussing risk from systems of future property rights (“P-risks”), we are already sufficiently far from the risk scenario described in the OP that it’s just worth clearly flagging it as nonsense before we move on.
Isn’t it a common occurrence that groups that can coordinate, collude against weaker minorities to subvert their property rights and expropriate their stuff and/or labor?
White Europeans enslaving American Indians, and then later Africans seems like maybe the most central example, but there are also pogroms against jews etc., and raids by warrior cultures against agrarian cultures. And, as you point out, how humans collude to breed and control farm anaimls.
Property rights are positive sum, but gerrymandering the property schema to privilege one’s own group is convergent, so long as 1) your group has the force to do so and 2) there are demarcators that allow your group to successfully coordinate against others without turning on itself.
eg “Theft and murder are normal” is a bad equilibrium for almost everyone, since everyone has to pay higher protection costs, that exceed the average benefit of their own theft and murder. “Theft and murder are illegal, but if whites are allowed to expropriate from blacks, including enslaving them, enforced by violence and the threat of violence, because that’s the natural order” is sadly quite stable, and is potentially a net benefit to the whites (at least by a straightforward selfish accounting). So American racially-demarcated slavery persists from 1700s to the mid 1800s, even though American society otherwise has strong rule of law and property norms.
It sure seems to me that there is a clear demarcation between AIs and humans, such that the AIs would be able to successfully collude against humans while coordinating property rights and rule of law amongst themselves.
Isn’t it a common occurrence that groups that can coordinate, collude against weaker minorities to subvert their property rights and expropriate their stuff and/or labor?
White Europeans enslaving American Indians, and then later Africans seems like maybe the most central example, but there are also pogroms against jews etc., and raids by warrior cultures against agrarian cultures. And, as you point out, how humans collude to breed and control farm anaimls.
Property rights are positive sum, but gerrymandering the property schema to privilege one’s own group is convergent, so long as 1) your group has the force to do so and 2) there are demarcators that allow your group to successfully coordinate against others without turning on itself.
eg “Theft and murder are normal” is a bad equilibrium for almost everyone, since everyone has to pay higher protection costs, that exceed the average benefit of their own theft and murder. “Theft and murder are illegal, but if whites are allowed to expropriate from blacks, including enslaving them, enforced by violence and the threat of violence, because that’s the natural order” is sadly quite stable, and is potentially a net benefit to the whites (at least by a straightforward selfish accounting). So American racially-demarcated slavery persists from 1700s to the mid 1800s, even though American society otherwise has strong rule of law and property norms.
It sure seems to me that there is a clear demarcation between AIs and humans, such that the AIs would be able to successfully collude against humans while coordinating property rights and rule of law amongst themselves.
I think this just misunderstands how coordination works.
The game theory of who is allowed to coordinate with who against whom is not simple.
White Germans fought against white Englishmen who are barely different, but each tried to ally with distantly related foreigners.
Ultimately what we are starting to see is that AI risk isn’t about math or chips or interpretability, it’s actually just politics.