Not sure if I’ve missed something, but this seems like a risky proposal from the POV ofhow humans make deals/contracts/laws with one another.
As a shortcut to this reasoning, consider making a deal with a sociopath, someone you know to be amoral or immoral, self-interested and without any of the social emotions (guilt, shame, compassion, remorse). If they have the slightest chance, they’ll renege on the deal. So you would only make a deal with a sociopath if you were confident in enforcement mechanisms and those mechanisms have to be ones that work on sociopaths not just normal people (e.g. shame and opprobrium work on normies but not sociopaths). Even then, it’s risky and maybe best not to deal with sociopaths at all if you can avoid it, because they’ll also try to subvert the enforcement mechanisms.
Still, it can be done, because rational sociopaths tend to avoid crimes when they don’t think they’ll get away with it. But this is only because we can bank on sociopaths being: (a) hurt, physically and emotionally, by being locked up; and (b) rewarded by money or something else that satisfies their purely selfish emotions.
Unless the dealmaking AIs are liable to reward and punishment in this or a similar way, how could we ever have any confidence that they’ll honour contracts, obey laws, etc.?
This doesn’t seem exactly wrong but I think isn’t really modeling the differences between how much control we have over AIs vs humans (see Making deals with early schemers for more details)
Not sure if I’ve missed something, but this seems like a risky proposal from the POV of how humans make deals/contracts/laws with one another.
As a shortcut to this reasoning, consider making a deal with a sociopath, someone you know to be amoral or immoral, self-interested and without any of the social emotions (guilt, shame, compassion, remorse). If they have the slightest chance, they’ll renege on the deal. So you would only make a deal with a sociopath if you were confident in enforcement mechanisms and those mechanisms have to be ones that work on sociopaths not just normal people (e.g. shame and opprobrium work on normies but not sociopaths). Even then, it’s risky and maybe best not to deal with sociopaths at all if you can avoid it, because they’ll also try to subvert the enforcement mechanisms.
Still, it can be done, because rational sociopaths tend to avoid crimes when they don’t think they’ll get away with it. But this is only because we can bank on sociopaths being:
(a) hurt, physically and emotionally, by being locked up; and
(b) rewarded by money or something else that satisfies their purely selfish emotions.
Unless the dealmaking AIs are liable to reward and punishment in this or a similar way, how could we ever have any confidence that they’ll honour contracts, obey laws, etc.?
This doesn’t seem exactly wrong but I think isn’t really modeling the differences between how much control we have over AIs vs humans (see Making deals with early schemers for more details)