The main exception here is self-fulfilling prophecy, but that’s not obviously centrally involved in whatever “I decided to trust her” means.
I think it might be.
I think the core of the contractual interpretation is right, but I’d phrase it differently. I think “deciding to trust” someone implies a deal where you ally yourself with them in a way that leaves you vulnerable to hostile actions from their side – you’re choosing “cooperate” in the Prisoner’s Dilemma in expectation of greater mutual gain. In exchange, they (1) commit not to take said hostile actions, (2) potentially “reciprocate trust” by showing symmetrical vulnerability (either to ensure MAD, or because pooling resources improves your ability to work together), and (3) bias their policy towards satisfying your preferences, which includes being more responsible when interacting with you and giving you some veto on their actions (and if they fail that, they’ve “betrayed” your trust).
And there’s some degree of acausal culture-based negotiation going on. E. g., it’s not that people explicitly track all of the above consciously, it’s that we have subconscious pre-computed scripts of “if someone utters ‘I trust you’ and your model marks them as an ally, modify the prior used by your plan-making processes towards satisfying their preferences more”. Which is implemented by automatically-activated emotional responses; e. g., the feeling of responsibility automatically flaring up when you’re thinking about someone who put their trust in you.
So to some extent it is a self-fulfilling prophecy. If you’ve decided to trust someone, and their learned-instinct systems recognize that, they’ll automatically reciprocate and become more trustworthy. (Or so the underlying calculations go. It might not work: if they jailbroke out of these systems, if they’ve never had them installed (psychopaths), or if other systems overrode that.)
Not that there isn’t all kinds of sloppy language involved, e. g. when “I trust her” actually just means “I’ve evaluated that the process she uses for [whatever] is reliable” or “leaving myself vulnerable here is a risk, but the CBA is in favour”, instead of “I’ve initiated the ‘trust’ social dance with her and I don’t think she’s defecting”.
And there’s some degree of acausal culture-based negotiation going on
Actually, this seems like an extremely clever instance of that. What evolution (either biological or cultural/memetic) has on its hands is a bunch of scheming, myopic mesa-optimizers walking around in suits of autonomous heuristics. It wants them to be able to cooperate, but the mesa-optimizers are at once too myopic and too clever for that – they pretend to ally then backstab each other.
So what it does is instill a joint social and cognitive protocol which activates whenever someone shows vulnerability to someone else: it initiates a process in the trustee’s mind which immediately strongly biases their short-term plans towards reciprocating that trust by showing vulnerability in turn (which reads to us as, e. g., warm feelings of empathy), which establishes mutually assured destruction, which sets up a lasting incentive structure that ensures the scheming optimizer won’t immediately betray even if the biochemistry stops actively brainwashing them (which it must stop, because that impairs clear thinking).
Or at least that’s the angle on the issue that just occurred to me. May be missing some important pieces, obviously.
I think it might be.
I think the core of the contractual interpretation is right, but I’d phrase it differently. I think “deciding to trust” someone implies a deal where you ally yourself with them in a way that leaves you vulnerable to hostile actions from their side – you’re choosing “cooperate” in the Prisoner’s Dilemma in expectation of greater mutual gain. In exchange, they (1) commit not to take said hostile actions, (2) potentially “reciprocate trust” by showing symmetrical vulnerability (either to ensure MAD, or because pooling resources improves your ability to work together), and (3) bias their policy towards satisfying your preferences, which includes being more responsible when interacting with you and giving you some veto on their actions (and if they fail that, they’ve “betrayed” your trust).
And there’s some degree of acausal culture-based negotiation going on. E. g., it’s not that people explicitly track all of the above consciously, it’s that we have subconscious pre-computed scripts of “if someone utters ‘I trust you’ and your model marks them as an ally, modify the prior used by your plan-making processes towards satisfying their preferences more”. Which is implemented by automatically-activated emotional responses; e. g., the feeling of responsibility automatically flaring up when you’re thinking about someone who put their trust in you.
So to some extent it is a self-fulfilling prophecy. If you’ve decided to trust someone, and their learned-instinct systems recognize that, they’ll automatically reciprocate and become more trustworthy. (Or so the underlying calculations go. It might not work: if they jailbroke out of these systems, if they’ve never had them installed (psychopaths), or if other systems overrode that.)
Not that there isn’t all kinds of sloppy language involved, e. g. when “I trust her” actually just means “I’ve evaluated that the process she uses for [whatever] is reliable” or “leaving myself vulnerable here is a risk, but the CBA is in favour”, instead of “I’ve initiated the ‘trust’ social dance with her and I don’t think she’s defecting”.
Actually, this seems like an extremely clever instance of that. What evolution (either biological or cultural/memetic) has on its hands is a bunch of scheming, myopic mesa-optimizers walking around in suits of autonomous heuristics. It wants them to be able to cooperate, but the mesa-optimizers are at once too myopic and too clever for that – they pretend to ally then backstab each other.
So what it does is instill a joint social and cognitive protocol which activates whenever someone shows vulnerability to someone else: it initiates a process in the trustee’s mind which immediately strongly biases their short-term plans towards reciprocating that trust by showing vulnerability in turn (which reads to us as, e. g., warm feelings of empathy), which establishes mutually assured destruction, which sets up a lasting incentive structure that ensures the scheming optimizer won’t immediately betray even if the biochemistry stops actively brainwashing them (which it must stop, because that impairs clear thinking).
Or at least that’s the angle on the issue that just occurred to me. May be missing some important pieces, obviously.
Seems fascinating, regardless.