1. “Trust” does seem to me to often be an epistemically broken thing that rides on human-peculiar social dynamics and often shakes out to gut-understandings of honor and respect and loyalty etc.
2. I think there is a version that doesn’t route through that stuff. Trust in the “trust me” sense is a bid for present-but-not-necessarily-permanent suspension of disbelief, where the stakes are social credit. I.e. When I say, “trust me on this,” I’m really saying something like, “All of that anxious analysis you might be about to do to determine if X is true? Don’t do it. I claim that using my best-effort model of your values, the thing you should assume/do to fulfill them in this case is X. To the extent that you agree that I know you well and want to help you and tend to do well for myself in similar situations, defer to me on this. I predict you’ll thank me for it (because, e.g., confirming it yourself before acting is costly), and if not...well I’m willing to stake some amount of the social credit I have with you on it.” [Edit: By social credit here I meant something like: The credence you give to it being a good idea to engage with me like this.]
Similarly:
“I decided to trust her” → “I decided to defer to her claims on this thing without looking into it much myself (because it would be costly to do otherwise and I believe—for some reason—that she is sufficiently likely to come to true conclusions on this, is probably trying to help me, knows me fairly well etc.) And if this turns out badly, I’ll (hopefully) stop deciding to do this.”
“Should I trust him?” → “Does the cost/benefit analysis gestured at above come out net positive in expectation if I defer to him on this?”
“They offered me their trust” → “They believe that deferring to me is their current best move and if I screw this up enough, they will (hopefully) stop thinking that.”
So, I feel like I’ve landed fairly close to where you did but there is a difference in emphasis or maybe specificity. There’s more there than asking “what do they believe, and what caused them to believe it?” Like, that probably covers it but more specifically the question I can imagine people asking when wondering whether or not to “trust” someone is instead, “do I believe that deferring these decisions/assumptions to them in this case will turn out better for me than otherwise?” Where the answer can be “yes” because of things like cost-of-information or time constraints etc. If you map “what do they believe” to “what do they believe that I should assume/do” and “what caused them to believe it” to “how much do they want to help me, how well do they know me, how effective are they in this domain, …” then we’re on the same page.
Anecdotal 2¢: This is very accurate in my experience. Basically every time I talk to someone outside of tech/alignment about AI risk, I have to go through the whole “we don’t know what algorithms the AI is running to do what it does. Yes, really.” thing. Every time I skip this accidentally, I realize after a while that this is where a lot of confusion is coming from.