Uninformed Elevation of Trust

I don’t know if there is a standard name for this phenomenon (other than the related Gell-Mann amnesia effect, e.g. “The Sequences are great, except in my area of expertise, where they are terrible.”).

Here is the gist: we trust the data as much as we trust the source, regardless of how much the source trusts the data.

This sounds unobjectionable on the surface. We tend to equate the reliability of the data with the subjectively perceived trustworthiness of the source of data whenever we have no independent means of checking the veracity of the data. What is lost in this near-automatic logic is one small piece: the credence the source itself assigns to the data.

The (faulty) Bayesian math here is pretty straightforward and is left as an exercise to the reader.

A few examples:

  • An investment tip from an insider is perceived as reliable even if the insider themselves would consider it a speculation at best.

  • Life on Venus!

  • US election fraud!

  • A tupperware party.

  • A potential hire introduced by a trusted friend or a coworker.

  • A CFAR workshop. Just kidding, those are super trustworthy.

I think the titular description, Uninformed Elevation of Trust, tends to capture the essence of what happens, quickly and naturally, whenever we forget or neglect to engage our critical reasoning full on. It’s an elevation of trust, not just adjustment of it, because the scenario where we skip the critical evaluation happens more the more we trust the source of data. We just naturally assume that whatever they tell us is as trustworthy as they themselves are, even though it’s obvious that it should be discounted by the factor equal to the source’s degree of belief in what they tell us.

Please feel free to offer your own examples and counter examples.