I would guess that even the “in the know” people are over-updating, because they usually are Not Measuring What They Think They Are Measuring even qualitatively. Like, the proxies are so weak that the hypothesis “this result will qualitatively generalize to <whatever they actually want to know about>” shouldn’t have been privileged in the first place, and the right thing for a human to do is ignore it completely.
I would guess that even the “in the know” people are over-updating, because they usually are Not Measuring What They Think They Are Measuring even qualitatively. Like, the proxies are so weak that the hypothesis “this result will qualitatively generalize to <whatever they actually want to know about>” shouldn’t have been privileged in the first place, and the right thing for a human to do is ignore it completely.