Here, the author is keeping in mind Conservation of Expected Evidence. If you could anticipate in advance the direction of any update, you should just update now. You should not expect to be able to get the right answer right away and never need to seriously update it.
There has to be a better way to put this.
The problem is that sometimes you can anticipate the direction. For example, if someone’s flipping a coin, and you think it might have two heads. This is a simple example because a heads is always evidence in favor of the two-heads hypothesis, and a tails is always evidence in favor of the normal-coin hypothesis. We can see you become sure of the direction of evidence in this scenario: If the prior prob of two heads is 1⁄2, then after about ten heads you’re 99% sure the eleventh is also going to be heads.
However, I do think that this is just because of very artificial features of the example that would never hold when making first impressions of people. Specifically, what’s going on in the coin example is a hypothesis that we’re very sure of, that makes very specific predictions. I can’t prove it, but I think that’s what allows you to be very sure of the update direction.
This never happens in social situations where you’ve just recently met someone—you’re never sure of a hypothesis that makes very specific predictions, are you?
I don’t know. I do know that there’s some element of the situation besides conservation of probability going into this. It takes more than just that to derive that updates will be gradual and in an unpredictable direction.
(EDIT: I didn’t emphasize this but updates aren’t necessarily gradual in the coin example—a tails leads to an extreme update. I think that might be related—an extreme update in an unexpected direction balancing a small one in a known direction?)
There has to be a better way to put this.
The problem is that sometimes you can anticipate the direction. For example, if someone’s flipping a coin, and you think it might have two heads. This is a simple example because a heads is always evidence in favor of the two-heads hypothesis, and a tails is always evidence in favor of the normal-coin hypothesis. We can see you become sure of the direction of evidence in this scenario: If the prior prob of two heads is 1⁄2, then after about ten heads you’re 99% sure the eleventh is also going to be heads.
However, I do think that this is just because of very artificial features of the example that would never hold when making first impressions of people. Specifically, what’s going on in the coin example is a hypothesis that we’re very sure of, that makes very specific predictions. I can’t prove it, but I think that’s what allows you to be very sure of the update direction.
This never happens in social situations where you’ve just recently met someone—you’re never sure of a hypothesis that makes very specific predictions, are you?
I don’t know. I do know that there’s some element of the situation besides conservation of probability going into this. It takes more than just that to derive that updates will be gradual and in an unpredictable direction.
(EDIT: I didn’t emphasize this but updates aren’t necessarily gradual in the coin example—a tails leads to an extreme update. I think that might be related—an extreme update in an unexpected direction balancing a small one in a known direction?)