You can expect surprise, because “surprise” isn’t a linear function of the data.
Simplest example: assign a prior probability distribution to some uncertain real variable x. Your expected value E[x] is the average of this distribution. Your expected error E[E[x] - x] is indeed zero! But unless you already know the exact true value, your expected error magnitude E[abs(E[x] - x)] is always positive.
You can expect surprise, because “surprise” isn’t a linear function of the data.
Simplest example: assign a prior probability distribution to some uncertain real variable x. Your expected value E[x] is the average of this distribution. Your expected error E[E[x] - x] is indeed zero! But unless you already know the exact true value, your expected error magnitude E[abs(E[x] - x)] is always positive.