My point is that this new biased estimate is not your ‘real estimate’ - this is simply not your best guess/posterior distribution given your information.
Sure it is my “real” estimate—because I take real action on its basis.
Let me make a few observations.
First, any “best” estimate narrower than a complete probability distribution implies some loss function which you are minimizing in order to figure out which estimate is “best”. Let’s take the plain-vanilla case of estimating the central point of a distribution which produced some sample of real numbers. The usual estimate for that is the average of the sample numbers (the sample mean) and it is indeed optimal (“the best”) for a particular, quadratic, loss function. But, for example, change the loss function to absolute deviation (L1) and now the median becomes “the best estimate”.
The point is that to prefer any estimate over some other estimate, you must have a loss function already. If you are calling some estimate “best”, this implies a particular loss function.
Second, the usefulness of any estimate is determined by the use you intend for it. “Suitability for a purpose” is an overriding criterion for estimates you produce. Different purposes (“produce an unbiased estimate” and “select a course of action” are different purposes) often require different estimates.
Third, “unbiased” is not an unalloyed blessing. In many situations you face the bias-variance tradeoff and sometimes you do want to have some bias.
Sure it is my “real” estimate—because I take real action on its basis.
Let me make a few observations.
First, any “best” estimate narrower than a complete probability distribution implies some loss function which you are minimizing in order to figure out which estimate is “best”. Let’s take the plain-vanilla case of estimating the central point of a distribution which produced some sample of real numbers. The usual estimate for that is the average of the sample numbers (the sample mean) and it is indeed optimal (“the best”) for a particular, quadratic, loss function. But, for example, change the loss function to absolute deviation (L1) and now the median becomes “the best estimate”.
The point is that to prefer any estimate over some other estimate, you must have a loss function already. If you are calling some estimate “best”, this implies a particular loss function.
Second, the usefulness of any estimate is determined by the use you intend for it. “Suitability for a purpose” is an overriding criterion for estimates you produce. Different purposes (“produce an unbiased estimate” and “select a course of action” are different purposes) often require different estimates.
Third, “unbiased” is not an unalloyed blessing. In many situations you face the bias-variance tradeoff and sometimes you do want to have some bias.