I don’t know. (As above, “When [users] tell you exactly what they think is wrong and how to fix it, they are almost always wrong.”)
A scoring rule that’s proper in linear space (as you say, “scored on how close their number is to the actual number”) would accomplish this—either for scoring point estimates, or distributions. I don’t think it’s possible to extract an expected value from a confidence interval that covers orders of magnitude, so I expect that would work less well.
I don’t know. (As above, “When [users] tell you exactly what they think is wrong and how to fix it, they are almost always wrong.”)
A scoring rule that’s proper in linear space (as you say, “scored on how close their number is to the actual number”) would accomplish this—either for scoring point estimates, or distributions. I don’t think it’s possible to extract an expected value from a confidence interval that covers orders of magnitude, so I expect that would work less well.