Ah oops, I now see that one of Drake’s follow-up comments was basically about this!
One suggestion that I made to Drake, which I’ll state here in case anyone else is interested:
Define a utility function: for example, utility = -(dollars paid out) - c*(variance of your estimator). Then, see if you can figure out how to sample people to maximize your utility.
I think this sort of analysis may end up being more clear-eyed in terms of what you actually want and how good different sampling methods are at achieving that.
Ah oops, I now see that one of Drake’s follow-up comments was basically about this!
One suggestion that I made to Drake, which I’ll state here in case anyone else is interested:
I think this sort of analysis may end up being more clear-eyed in terms of what you actually want and how good different sampling methods are at achieving that.