This is a really cool mechanism! I’m surprised I haven’t seen it before—maybe it’s original :)
After thinking about it more, I have a complaint about it, though. The complaint is that it doesn’t feel natural to value the act of reaching out to someone at $X. It’s natural to value an actual sample at $X, and you don’t get a sample every time you reach out to someone, only when they respond.
Like, imagine two worlds. In world A, everyone’s fair price is below X, so they’re guaranteed to respond. You decide you want 1000 samples, so you pay $1000X. In world B, everyone has a 10% chance of responding in your mechanism. To get a survey with the same level of precision (i.e. variance), you still need to get 1000 responses, and not just reach out to 1000 people.
My suspicion is that if you’re paying per (effective) sample, you probably can’t mechanism-design your way out of paying more for people who value their time more. I haven’t tried to prove that, though.
Ah oops, I now see that one of Drake’s follow-up comments was basically about this!
One suggestion that I made to Drake, which I’ll state here in case anyone else is interested:
Define a utility function: for example, utility = -(dollars paid out) - c*(variance of your estimator). Then, see if you can figure out how to sample people to maximize your utility.
I think this sort of analysis may end up being more clear-eyed in terms of what you actually want and how good different sampling methods are at achieving that.
This is a really cool mechanism! I’m surprised I haven’t seen it before—maybe it’s original :)
After thinking about it more, I have a complaint about it, though. The complaint is that it doesn’t feel natural to value the act of reaching out to someone at $X. It’s natural to value an actual sample at $X, and you don’t get a sample every time you reach out to someone, only when they respond.
Like, imagine two worlds. In world A, everyone’s fair price is below X, so they’re guaranteed to respond. You decide you want 1000 samples, so you pay $1000X. In world B, everyone has a 10% chance of responding in your mechanism. To get a survey with the same level of precision (i.e. variance), you still need to get 1000 responses, and not just reach out to 1000 people.
My suspicion is that if you’re paying per (effective) sample, you probably can’t mechanism-design your way out of paying more for people who value their time more. I haven’t tried to prove that, though.
Ah oops, I now see that one of Drake’s follow-up comments was basically about this!
One suggestion that I made to Drake, which I’ll state here in case anyone else is interested:
I think this sort of analysis may end up being more clear-eyed in terms of what you actually want and how good different sampling methods are at achieving that.