“A good volunteer is hard to find”
From the GiveWell blog, which is often interesting & applicable to our interests, comes a post on the quality of their volunteers:
“In our experience, valuable volunteers are rare. The people who email us about volunteer opportunities generally seem enthusiastic about GiveWell’s mission, and motivated by a shared belief in our goals to give up their free time to help us. Yet, the majority of these people never complete useful work for us.
We ask new volunteers to first complete a test assignment that takes about 2-4 hours. The assignment involves fixing the formatting of our list of sources on two practice pages and allows us to get a sense of their attention to detail and commitment to volunteer hours. Of the 34 people who emailed us expressing an interest in volunteering between September 2010 (when we started keeping track) and May 2011, only 7 have completed the test assignment and gone on to complete valuable work for us.
Of the 34, 10 never responded to my email outlining what GiveWell volunteers do and asking them if they’d like me to send the first assignment. 13 responded to this email and I sent them the first assignment, but they didn’t complete it. The final 4 completed the test assignment, but didn’t send back the next (real) assignment I sent.
It seems rather surprising that almost 80% of people who take the initiative to seek us out and ask for unpaid work fail to complete a single assignment. But maybe this shouldn’t be surprising. Writing an email is quick and exciting; spending a few hours fixing punctuation is not.”
(The dropout rate is probably not due to the perceived low utility of the work—GiveWell seems to be up-front that the test assignment is a test.)
I draw a few lessons from this:
there is likely low-hanging fruit for volunteers in charities or communities in the area of sustained tedious tasks; collecting anecdotes, reports, links, that sort of thing come to mind as LW examples
additional incentives like jsalvatier’s contests may be necessary to draw out community volunteer resources
tricking volunteers into work might be a fruitful approach—perhaps asking for explicit pointers to research or other help might not work, but presenting a half-complete version will elicit useful responses one can mine. (A more productive kind of trolling.)
- Experiment: a good researcher is hard to find by (30 Apr 2012 17:13 UTC; 51 points)
- 's comment on Assessing Kurzweil: the gory details by (17 Jan 2013 1:02 UTC; 15 points)
- 's comment on Smart, (young), ambitious and clueless—what to do to maximize goodness? by (14 Jul 2011 18:40 UTC; -2 points)
Another possibility is that people really don’t like the idea of having to do a 2 to 4 hour deliberately useless test assignment.
It’s a tempting hypothesis, but I’ve heard enough complaints about volunteers from organizations that don’t do that sort of test that I don’t think the test is a crucial factor.
Identifying conscientious people seems to be a hard problem. Do you have suggestions about how it can be done?
I posted to this thread basically to add my complaint. I am involved with a not-for-profit and there is a huge problem with people who enthusiastically volunteer but end up doing little or nothing of value for the organization.
So yeah, I don’t think it’s that people find opportunities elsewhere or that people resent make-work.
Here’s a thought experiment: Suppose that Givewell continues to test volunteers, but instead of something boring and tedious, the “test” is something fun and interesting. But still it is admittedly make-work. I predict that the percentage of applicants who complete the test will rise dramatically.
I presume that the reason they put this program in place and began to keep statistics is that they perceived a significant issue with volunteers never completing assignments. Yes it would be nice if we had more data on that, but without that data it would be really hard to tease out the significance of this marginal difference.
Volunteers who fail to complete real assignments can be very disruptive to an organization and a drain on its other resources who must put in time on training, review and various forms of hand-holding. So, it is probably not a bad idea for Givewell to do this. Naturally volunteers do not want to spend several hours on a make-work assignment but if they are truly interested in benefiting this particular organization then they should be able to see the larger picture that makes it necessary.
The characterization of the test assignment as deliberately useless strikes me as unfair; formatting the lists of sources is useful even if marginally so.
It seems like most of this can already be accomplished very cheaply by Mechanical Turk.
I’m sure, but from reading the Mechanical Turk literature (eg. my boss is a robot), you need really specific tasks and a framework for working through MT so you can do majority voting on submissions and that sort of thing.
I have a hard time thinking of things you would want to do that are mechanical enough to get good results out of Turk, and large-scale enough that you would recover all the overhead of using MT and the actual fees (to say nothing of learning how to use MT and your framework!).
The article resonated with me because a number of my own activities are pretty repetitive volunteer stuff; like reading through the archives of the Evangelion ML looking for forgotten gems and information, but I can’t imagine trusting that to Turkers because so many of the important parts are things I can’t explain and only recognize their importance serendipitously and sometimes only in retrospect after I have learned about a related topic. Or writing the DNB FAQ, something which any volunteer could do if they just read through the DNB ML’s emails to see what is important and consolidated it all, but again not something you could really write explicit instructions for.
Another option might be something less mechanized like freelancer.com (which another LW recommended as useful). Do you see that as more viable?
That’d probably work much better as long as you are able to pick a freelancer with skills or credentials matching your needs. Still wouldn’t help with my Evangelion example, but would with the DNB FAQ (pick someone with Internet and psychology skills and tell them ‘flag any email that looks interesting for an FAQ etc.’).
On the other hand, you would also be paying even more on freelance or elance, and at that point, you’re not really dealing with volunteer labor but lightweight outsourcing. (I don’t know if freelancer.com even lets you pay $0 for volunteers, like Mechanical Turk does.)
Yes, good point.
I have a hypothesis, which is that a number of those people found somewhere else to spend their hours in the meantime (Possibly becoming valuable volunteers at other locations) and simply forgot to inform Givewell.
As an example of how this might work, I’ll imagine a scenario where I apply to do something at 125 organizations. I hear back from 25 of them. I respond to 5 interviews or tests. An organization and I eventually select 1 final place to spend my time at.
This would be a circumstance where there is only one organization which thinks “Ah, a valuable worker.” out of 125 organizations who saw my application. It leaves also leaves a number of loose ends where people can drop off.
Although, as evidence against this hypothesis, it seems a bit odd that not one person Givewell tracked had a response along the lines of “Thank you for your interest, but I have found another opportunity elsewhere.” I mean, if my hypothesis happened to be the case, it seems like I should expect at least a one person to have done this out of the 27 who applied but did not end up doing valuable work.
I can’t test the hypothesis myself, but Givewell could always send out a followup survey if they wanted to test the idea.
Possible solution, avoid make-work: http://lesswrong.com/r/discussion/lw/c4c/experiment_a_good_researcher_is_hard_to_find/