Bizarre Failures to Acquire Relevant Evidence
As lukeprog noted, you did not run this post by anyone within CEA who had sufficient knowledge to correct you on some of the matters given above. Lukeprog describes this as ‘common courtesy’. But, more than that, it’s a violation of a good epistemic principle that one should gain easily accessible relevant information before making a point publicly.
The most egregious violation of this principle is that, though you say you focus on the idea that donating to CEA has a ROI greater than 1, and though you repeatedly ask for a ‘calculation’ of impact and claim that CEA is not credible for not being able to provide such a calculation, you haven’t contacted me for the calculation of GWWC’s impact per dollar invested. This isn’t something I’ve been shy about — in a blog post that you link to (as well as elsewhere) I prominently describe this calculated impact-assessment, and invite people to contact me if they want the spreadsheet with the calculation. Insofar as this was the cornerstone of your concern, it’s odd that you didn’t contact me for the spreadsheet. Comments on that impact-assessment would have been helpful, but as far as I’m aware you haven’t read it.
Another example is where you suggest that little thought went into the change of the 80,000 Hours’ declaration of intent. Again, this is information that would have been easily accessible via a quick email to me or Ben Todd. As it happens, the declaration has gone through several iterations; there has been discussion on the core 80,000 Hours’ lists; Ben, myself and other have independently written proposals; and we commissioned one of our best interns to research the topic as part of our general marketing strategy. We concluded that having a lower initial barrier to entry was wise, because it would increase the total number of members, allow us to be more mainstream, and increase the total (though not the proportion) of members who make significant changes to their careers and thereby make the world a significantly better place. (We are also currently discussing whether to introduce a further pledge along the lines of “I intend to dedicate my life to whatever does the most good.”) It wouldn’t be an underestimate to say that several person-weeks of thought and research have gone into the pledges.
A further example is where you guess the number of researchers we have. Again, you could have e-mailed for this information, rather than trying to guess on the basis of the names listed on the website. For this reason, you substantially overestimated how many person-hours we command. Between CEA, over the last six months we have had the equivalent of 3.7 full-time staff. The first 2.6 of these started in July last year, another joined in late September and another in January. GWWC currently has the equivalent of two full-time staff; 80,000 Hours has the equivalent of two and a half full-time staff. For this reason (and perhaps also the planning fallacy), I think you severely overestimate the amount of research we could reasonably expect to deliver in that time.
Another example is where you quote the number of people we have on our mailing lists. This is a good example, because it’s one where I spoke incorrectly in Cambridge. I said that one third of Oxford students were on our mailing list; what I should have said was that about 20% of students coming through fresher’s fair were on our mailing list. It’s precisely errors like these — easy to make in the context of an impromptu group discussion — that show the value of making sure that one’s evidence is reliable.
A further example is where you say “it has been stated that GWWC has an internal price of around £1700 for new pledges” and then, in your response to my query about where this number came from, said that it came from Jacob Trefethen — a volunteer at a chapter, and not currently involved with core GWWC and 80k activities. Again, this is not the sort of evidence on which it’s rational to base a critique — when the option of simply asking me or someone else who works on strategy within CEA was merely an email away.
Another example was: “a large fraction of the people involved with 80,000 hours or GWWC behave like dilettantes”… “Nor do they seem to act as if they wish to seriously optimise the world.” But, as far as I know, you know only one person who works at CEA, Adam Casey, who is an unpaid intern, and you have about one hour’s worth of contact with me. I doubt that, if you knew us personally, and not through material written for an audience encountering the ideas of effective altruism for the first time, you would doubt our intention and commitment to “seriously optimise the world” as you put it. Seeing as this is LessWrong, I’ll quote Eliezer Yudkowsky (stated in an independent internet conversation on Ycombinator). In response to the question, “What application of $4B would, right now, generate the most utility for humanity?” he replied:
“If you know the word “utility”, the people who actually seriously try to figure out the answer to that question live at:
Embarrassingly Poor Arguments
First: You ask:
“For example, the world bank throws ~$43B/year around. Which is easier: To upscale GWWC by a factor of ~17000, or double the mean effectiveness of the World Bank? This should not be a hypothetical question; it should be answered.”
There are a few mistakes here:
First, your comment suggests that you know that we haven’t thought about this. But that’s misleading, because you haven’t ever asked us if we’ve thought about it.
Second, I have no idea where your numbers come from. After searching (inc. here) I still don’t know where $43bn number comes from. And, after trying to figure it out, I also don’t know where your “17 000” figure comes from. GWWC has so far moved $2.5 million and raised $100mn in pledges. Even discounting the literal pledges by 99% and valuing them at $1mn (which would be far too steep in my view), the appropriate figure would be 12 300. So, whatever the basis, 17 000 seems too high.
Third, even neglecting the above points, your figure would only be correct if the cost-effectiveness of the World Bank’s spending were the same as the cost-effectiveness of GWWC top-recommended charities. But we think, and presumably you agree, that the cost-effectiveness of GWWC’s top-recommended charities are significantly better than the World Bank’s mean cost-effectiveness. Aside from anything else, there’s a major difference between donations and loans.
Fourth, if you want to maximize impact yours is not the correct question to ask. If it will get progressively harder to grow GWWC, and if one think that the likelihood of achieving either outcome is very low (both reasonable assumptions), then it could be true that (i) it is easier to double the mean effectiveness of the World Bank than to increase GWWC’s size by a factor of 17000 and (ii) that one ought to use one’s marginal time and resources to grow GWWC. The reason these could both be true is that the marginal benefits from growing GWWC are greater than the marginal benefits of trying to double the effectiveness of the World Bank. Given this, it’s unclear why this question “should be answered”.
Fifth, the question implicitly neglects the fact that growing GWWC has substantial knock-on benefits, including increasing the ability of some GWWC members to influence major international organisations like the World Bank (see the background on Toby’s activities, above).
In general:
i) Starting with something smaller and easier to achieve has instrumental cumulative benefits and option value in a way that staking everything on one big goal does not.
ii) Directly doubling the effectiveness of the World Bank – and other similar projects – is not the comparative advantage of existing EAs in Oxford. Given our success generating and mobilising talented altruists, I think the team here will have greater success taking an indirect route than by attempting to do it directly ourselves. We can use e.g. 80,000 Hours to identify precisely those who have or could develop the requisite skills, credentials and values, and provide them the encouragement, information and practical assistance required to get into positions of major influence over aid effectiveness. Finding and convincing someone to pursue this career is much easier than dedicating your entire life to it yourself, which is what led us to set up 80,000 Hours in the first place.
That’s not to say we aren’t open to the idea. It’s one of my main concerns about my current activities. But it’s misleading to suggest that you have good evidence to believe that we haven’t considered it.
What track record is Eliezer referring to? Is there some external organization that evaluates Givewell? I don’t see any quick, easy way to evaluate their impact or figure out if they’re working as advertised.
The $43bn figure (the amount the World Bank (WB) lent in 2011) can be found on the WB website here, the factor of 17000 comes (I think) from dividing $43 bn by the expected annual donations from the pledges ($43 bn / ($112 mn in pledges / 45 years of work) ~ 17000).
However, obviously, as you state, doubling the effectiveness of WB activities will not have the same impact as bringing CEA up to the size of the WB, unless one (unrealistically) assumes that the GWWC recommended charities are only twice as effective as the average WB intervention (though ideally one should take into account the diminishing marginal returns of GWWC and 80k).
(part 2) The most important mistakes in the post
Bizarre Failures to Acquire Relevant Evidence As lukeprog noted, you did not run this post by anyone within CEA who had sufficient knowledge to correct you on some of the matters given above. Lukeprog describes this as ‘common courtesy’. But, more than that, it’s a violation of a good epistemic principle that one should gain easily accessible relevant information before making a point publicly.
The most egregious violation of this principle is that, though you say you focus on the idea that donating to CEA has a ROI greater than 1, and though you repeatedly ask for a ‘calculation’ of impact and claim that CEA is not credible for not being able to provide such a calculation, you haven’t contacted me for the calculation of GWWC’s impact per dollar invested. This isn’t something I’ve been shy about — in a blog post that you link to (as well as elsewhere) I prominently describe this calculated impact-assessment, and invite people to contact me if they want the spreadsheet with the calculation. Insofar as this was the cornerstone of your concern, it’s odd that you didn’t contact me for the spreadsheet. Comments on that impact-assessment would have been helpful, but as far as I’m aware you haven’t read it.
Another example is where you suggest that little thought went into the change of the 80,000 Hours’ declaration of intent. Again, this is information that would have been easily accessible via a quick email to me or Ben Todd. As it happens, the declaration has gone through several iterations; there has been discussion on the core 80,000 Hours’ lists; Ben, myself and other have independently written proposals; and we commissioned one of our best interns to research the topic as part of our general marketing strategy. We concluded that having a lower initial barrier to entry was wise, because it would increase the total number of members, allow us to be more mainstream, and increase the total (though not the proportion) of members who make significant changes to their careers and thereby make the world a significantly better place. (We are also currently discussing whether to introduce a further pledge along the lines of “I intend to dedicate my life to whatever does the most good.”) It wouldn’t be an underestimate to say that several person-weeks of thought and research have gone into the pledges.
A further example is where you guess the number of researchers we have. Again, you could have e-mailed for this information, rather than trying to guess on the basis of the names listed on the website. For this reason, you substantially overestimated how many person-hours we command. Between CEA, over the last six months we have had the equivalent of 3.7 full-time staff. The first 2.6 of these started in July last year, another joined in late September and another in January. GWWC currently has the equivalent of two full-time staff; 80,000 Hours has the equivalent of two and a half full-time staff. For this reason (and perhaps also the planning fallacy), I think you severely overestimate the amount of research we could reasonably expect to deliver in that time.
Another example is where you quote the number of people we have on our mailing lists. This is a good example, because it’s one where I spoke incorrectly in Cambridge. I said that one third of Oxford students were on our mailing list; what I should have said was that about 20% of students coming through fresher’s fair were on our mailing list. It’s precisely errors like these — easy to make in the context of an impromptu group discussion — that show the value of making sure that one’s evidence is reliable.
A further example is where you say “it has been stated that GWWC has an internal price of around £1700 for new pledges” and then, in your response to my query about where this number came from, said that it came from Jacob Trefethen — a volunteer at a chapter, and not currently involved with core GWWC and 80k activities. Again, this is not the sort of evidence on which it’s rational to base a critique — when the option of simply asking me or someone else who works on strategy within CEA was merely an email away.
Another example was: “a large fraction of the people involved with 80,000 hours or GWWC behave like dilettantes”… “Nor do they seem to act as if they wish to seriously optimise the world.” But, as far as I know, you know only one person who works at CEA, Adam Casey, who is an unpaid intern, and you have about one hour’s worth of contact with me. I doubt that, if you knew us personally, and not through material written for an audience encountering the ideas of effective altruism for the first time, you would doubt our intention and commitment to “seriously optimise the world” as you put it. Seeing as this is LessWrong, I’ll quote Eliezer Yudkowsky (stated in an independent internet conversation on Ycombinator). In response to the question, “What application of $4B would, right now, generate the most utility for humanity?” he replied: “If you know the word “utility”, the people who actually seriously try to figure out the answer to that question live at:
http://www.givingwhatwecan.org/ http://www.givewell.org/ http://80000hours.org/ ”
Embarrassingly Poor Arguments First: You ask: “For example, the world bank throws ~$43B/year around. Which is easier: To upscale GWWC by a factor of ~17000, or double the mean effectiveness of the World Bank? This should not be a hypothetical question; it should be answered.”
There are a few mistakes here:
First, your comment suggests that you know that we haven’t thought about this. But that’s misleading, because you haven’t ever asked us if we’ve thought about it.
Second, I have no idea where your numbers come from. After searching (inc. here) I still don’t know where $43bn number comes from. And, after trying to figure it out, I also don’t know where your “17 000” figure comes from. GWWC has so far moved $2.5 million and raised $100mn in pledges. Even discounting the literal pledges by 99% and valuing them at $1mn (which would be far too steep in my view), the appropriate figure would be 12 300. So, whatever the basis, 17 000 seems too high.
Third, even neglecting the above points, your figure would only be correct if the cost-effectiveness of the World Bank’s spending were the same as the cost-effectiveness of GWWC top-recommended charities. But we think, and presumably you agree, that the cost-effectiveness of GWWC’s top-recommended charities are significantly better than the World Bank’s mean cost-effectiveness. Aside from anything else, there’s a major difference between donations and loans. Fourth, if you want to maximize impact yours is not the correct question to ask. If it will get progressively harder to grow GWWC, and if one think that the likelihood of achieving either outcome is very low (both reasonable assumptions), then it could be true that (i) it is easier to double the mean effectiveness of the World Bank than to increase GWWC’s size by a factor of 17000 and (ii) that one ought to use one’s marginal time and resources to grow GWWC. The reason these could both be true is that the marginal benefits from growing GWWC are greater than the marginal benefits of trying to double the effectiveness of the World Bank. Given this, it’s unclear why this question “should be answered”. Fifth, the question implicitly neglects the fact that growing GWWC has substantial knock-on benefits, including increasing the ability of some GWWC members to influence major international organisations like the World Bank (see the background on Toby’s activities, above).
In general: i) Starting with something smaller and easier to achieve has instrumental cumulative benefits and option value in a way that staking everything on one big goal does not. ii) Directly doubling the effectiveness of the World Bank – and other similar projects – is not the comparative advantage of existing EAs in Oxford. Given our success generating and mobilising talented altruists, I think the team here will have greater success taking an indirect route than by attempting to do it directly ourselves. We can use e.g. 80,000 Hours to identify precisely those who have or could develop the requisite skills, credentials and values, and provide them the encouragement, information and practical assistance required to get into positions of major influence over aid effectiveness. Finding and convincing someone to pursue this career is much easier than dedicating your entire life to it yourself, which is what led us to set up 80,000 Hours in the first place.
That’s not to say we aren’t open to the idea. It’s one of my main concerns about my current activities. But it’s misleading to suggest that you have good evidence to believe that we haven’t considered it.
Eliezer’s HN comment: http://news.ycombinator.com/item?id=4726651
What track record is Eliezer referring to? Is there some external organization that evaluates Givewell? I don’t see any quick, easy way to evaluate their impact or figure out if they’re working as advertised.
The $43bn figure (the amount the World Bank (WB) lent in 2011) can be found on the WB website here, the factor of 17000 comes (I think) from dividing $43 bn by the expected annual donations from the pledges ($43 bn / ($112 mn in pledges / 45 years of work) ~ 17000).
However, obviously, as you state, doubling the effectiveness of WB activities will not have the same impact as bringing CEA up to the size of the WB, unless one (unrealistically) assumes that the GWWC recommended charities are only twice as effective as the average WB intervention (though ideally one should take into account the diminishing marginal returns of GWWC and 80k).