Thanks! I think these questions do matter, though it’s certainly exploratory work, and at the moment tractability is still unclear. I talk about some of the practical upshots in the final essay.
wdmacaskill
Karma: 1,072
Thanks. On the EA Forum Ben West points out the clarification: I’m not assuming x-risk is lower than 10%; in my illustrative example I suggest 20%. (My own views are a little lower than that, but not an OOM lower, especially given that this % is about locking into near-0 value futures, not just extinction.) I wasn’t meaning to be placing a lot of weight on the superforecasters’ estimates.
Actually, because I ultimately argue that Flourishing is only 10% or less, for this part of the argument to work (i.e., for Flourishing to be be greater in scale than Surviving), I only need x-risk this century to be less than 90%! (Though the argument gets a lot weaker the higher your p(doom)).
The point is that we’re closer to the top of the x-axis than we are to the top of the y-axis.
Ah, by the “software feedback loop” I mean: “At the point of time at which AI has automated AI R&D, does a doubling of cognitive effort result in more than a doubling of output? If yes, there’s a software feedback loop—you get (for a time, at least) accelerating rates of algorithmic efficiency progress, rather than just a one-off gain from automation.”
I see now why you could understand “RSI” to mean “AI improves itself at all over time”. But even so, the claim would still hold—even if (implausibly) AI gets no smarter than human-level, you’d still get accelerated tech development, because the quantity of AI research effort would increase at a growth rate much faster than the quantity of human research effort.
There’s definitely a new trend towards custom-website essays. Forethought is a website for lots of research content, though (like Epoch), not just PrepIE.
And I don’t think it’s because of people getting more productive because of reasoning models—AI was helpful for PrepIE but more like 10-20% productivity boost than 100% boost, and I don’t think AI was used much for SA, either.
Thanks—appreciate that! It comes up a little differently for me, but still an issue—we’ve asked the devs to fix.
Argh! Original post didn’t go through (probably my fault), so this will be shorter than it should be:
First point:
I know very little about CEA, and a brief check of their website leaves me a little unclear on why Luke recommends them, aside from the fact that they apparently work closely with FHI.
CEA = Giving What We Can, 80,000 Hours, and a bit of other stuff
Reason → donations to CEA predictably increase the size and strength of the EA community, a good proportion of whom take long-run considerations very seriously and will donate to / work for FHI/MIRI, or otherwise pursue careers with the aim of extinction risk mitigation. It’s plausible that $1 to CEA generates significantly more than $1′s worth of x-risk-value [note: I’m a trustee and founder of CEA].
Second point:
Don’t forget CSER. My view is that they are even higher-impact than MIRI or FHI (though I’d defer to Sean_o_h if he disagreed). Reason: marginal donations will be used to fund program management + grantwriting, which would turn ~$70k into a significant chance of ~$1-$10mn, and launch what I think might become one of the most important research institutions in the world. They have all the background (high profile people on the board; an already written previous grant proposal that very narrowly missed out on being successful). High leverage!
CEA and CFAR don’t do anything, to my knowledge, that would increase these odds, except in exceedingly indirect ways.
People from CEA, in collaboration with FHI, have been meeting with people in the UK government, and are producing policy briefs on unprecedented risks from new technologies, including AI (the first brief will go on the FHI website in the near future). These meetings arose as a result of GWWC media attention. CEA’s most recent hire, Owen Cotton-Barratt, will be helping with this work.
your account of effective altruism seems rather different from Will’s: “Maybe you want to do other things effectively, but >then it’s not effective altruism”. This sort of mixed messaging is exactly what I was objecting too.
I think you’ve revised the post since you initially wrote it? If so, you might want to highlight that in the italics at the start, as otherwise it makes some of the comments look weirdly off-base. In particular, I took the initial post to aim at the conclusion:
EA is utilitarianism in disguise which I think is demonstrably false.
But now the post reads more like the main conclusion is:
EA is vague on a crucial issue, which is whether the effective pursuit of non-welfarist goods counts as effective altruism. which is a much more reasonable thing to say.
I think the simple answer is that “effective altruism” is a vague term. I gave you what I thought was the best way of making it precise. Weeatquince, and Luke Muelhauser wanted to make it precise in a different way. We could have a debate about which is the more useful precisifcation, but I don’t think that here is the right place for that.
On either way of making the term precise, though, EA is clearly not trying to be the whole of morality, or to give any one very specific conception of morality. It doesn’t make a claim about side-constraints; it doesn’t make a claim about whether doing good is supererogatory or obligatory; it doesn’t make a claim about the nature of welfare. EA is broad tent, and deliberately so: very many different ethical perspectives will agree, for example, that it’s important to find out which charities do the most to improve the welfare of those living in extreme poverty (as measured by QALYs etc), and then encouraging people to give to those charities. If so, then we’ve got an important activity that people of very many different ethical backgrounds can get behind—which is great!
Hi,
Thanks for this post. The relationship between EA and well-known moral theories is something I’ve wanted to blog about in the past.
So here are a few points:
1. EA does not equal utilitarianism.
Utilitarianism makes many claims that EA does not make:
EA does not claim whether it’s obligatory or merely supererogatory to spend one’s resources helping others; utilitarianism claims that it is obligatory.
EA does not make a claim about whether there are side-constraints - certain things that it is impermissible to do, even if it were for the greater good. Utilitarianism claims that it’s always obligatory to act for the greater good.
EA does not claim that there are no other things besides welfare that are of value; utilitarianism does claim this.
EA does not make a precise claim about what promoting welfare consists in (for example, whether it’s more important to give one unit of welfare to someone who is worse-off than someone who is better-off; or whether hedonistic, preference-satisfactionist or objective list theories of wellbeing are correct); any specific form of utilitarianism does make a precise claim about this.
Also, note that some eminent EAs are not even consequentialist leaning, let alone utilitarian: e.g. Thomas Pogge (political philosopher) and Andreas Mogensen (Assistant Director of Giving What We Can) explicitly endorse a rights-based theory of morality; Alex Foster (epic London EtG-er) and Catriona MacKay (head of the GWWC London chapter) are both Christian (and presumably not consequentialist, though I haven’t asked).
2. Rather, EA is something that almost every plausible moral theory is in favour of.
Almost every plausible moral theory thinks that promoting the welfare of others in an effective way is a good thing to do. Some moral theories that promoting the welfare of others is merely supererogatory, and others think that there are other values at stake. But EA is explicitly pro promoting welfare; it’s not anti other things, and it doesn’t claim that we’re obligated to be altruistic, merely that it’s a good thing to do.
3. Is EA explicitly welfarist?
The term ‘altruism’ suggests that it is. And I think that’s fine. Helping others is what EAs do. Maybe you want to do other things effectively, but then it’s not effective altruism—it’s “effective justice”, “effective environmental preservation”, or something. Note, though, that you may well think that there are non-welfarist values—indeed, I would think that you would be mistaken not to act as if there were, on moral uncertainty grounds alone—but still be part of the effective altruism movement because you think that, in practice, welfare improvement is the most important thing to focus on.
So, to answer your dilemma:
EA is not trying to be the whole of morality.
It might be the whole of morality, if being EA is the only thing that is required of one. But it’s not part of the EA package that EA is the whole of morality. Rather, it represents one aspect of morality—an aspect that is very important for those living in affluent countries, and who have tremendous power to help others. The idea that we in rich countries should be trying to work out how to help others as effectively as possible, and then actually going ahead and doing it, is an important part of almost every plausible moral theory.
I explicitly address this in the second paragraph of the “The history of GiveWell’s estimates for lives saved per dollar” section of my post as well as the “Donating to AMF has benefits beyond saving lives” section of my post.
Not really. You do mention the flow-on benefits. But you don’t analyse whether your estimate of “good done per dollar” has increased or decreased. And that’s the relevant thing to analyse. If you argued “cost per life saved has had greater regression to your prior than you’d expected; and for that reason I expect my estimates of good done per dollar to regress really substantially” (an argument I think you would endorse), I’d accept that argument, though I’d worry about how much it generalises to cause-areas other than global poverty. (e.g. I expect there to be much less of an ‘efficient market’ for activities where there are fewer agents with the same goals/values, like benefiting non-human animals, or making sure the far-future turn out well). Optimism bias still holds, of course.
You say that “cost-effectiveness estimates skew so negatively.” I was just pointing out that for me that hasn’t been the case (for good done per $), because long-run benefits strike me as swamping short-term benefits, a factor that I didn’t initially incorporate into my model of doing good. And, though I agree with the conclusion that you want as many different angles as possible (etc), focusing on cost per life saved rather than good done per dollar might lead you to miss important lessons (e.g. “make sure that you’ve identified all crucial normative and empirical considerations”). I doubt that you personally have missed those lessons. But they aren’t in your post. And that’s fine, of course, you can’t cover everything in one blog post. But it’s important for the reader not to overgeneralise.
I agree with this. I don’t think that my post suggests otherwise.
I wasn’t suggesting it does.
Good post, Jonah. You say that: “effective altruists should spend much more time on qualitative analysis than on quantitative analysis in determining how they can maximize their positive social impact”. What do you mean by “qualitative analysis”? As I understand it, your points are: i) The amount by which you should regress to your prior is much greater than you had previously thought, so ii) you should favour robustness of evidence more than you had previously. But that doesn’t favour qualitative vs non-qualitative evidence. It favours more robust evidence of lower but good cost-effectiveness over less robust evidence of higher cost-effectiveness. The nature of the evidence could be either qualitative or quantitative, and the things you mention in “implications” are generally quantitative.
In terms of “good done per dollar”—for me that figure is still far greater than I began with (and I take it that that’s the question that EAs are concerned with, rather than “lives saved per dollar”). This is because, in my initial analysis—and in what I’d presume are most people’s initial analyses—benefits to the long-term future weren’t taken into account, or weren’t thought to be morally relevant. But those (expected) benefits strike me, and strike most people I’ve spoken with who agree with the moral relevance of them, to be far greater than the short-term benefits to the person whose life is saved. So, in terms of my expectations about how much good I can do in the world, I’m able to exceed those by a far greater amount than I’d previously thought likely. And that holds true whether it costs $2000 or $20000 to save a life. I’m not mentioning that either to criticise or support your post, but just to highlight that the lesson to take from past updates on evidence can look quite different depending on whether you’re talking about “good done per dollar” or “lives saved per dollar”, and the former is what we ultimately care about.
Final point: Something you don’t mention is that, when you find out that your evidence is crappier than you’d thought, two general lessons are to pursue things with high option value and to pay to gain new evidence (though I acknowledge that this depends crucially on how much new evidence you think you’ll be able to get). Building a movement of people who are aiming to do the most good with their marginal resources, and who are trying to work out how best to do that, strikes me as a good way to achieve both of these things.
Thanks for mentioning this—I discuss Nozick’s view in my paper, so I’m going to edit my comment to mention this. A few differences:
As crazy88 says, Nozick doesn’t think that the issue is a normative uncertainty issue—his proposal is another first-order decision theory, like CDT and EDT. I argue against that account in my paper. Second, and more importantly, Nozick just says “hey, our intuitions in Newcomb-cases are stakes-sensitive” and moves on. He doesn’t argue, as I do, that we can explain the problematic cases in the literature by appeal to decision-theoretic uncertainty. Nor does he use decision-theoretic uncertainty to respond to arguments in favour of EDT. Nor does he respond to regress worries, and so on.
Don’t worry, that’s not an uncomfortable question. UDT and MDT are quite different. UDT is a first-order decision theory. MDT is a way of extending decision theories—so that you take into account uncertainty about which decision theory to use. (So, one can have meta causal decision theory, meta evidential decision theory, and (probably, thought I haven’t worked through it) meta updateless decision theory.)
UDT, as I understand it (and note I’m not at all fluent in UDT or TDT) always one-boxes; whereas if you take decision-theoretic uncertainty into account you should sometimes one-box and sometimes two-box, depending on the relative value of the contents of the two boxes. Also, UDT gets what most decision-theorists consider the wrong answer in the smoking lesion case, whereas the account I defend, meta causal decision theory, doesn’t (or, at least, doesn’t, depending on one’s credences in first-order decision theories).
To illustrate, consider the case:
High-Stakes Predictor II (HSP-II) Box C is opaque; Box D, transparent. If the Predictor predicts that you choose Box C only, then he puts one wish into Box C, and also a stick of gum. With that wish, you save the lives of 1 million terminally ill children. If he predicts that you choose both Box C and Box D, then he puts nothing into Box C. Box D — transparent to you — contains an identical wish, also with the power to save the lives of 1 million children, so if one had both wishes one would save 2 million children in total. However, Box D contains no gum. One has two options only: choose Box C only, or both Box C and Box D.
In this case, intuitively, should you one box, or two box? My view is clear: that if someone one-boxes in the above case, they made the wrong decision. And it seems to me that this is best explained with appeal to decision-theoretic uncertainty.
Other questions: Bostrom’s parliamentary model is different. Between EDT and CDT, the intertheoretic comparisons of value are easy, so there’s no need to use the parliamentary analogy—one can just straightforwardly take an expectation over decision theories.
Pascal’s Mugging (aka the “Fanaticism” worry). This is a general issue for attempts to take normative uncertainty into account in one’s decision-making, and not something I discuss in my paper. But if you’re concerned about Pascal’s mugging and, say, think that a bounded Decision Theory is the best way to respond to the problem—then at the meta level you should also have a bounded decision theory (and at the meta meta level, and so on).
(part 3; final part)
Second: The GWWC Pledge. You say:
“The GWWC site, for example, claims that from 291 members there will be £72.68M pledged. This equates to £250K / person over the course of their life. Claiming that this level of pledging will occur requires either unreasonable rates of donation or multi-decade payment schedules. If, in line with GWWC’s projections, around 50% of people will maintain their donations, then assuming a linear drop off the expected pledge from a full time member is around £375K. Over a lifetime, this is essentially £10K / year. It seems implausible that expected mean annual earnings for GWWC members is of order £100K.”
Again, there are quite a few mistakes:
First, in comments you twice say that “£112.8M” has been pledged rather than “$112.8M”. I know that’s just a typo but it’s an important one.
Second, you say that the GWWC site claims that, “there will be £72.68M pledged” (future tense). It doesn’t, it says, “$112.8mn pledged” (past tense). It’s a pretty important difference – the pledging is something that has happened, not something that will happen. This might partly explain the confusion discussed in point 4, below. Third, and more substantively, you don’t consider the idea, raised in other comments, that some donors might be donating considerably more than 10%, or that some donors might be donating considerably more than the mean. Both are true of GWWC pledgers.
Fourth, you seem to wilfully misunderstand the verb ‘to pledge’. I regularly make the following statement: “I have pledged to give everything I earn above £20 000 p.a. [PPP and inflation-adjusted to Oxford 2009]”. Am I lying when I say that? Using synonyms, I could have said “I promise to give…”, “I commit to give…” or “I sincerely intend to give…”. None of these entail “I am certain that I will donate everything above £20 000 p.a.”. Using my belief that I will earn on average over £42 000 p.a. [PPP and inflation-adjusted to Oxford 2009] over the course of my life, and that I will work until I’m 68, I can infer that I’ve pledged to give over £1 000 000 over the course of my life, which is also something I say. Am I lying when I say that? (Also note that if only 73 people made the same pledge as me, then we would have jointly pledged the current GWWC amount).
Fifth, I don’t know why you took us to use the $100mn pledged figure as an estimate of our impact. In fact you had evidence to the contrary. In a blog post that you cite I said: “As of last March, we’d invested $170 000’s worth of volunteer time into Giving What We Can, and had moved $1.7 million to GiveWell or GWWC top-recommended development charities, and raised a further $68 million in pledged donations. Taking into account the facts that some proportion of this would have been given anyway, there will be some member attrition, and not all donations will go to the very best charities (and using data for all these factors when possible), we estimate that we had raised $8 in realised donations and $130 in future donations for every $1’s worth of volunteer time invested in Giving What We Can.” (emphasis added).
Finally, I think that the GWWC pledge is misleading only if it’s taken to be a measure of our impact. But we don’t advertise it as that. We could try to make it some other number. We could adjust the number downwards, in order to take into account: how much would have been given anyway; member attrition; a discount rate. Or we could adjust the number upwards, in order to take into account: overgiving; real growth of salaries, and inflation. It could also be adjusted downward to take into account that not all donations are to GW or GWWC recommended charities, or (perhaps) upwards to take into account the idea that we will have better evidence about the best giving opportunities in a few years’ time, and thereby be able to donate to charities better than AMF, SCI or DtW. But any number we gave based on these adjustments would be more misleading and arbitrary than the literal amount pledged. It would also be more confusing for the large majority of our website viewers who haven’t thought about things like counterfactual giving or whether the discount rate should be positive or negative over the next few years; they’re used to the social norm which is to advertise pledges as stated. Until you, no-one who does understand issues such as counterfactual giving and discount rates has understood the amount pledged figure as an impact-assessment.
In comments there was some uncertainty about how we come up with the total pledged figure. What we do is as follows. Each member, when they return their pledge form, states a) what percentage they commit to (or, if taking the Further Pledge, the baseline income above which they give everything); b) their birthdate; c) their expected average earnings per annum. Assuming a (conservative) standard retirement age, that allows us to calculate their expected donations. In some cases, members understandably don’t want to reveal their expected earnings. What we used to do, in such cases, is to use the mean earnings of all the other members who have given their incomes. However, when, recently, one member joined with very large expected earnings (pursuing earning to give), we raised the question whether this method suffers from sample bias, because people who expect to earn a lot will be more likely to report. I’m not sure that’s true: I could imagine that people who earn more often don’t want to flaunt that fact. However, wanting to be conservative, we decided instead to use the mean earnings of the country in which the member works.
Bottom Line for Readers If you’re interested in the question of whether 80,000 Hours and Giving What We Can have acted optimally or will act optimally in the future, the answer is simple: certainly not. We inevitably do some things worse than we could have done, and we value your input on concrete suggestions about how our organisations can improve.
If you’re interested in the question of whether $1 invested in 80,000 Hours or Giving What We Can produces more than $1’s worth of value for the best causes, read here, here, here and here and, most of all, contact me for the calculations and, if you’d like, our latest business plan, at will dot crouch at 80000hours.org. So far, I haven’t seen any convincing arguments to the conclusion that we fail to have a ROI greater than 1; however, it’s something I’d love additional input on, as the outside view makes me wary about believing that I work for the best charity I know of.
(part 2) The most important mistakes in the post
Bizarre Failures to Acquire Relevant Evidence As lukeprog noted, you did not run this post by anyone within CEA who had sufficient knowledge to correct you on some of the matters given above. Lukeprog describes this as ‘common courtesy’. But, more than that, it’s a violation of a good epistemic principle that one should gain easily accessible relevant information before making a point publicly.
The most egregious violation of this principle is that, though you say you focus on the idea that donating to CEA has a ROI greater than 1, and though you repeatedly ask for a ‘calculation’ of impact and claim that CEA is not credible for not being able to provide such a calculation, you haven’t contacted me for the calculation of GWWC’s impact per dollar invested. This isn’t something I’ve been shy about — in a blog post that you link to (as well as elsewhere) I prominently describe this calculated impact-assessment, and invite people to contact me if they want the spreadsheet with the calculation. Insofar as this was the cornerstone of your concern, it’s odd that you didn’t contact me for the spreadsheet. Comments on that impact-assessment would have been helpful, but as far as I’m aware you haven’t read it.
Another example is where you suggest that little thought went into the change of the 80,000 Hours’ declaration of intent. Again, this is information that would have been easily accessible via a quick email to me or Ben Todd. As it happens, the declaration has gone through several iterations; there has been discussion on the core 80,000 Hours’ lists; Ben, myself and other have independently written proposals; and we commissioned one of our best interns to research the topic as part of our general marketing strategy. We concluded that having a lower initial barrier to entry was wise, because it would increase the total number of members, allow us to be more mainstream, and increase the total (though not the proportion) of members who make significant changes to their careers and thereby make the world a significantly better place. (We are also currently discussing whether to introduce a further pledge along the lines of “I intend to dedicate my life to whatever does the most good.”) It wouldn’t be an underestimate to say that several person-weeks of thought and research have gone into the pledges.
A further example is where you guess the number of researchers we have. Again, you could have e-mailed for this information, rather than trying to guess on the basis of the names listed on the website. For this reason, you substantially overestimated how many person-hours we command. Between CEA, over the last six months we have had the equivalent of 3.7 full-time staff. The first 2.6 of these started in July last year, another joined in late September and another in January. GWWC currently has the equivalent of two full-time staff; 80,000 Hours has the equivalent of two and a half full-time staff. For this reason (and perhaps also the planning fallacy), I think you severely overestimate the amount of research we could reasonably expect to deliver in that time.
Another example is where you quote the number of people we have on our mailing lists. This is a good example, because it’s one where I spoke incorrectly in Cambridge. I said that one third of Oxford students were on our mailing list; what I should have said was that about 20% of students coming through fresher’s fair were on our mailing list. It’s precisely errors like these — easy to make in the context of an impromptu group discussion — that show the value of making sure that one’s evidence is reliable.
A further example is where you say “it has been stated that GWWC has an internal price of around £1700 for new pledges” and then, in your response to my query about where this number came from, said that it came from Jacob Trefethen — a volunteer at a chapter, and not currently involved with core GWWC and 80k activities. Again, this is not the sort of evidence on which it’s rational to base a critique — when the option of simply asking me or someone else who works on strategy within CEA was merely an email away.
Another example was: “a large fraction of the people involved with 80,000 hours or GWWC behave like dilettantes”… “Nor do they seem to act as if they wish to seriously optimise the world.” But, as far as I know, you know only one person who works at CEA, Adam Casey, who is an unpaid intern, and you have about one hour’s worth of contact with me. I doubt that, if you knew us personally, and not through material written for an audience encountering the ideas of effective altruism for the first time, you would doubt our intention and commitment to “seriously optimise the world” as you put it. Seeing as this is LessWrong, I’ll quote Eliezer Yudkowsky (stated in an independent internet conversation on Ycombinator). In response to the question, “What application of $4B would, right now, generate the most utility for humanity?” he replied: “If you know the word “utility”, the people who actually seriously try to figure out the answer to that question live at:
Embarrassingly Poor Arguments First: You ask: “For example, the world bank throws ~$43B/year around. Which is easier: To upscale GWWC by a factor of ~17000, or double the mean effectiveness of the World Bank? This should not be a hypothetical question; it should be answered.”
There are a few mistakes here:
First, your comment suggests that you know that we haven’t thought about this. But that’s misleading, because you haven’t ever asked us if we’ve thought about it.
Second, I have no idea where your numbers come from. After searching (inc. here) I still don’t know where $43bn number comes from. And, after trying to figure it out, I also don’t know where your “17 000” figure comes from. GWWC has so far moved $2.5 million and raised $100mn in pledges. Even discounting the literal pledges by 99% and valuing them at $1mn (which would be far too steep in my view), the appropriate figure would be 12 300. So, whatever the basis, 17 000 seems too high.
Third, even neglecting the above points, your figure would only be correct if the cost-effectiveness of the World Bank’s spending were the same as the cost-effectiveness of GWWC top-recommended charities. But we think, and presumably you agree, that the cost-effectiveness of GWWC’s top-recommended charities are significantly better than the World Bank’s mean cost-effectiveness. Aside from anything else, there’s a major difference between donations and loans. Fourth, if you want to maximize impact yours is not the correct question to ask. If it will get progressively harder to grow GWWC, and if one think that the likelihood of achieving either outcome is very low (both reasonable assumptions), then it could be true that (i) it is easier to double the mean effectiveness of the World Bank than to increase GWWC’s size by a factor of 17000 and (ii) that one ought to use one’s marginal time and resources to grow GWWC. The reason these could both be true is that the marginal benefits from growing GWWC are greater than the marginal benefits of trying to double the effectiveness of the World Bank. Given this, it’s unclear why this question “should be answered”. Fifth, the question implicitly neglects the fact that growing GWWC has substantial knock-on benefits, including increasing the ability of some GWWC members to influence major international organisations like the World Bank (see the background on Toby’s activities, above).
In general: i) Starting with something smaller and easier to achieve has instrumental cumulative benefits and option value in a way that staking everything on one big goal does not. ii) Directly doubling the effectiveness of the World Bank – and other similar projects – is not the comparative advantage of existing EAs in Oxford. Given our success generating and mobilising talented altruists, I think the team here will have greater success taking an indirect route than by attempting to do it directly ourselves. We can use e.g. 80,000 Hours to identify precisely those who have or could develop the requisite skills, credentials and values, and provide them the encouragement, information and practical assistance required to get into positions of major influence over aid effectiveness. Finding and convincing someone to pursue this career is much easier than dedicating your entire life to it yourself, which is what led us to set up 80,000 Hours in the first place.
That’s not to say we aren’t open to the idea. It’s one of my main concerns about my current activities. But it’s misleading to suggest that you have good evidence to believe that we haven’t considered it.
(part 1) Summary Thanks once again, Jonathan, for taking the time to write publicly about CEA, and to make some suggestions about ways in which CEA might be falling short. In what follows I’ll write a candid response to your post, which I hope you’ll take as a sign of respect — this is LW and I know that honesty in this community is valued far more than sugarcoating. Ultimately, we’re all aiming here to proportion our beliefs to our evidence, and beating around the bush doesn’t help with that aim.
In your post you raise some important issues — often issues that those within CEA have also been thinking about. In general, however, the methodology by which you researched and wrote your post was poor. For this reason, there are crucial factual errors in your post that could easily have been avoided, and errors of argumentation that border on embarrassing. This is unfortunate. Powerful criticism of CEA’s activities is extremely important to us: in fact, in the absence of more direct forms of feedback (like profit and loss), it’s vital. But writing poorly researched and poorly thought-through criticism adds more noise than signal; this makes it harder for us in the future to distinguish the incisive and well-evidenced criticism from the rest, which just harms everyone.
I’ll mention some of the issues that you’ve raised that I think are important to think about, before going on to detail some of the mistakes you make in your post. I’ll note just now that, because of other commitments, this post will be the last I make on this thread.
Some Important Points
Individuals vs Large Organizations You ask why we focus on individuals, rather than large foundations, or governments, or intergovernmental institutions like the World Bank. This is a good question, and something we wrestle with. Indeed, it’s also something we’ve pursued. The media attention generated by Giving What We Can has provided a platform for Dr Toby Ord, the principal founder of GWWC, to travel to and speak to the UK Secretary of State for Development, the UK’s Department for International Development, the Centre for Global Development, 10 Downing Street, the Disease Control Priorities Network, the WHO and as it happens, the World Bank, about aid cost effectiveness and how to increase it. He has already had some success in this regard, which wouldn’t have been possible without GWWC, and he expects to spend a significant proportion of his career on this issue.
The question of whether to spend marginal resources influencing individuals versus governmental and international organisations is non-trivial to answer: international organisations have larger budgets, but are more difficult to access and more difficult to influence. If you think it obvious that we should be influencing the latter, I’d be interested to know your reasons. Later in this response, I’ll discuss your suggestion in more depth.
Transparency You raised concerns about the transparency of GWWC and 80,000 Hours. I agree that this is something that both organisations could work on. We have taken steps so far in the direction of transparency, especially in making the organisations transparent to donors and potential donors. Both 80k and GWWC have in-depth 6-monthly reviews, where their progress is assessed internally by the trustees (myself, Nick Beckstead and Toby Ord), and externally, by people, often donors, within the effective altruism community who are not closely involved with the running of the organisation. GWWC has posted on this here, and noted that if you wanted to read the reports from the review you are able to request them. 80,000 hours will make a similar post soon.
In addition, at the request of Giles, I opened CEA up for questioning on LessWrong, and wrote a detailed response to the questions posted there. I try to provide in-depth responses to any questions I receive via e-mail. And I provide the spreadsheet and explanation of an in-depth calculation of GWWC’s impact per dollar to anyone who asks (accurate as of ~March 2012 – we plan to do this annually).
One issue in keeping a start-up organisation transparent is that the nature of our activities changes rapidly. The very idea of 80,000 Hours as primarily a service organization, providing free careers advice, was only thought up in early July 2012. People switch positions regularly while we get a better understanding of whose comparative advantage lies where. It’s difficult to be transparent and non-misleading when you know that the facts might change radically within the space of a few months. There are also many things to be done, and investing in increased transparency has to be weighed against raising more money, or pledges, or making more career changes. So far, we’ve focused on being transparent to our donors and potential donors, which I still think is the right call — but it’s important to think about and reassess this on a regular basis. I’d welcome further thoughts on if you think that we’ve made the wrong trade-off here.
Publishing You briefly suggest the idea that we should use publications as a metric of research output. This is also something that’s worth thinking about. Publishing increases one’s academic reputability, and the scrutiny of peer review improves the reliability of one’s research. However, it is far more time-consuming than one might expect, because one has to tailor one’s research to the norms of the journal, and is especially slow if one is publishing within philosophy journals. (A paper of mine was under review for 10 months from one journal.) It also biases research towards ideas that are publishable, even if less important. So it’s a difficult issue.
For reasons of time, GiveWell don’t publish at all (but the resulting lack of peer review is something I’ve raised as a concern about their research); whereas, in order to boost reputation, MIRI are aiming to publish. At the moment, publishing isn’t a high priority for us, but we do some. I’ve published the central argument in favour of earning to give (it’s forthcoming in Ethical Theory and Moral Practice, available here), and I’m planning to write a book on effective altruism over the next year, from which I might publish a few articles. But beyond that, we’d rather focus on getting the ideas right. However, that’s something we could easily be mistaken about, and is worthy of discussion.
With these points noted, I’ll move on to the mistakes made within the post.
Some misleading aspects of the post
Factual Errors I mention these in my other comment on this post.
One other thing to note is that the 80k pledge was never focused on global poverty. The previous declaration was: I declare that I aim to pursue a career as an effective altruist.
This means that I intend to: (i) Devote a significant proportion of my time or resources to helping others. (ii) Use the time or resources I give as effectively as possible in helping others. (iii) Choose my career based at least in part on how it enables me to further my altruistic aims. And prior to that the declaration was: I pledge that, over my lifetime, I will dedicate 10% of my time or money (or any combination of the two) to those causes that I believe will do the most good with the resources I give them. I understand that it is difficult to know the best way of doing good in the world, and so I will choose those cause(s) on the basis of the best evidence that is available to me at the time. Further, I will deliberately pursue a career that will considerably improve my ability to further those causes I believe to be best.
The new declaration is: “I intend, at least in part, to use my career in an effective way to make the world a better place.” More discussion on these changes later.
Misleading statements “In recent conversation with Will Crouch”… “In conversation with Will Crouch”… “these discussions” I mention this in my other post but it’s worth repeating. Though your post suggests that we had at least two one-on-one conversations, this never happened. We spoke only during a question-and-answer session after a short talk I gave.
“There wasn’t any particular defence of the choice of wording [of the 80k declaration of intent] or any indication that there had been deep thought about precisely what that pledge should constitute.” This is technically true. However, it’s misleading insofar as I wasn’t asked why the declaration of intent was changed, nor was I asked how much time had gone into thinking about revising the declaration of intent.
“The key argument in favour of donating money to CEA which was presented by Will was that by donating $1 to CEA you produce more than $1 in donations to the most effective charities. We present some apparent difficulties with this remaining true on the margin.” This suggests that your post was primarily about difficulties with inferring marginal cost-effectiveness from past average cost-effectiveness. I think that that’s a very important topic (hey, maybe 99.9% of the value of CEA comes from me! In which case marginal cost-effectiveness would be much lower than past average cost-effectiveness), but as far as I can tell in your post you don’t address that issue anywhere.
Hi Jonathan,
First off, thanks for putting so much time into writing this extensive list of questions and doubts you have about CEA. Unlike for-profit activities, we don’t have immediate feedback effects telling us when we’re doing well and when we’re doing badly, so criticism is an important countermeasure to make sure we do things as well as possible. We therefore really welcome people taking a critical eye to our activities.
As the person who wrote the original CEA material here on LessWrong, and the person who you mention above, I feel I should be the one to collate a response to your questions. However, because of other commitments (managing; fundraising; writing my first piece for a magazine column), it will be a few days before I can get this to you in a form I’d feel happy with. I hope that’s ok.
Before then I’ll just mention a few things in order to make things a bit clearer to the audience.
In what you wrote a couple of comments made it sound as if you’d had an in-depth conversation with me on these issues; whereas really the context of the only exchange we’ve had is my giving a short talk to a group of about 15 people, of very varied backgrounds. You asked a few questions and there was discussion afterwards, but this must have only taken up about 10-15 minutes of time. Though I would very much like to, I haven’t ever spoken with you or Alexey one-on-one.
Similarly, in your response to Luke you say that Adam works full-time at CEA. I think there’s some disagreement between the two of you on the extent to which he had signed off on the content. But, at any rate, it’s worth noting that Adam is an intern at CEA. This means he does contribute a full working week for CEA, but he is not an employee. He’s therefore not the person to go when it comes to high-level evaluation of CEA.
You mention an internal estimate of £1700 for the value of a new pledge. None of us are familiar with this figure, and we’re confused about where it could have come from.
You suggest that CEA has ~4000 people on its mailing lists. The correct figure is less than half that (unless you include TLYCS, which you might have been thinking of, which does have in excess of 4000 on its mailing list).
You estimate GWWC’s research capacity at 6 staff for last year. This is actually more than an order of magnitude higher than the true figure. In fact, the average number of paid employees (full-time equivalent) we have had working on all aspects of 80,000 Hours and Giving What We Can over the last six months is only 3.7.
As a more general point, I think we should also be careful to distinguish whether CEA has acted optimally in terms of utility-maximization (to which the answer is certainly not), and whether it gets a return on investment which is better than 1:1.
In my follow-up comment, I’ll talk about some of the many concerns you’ve raised that we share, and the issues over which we might be making big mistakes. I’ll also be able to give a bit more background about our activities, and I’ll be able to answer your questions. Thanks again for taking the time to comment.
Best Wishes,
Will
At the moment the best thing to do would be to link to each of the organisations’ websites individually.
Hey Rob, thanks for writing this, and sorry for the slow response. In brief, I think you do misunderstand my views, in ways that Buck, Ryan and Habryka point out. I’ll clarify a little more.
Some areas where the criticism seems reasonable:
I think it’s fair to say that I worded the compute governance sentence poorly, in ways Habryka clarified.
I’m somewhat sympathetic to the criticism that there was a “missing mood” (cf e.g. here and here), given that a lot of people won’t know my broader views. I’m very happy to say: “I definitely think it will be extremely valuable to have the option to slow down AI development in the future,” as well as “the current situation is f-ing crazy”. (Though there was also a further vibe on twitter of “we should be uniting rather than disagreeing” which I think is a bad road to go down.)
Now, clarifying my position:
Here’s what I take IABI to be arguing (written by GPT5-Pro, on the basis of a pdf, in an attempt not to infuse my biases):
And what readers will think the book is about (again written by GPT5-Pro):
The core message of the book is not saying merely “AI x-risk is worryingly high” or “stopping or slowing AI development would be one good strategy among many.” I wouldn’t disagree with the former at all, and the latter disagreement would be more about the details.
Here’s a different perspective:
This is my overall strategic picture. If the book had argued for that (or even just the “kitchen sink” approach part) then I might have disagreed with the arguments, but I wouldn’t feel, “man, people will come away from this with a bad strategic picture”.
(I think the whole strategic picture would include:
But I think that’s fine not to discuss in a book focused on AI takeover risk.)
This is also the broad strategic picture, as I understand it, of e.g. Carl, Paul, Ryan, Buck. It’s true that I’m more optimistic than they are (on the 80k podcast I say 1-10% range for AI x-risk, though it depends on what exactly you mean by that) but I don’t feel deep worldview disagreement with them.
With that in mind, some reasons why I think the promotion of the Y&S view could be meaningfully bad:
If it means more people don’t pursue the better strategy of focusing on the easier wins.
Or they end up making the wrong tradeoffs. (e.g. intense centralisation of AI development in a way that makes misaligned human takeover risk more likely)
Or people might lapse into defeatism: “Ok we’re doomed, then: a decades-long international ban will never happen, so it’s pointless to work on AI x-risk.” (We already see this reaction to climate change, given doomerist messaging there. To be clear, I don’t think that sort of effect should be a reason for being misleading about one’s views.)
Overall, I feel pretty agnostic on whether Y&S shouting their message is on net good for the world.
I think I’m particularly triggered by all this because of a conversation I had last year with someone who takes AI takeover risk very seriously and could double AI safety philanthropy if they wanted to. I was arguing they should start funding AI safety, but the conversation was a total misfire because they conflated “AI safety” with “stop AI development”: their view was that that will never happen, and they were actively annoyed that they were hearing what they considered to be such a dumb idea. My guess was that EY’s TIME article was a big factor there.
Then, just to be clear, here are some cases where you misunderstand me, just focusing on the most-severe misunderstandings:
I really don’t think that!
I vibeswise disagree, because I expect massive acceleration and I think that’s *the* key challenge: See e.g. PrepIE, 80k podcast.
But there is a grain of truth in that my best guess is a more muted software-only intelligence explosion than some others predict. E.g. a best guess where, once AI fully automates AI R&D, we get 3-5 years of progress in 1 year (at current rates), rather than 10+ years’ worth, or rather than godlike superintelligence. This is the best analysis I know of on the topic. This might well be the cause of much of the difference in optimism between me and e.g. Carl.
(Note I still take the much larger software explosions very seriously (e.g. 10%-20% probability). And I could totally change my mind on this — the issue feels very live and open to me.)
Definitely disagree with this one! In general, society having more options and levers just seems great to me.
Definitely disagree!
Like, my whole bag is that I expect us to fuck up the future even if alignment is fine!! (e.g. Better Futures)
Definitely disagree! From my POV, it’s the IABI perspective that is closer to putting all the eggs in one basket, rather than advocating for the kitchen sink approach.
But “10% chance of ruin” is not what EY&NS, or the book, is arguing for, and isn’t what I was arguing against. (You could logically have the view of “10% chance of ruin and the only viable way to bring that down is a global moratorium”, but I don’t know anyone who has that view.)
Also not true, though I am more optimistic than many on the takeover side of things.
Also not true—e.g. I write here about the need to slow the intelligence explosion.
There’s a grain of truth in that I’m pretty agnostic on whether speeding up or slowing down AI development right now is good or bad. I flip-flop on it, but I currently lean towards thinking speed up at the moment is mildly good, for a few reasons: it stretches out the IE by bringing it forwards, means there’s more of a compute constraint and so the software-only IE doesn’t go as far, and means society wakes up earlier, giving more time to invest more in alignment of more-powerful AI.
(I think if we’d gotten to human-level algorithmic efficiency at the Dartmouth conference, that would have been good, as compute build-out is intrinsically slower and more controllable than software progress (until we get nanotech). And if we’d scaled up compute + AI to 10% of the global economy decades ago, and maintained it at that level, that also would have been good, as then the frontier pace would be at the rate of compute-constrained algorithmic progress, rather than the rate we’re getting at the moment from both algorithmic progress AND compute scale-up.)
In general, I think that how the IE happens and is governed is a much bigger deal than when it happens.
Which Will-version are you thinking of? Even in DGB I wrote about preventing near-term catastrophes as a top cause area.
Again really not intended! I think I’ve been clear about my views elsewhere (see previous links).
Ok, that’s all just spelling out my views. Going back, briefly, to the review. I said I was “disappointed” in the book — that was mainly because I thought that this was E&Y’s chance to give the strongest version of their arguments (though I understood they’d be simplified or streamlined), and the arguments I read were worse than I expected (even though I didn’t expect to find them terribly convincing).
Regarding your object-level responses to my arguments — I don’t think any of them really support the idea that alignment is so hard that AI takeover x-risk is overwhelmingly likely, or that the only viable response is to delay AI development by decades. E.g.
But if it’s a matter of “insufficiency”, the question is how one can be so confident that any work we do now (including with ~AGI assistance, including if we’ve bought extra time via control measures and/or deals with misaligned ~AGIs) is insufficient, such that the only thing that makes a meaningful difference to x-risk, even in expectation, is a global moratorium. And I’m still not seeing the case for that.
(I think I’m unlikely to respond further, but thanks again for the engagement.)