EA started off with global health and ended up pivoting hard to AI safety, AI governance, etc. You can think of this as “we started with one cause area and we found another using the same core assumptions” but another way to think about it is “the worldview which generated ‘work on global health’ was wrong about some crucial things”, and the ideology hasn’t been adequately refactored to take those things out.
Some of them include:
Focusing on helping the worst-off instead of empowering the best
Being very scared of political controversy (e.g. not being willing to discuss the real causes of African poverty)
Not deeply believing in technological progress (e.g. prioritizing bed nets over vaccines). Related to point 1.
Trusting the international order to mostly have its shit together (related to point 2)
You can partially learn these lessons from within the EA framework but it’s very unnatural and you won’t learn them well enough. E.g. now EAs are pivoting to politics but again they’re flinching away from anything remotely controversial and so are basically just propping up existing elites.
On a deeper ideological level a lot of this is downstream of utilitarianism/consequentialism being wrong. Again hard to compress but a few quick points:
Utilitarianism primarily treats individuals as the relevant actors. But the whole point of ethics is to allow us to (safely) act as coherent coordinated groups instead.
Utilitarianism exacerbates the natural human tendency to flinch away from fears. It’s far easier to soothe your ego by finding a rationalization for having caused good consequences, than to self-deceive about whether you’re a paragon of courage or honesty.
On a technical level utilitarianism is extremely non-incentive-compatible. You’re basically asking for people to set up systems to exploit you, as long as they do so with enough levels of indirection that you can fool yourself into thinking you’re doing good (e.g. a dictator keeping a population as poor as possible, in plausibly-deniable ways, so utilitarians will keep sending aid which they can steal).
A lot of utilitarians will say “whether or not our strategy is bad, consequences are still the only thing that ultimately matter”. But this is like a Marxist saying to an economist “whether or not our strategy is bad, liberating the workers is the only thing that ultimately matters“ and then using that as an excuse to not learn economics. There *are* deep principles of (internal and external) cooperation, but utilitarianism is very effective in making people look away from them and towards power-seeking strategies.
A second tweet, in response to someone disagreeing with the Marxism analogy because many utilitarians follow principles too:
I think “maximize expected utility while obeying some constraints” looks very different from actually taking non-consequentialist decision procedures seriously.
In principle the utility-maximizing decision procedure might not involve thinking about “impact” at all.
And this is not even an insane hypothetical, IMO thinking about impact is pretty corrosive to one’s ability to do excellent research for example.
It’s hard to engage too deeply here because I think the notion of a “utility-maximizing decision procedure” is very underdefined on an individual level, and most of the action is in correlating one’s decisions with others. But my meta-level point is that it’s these kinds of complexities which utilitarians tend to brush under the rug by focusing their intellectual energy on criteria of rightness and then adding on some principles almost as an afterthought.
It’s far easier to soothe your ego by finding a rationalization for having caused good consequences, than to self-deceive about whether you’re a paragon of courage or honesty.
It seems like the opposite to me, that it is extremely easy to self-deceive about whether you’re a virtuous person. In fact this seems like a quintessential example of self-deception that one encounters fairly commonly.
It is also quite easy to self-deceive about the consequences of ones actions, I agree, but in that case there is at least some empirical fact about the world which could drag you back from your self-deception of you were open to it. In contrast, it seems to me like if you self-deceive hard enough you can always view yourself as a paragon of virtue essentially regardless of how virtuous you actually are and you can always come up with rationalizations for your own actions. If you care about consequences your performance on those consequences can be contradicted empirically, but there isn’t a similar way of contradicting your virtue if you are willing to twist your impressions of what is virtuous enough.
I find this an interesting line of criticism, it is essentially pointing at the difficulty of finding good evidence and evaluating yourself on that evidence and making a disagreement about how easy it is.
I would like to bring in a perspective of more first principles modelling of how quickly you incorporate evidence pointing against the way you’re thinking.
One thing is the amount of disconfirming evidence you look for. Another thing is your ability to bring that information into your worldview, your openness to being wrong. Thirdly we might also mention is the speed of the feedback, how long time does it take for you to get feedback.
I think you’re saying that when you go into virtue ethics we often find failures in bringing in disconcerting information into a worldview. I don’t think this has to be the case personally as I do think there are ways to actually get feedback on whether you’re acting in a way that is aligned with your virtues, by mentioning examples of them and then having anonymous people give feedback or just normal reflection.
This is a lot easier to do if your loops are shorter which is the exact point that consequentalism and utilitarianism can fail on, it is a target that is quite far away and so the crispness and locality of the feedback is not high enough.
I think that virtue ethics outperforms consequentialism because it is better suited for bringing in information as well as for speed and crispness of that information. I personally think it is because it is a game theory optimal solution to consequentialism in environments where you have little information but that is probably beside the point.
This might just be a difference in terminology though? Would you agree with my 3 point characterisation above?
For clarity, I’m not trying to make the case for or against consequentialism/virtue ethics, I’m just trying to respond to the narrow point that I quoted. I don’t think people should choose an approach to ethical decision making based primarily on this one specific point.
That said, I take your central point to be that virtue is more direct or local compared to consequences that it is easier to have evidence on your virtues than the consequences of your actions.
My argument above is specifically about the robustness of these to self deception. Being more local and in a sense “personal” is what makes virtue more suscetible to self deception in my view. There can be things that a reasonable obsever might consider evidence about your virtue, but these are so closely tied to you personally that self deception will often be easy. It will be easy to dismiss people who decry your lack of virtue as unfair or bias or bad people themselves, precisely because it is the question of your virtue on the line!
In contrast, the evidence you get about the consequences of your actions may be harder to interpret in some cases because you have to analysze the causation and it could be noisier, but this also creates a seperation so that it is not as personal. It gives you the chance to admit that even if you had good intentions and high integrity things didn’t play out how you wanted in actual fact. For virtue ethics you can’t admit you were wrong without also admitting you had bad intentions or lacked integrity.
I think you can do virtue ethics and also work on your tendency to self deceive, but that doesn’t make it is robust if you are self deception (although the degree of delf seception could defintiely be relevant).
That is a fair point, since virtue is tied to your identity and self it is a lot easier to take things personally and therefore distort the truth.
A part of me is like “meh, skill issue, just get good at emotional intelligence and see through your self” but that is probably not a very valid solution at scale if I’m being honest.
There’s still something nice about it leading to repeated games and similar, something about how if you look at our past then cooperation arises from repeated games rather than individual games where you analyse things in detail. This is the specific point that Joshua Greene makes in his Moral Tribes book for example.
Maybe the core point here is not virtue versus utilitarian reasoning, it might more be about the ease of self-deception and how different time limits and how ways of evaluating your own outcomes and outputs should be done in a more impersonal way. Maybe one shouldn’t call this virtue ethics as it carries a large bag and camp, maybe heruistics ethics or something (though that feels stupid).
If you care about consequences your performance on those consequences can be contradicted empirically, but there isn’t a similar way of contradicting your virtue if you are willing to twist your impressions of what is virtuous enough.
If you care about the consequences of your crypto philanthropy strategy, your performance on those consequences can be contradicted empirically. But so what if the contradiction arrives in the form of your bankruptcy, along with blemishing the reputation of the movement you were trying to support?
“Basic virtue ethics” would probably prevent this (if not through by making you correct your strategy, at least it would light up more red flags in other people’s heads) (as would “non-naive consequentialism”). Of course, virtue ethics has its own failure modes. Virtue ethics and consequentialism have different failure profiles.
For clarity, I’m not trying to make the case for or against consequentialism/virtue ethics, I’m just trying to respond to the narrow point that I quoted. I don’t think people should choose an approach to ethical decision making based primarily on this one specific point.
If you care about the consequences of your crypto philanthropy strategy, your performance on those consequences can be contradicted empirically. But so what if the contradiction arrives in the form of your bankruptcy, along with blemishing the reputation of the movement you were trying to support?
I think this is consistent with my point. From what I can tell SBF continues to claim that he was acting with good intentions and high integrity, despite being convicted for fraud, which I think most people would reasonably assume demonstrates a strong lack of those characteristics. This seems like it might be a case of self deception about one’s own virtues. This is the kind of thing I meant when I said it seems like this is a quintisentially example of self deception. From what I can tell it is extremely common that people who aren’t virtuous still think of themselves as virtuous.
It seems to me like a lot of people involved in the SBF scandal admit that it was bad and that they made strategic mistakes by trusting SBF, but they often don’t say that this is related to virtue failures on their part, such as lacking integrity or honesty. In other words, they admit to their actions potentially having bad consequences as the result of evidence about those consequences, but don’t admit to these events shedding light on their virtues or character.
Focusing on helping the worst-off instead of empowering the best
I feel like this is getting at some directionally correct stuff but, feels off.
EA was the union of a few disparate groups, roughly encapsulated by:
Peter Singer / Giving What We Can folk
Early Givewell
Early LessWrong
There are other specific subgroups. But, my guess is early Givewell was most loadbearing in there ending up being an “EA” identity, in that there were concrete recommendations of what to do with the money that stood up to some scrutiny. Otherwise it’d have just been more “A”, or, “weird transhumanists with weird goals.”
Givewell started out with a wider variety of cause areas, including education in America. It just turned out that it seemed way more obviously cost effective to do specific global health interventions than to try to fix education in America. (I realize education in America isn’t particularly “empowering the best”, but, the flow towards “helping worst off” seems to me like it wasn’t actually the initial focus)
I agree some memeplex accreted around that, which had some of the properties you describe.
But meanwhile:
EA started off with global health and ended up pivoting hard to AI safety, AI governance, etc.
It seems off to say “started off in global health, and pivoted to AI”, when all the AI stuff was there from the beginning at the very first pre-EA-Global events, and just eventually became clear that it was real, and important. The worldview that generated AI was not (exactly) the same one generating global health, they were just two clusters of worldview that were in conversation with each other from the beginning.
It seems off to say “started off in global health, and pivoted to AI”, when all the AI stuff was there from the beginning at the very first pre-EA-Global events, and just eventually became clear that it was real, and important. The worldview that generated AI was not (exactly) the same one generating global health, they were just two clusters of worldview that were in conversation with each other from the beginning.
I agree with all the facts cited here, but I think it still understates the way that there was an intentional pivot.
The EA brand to broader world emphasized earning to give and effective global poverty charities in particular. That’s what most people who had heard of it associated with “effective altruism”. And most of the people who got involved before 2019 got involved with an EA bearing that brand.
I guess that in 2015, the average EAG-goer was mostly interested in GiveWell style effective charities, and gave a bit of difference to the more speculative x-risk stuff (because smart EAs seem to take it seriously), but mostly didn’t focus on it very much.
And while it’s true that AI risk was part of the discussion from the very beginning, there were explicit top-down pushes from the leadership to prioritize it and to give it more credibility.
(And more than that, I’m told that at least some of the leadership had the explicit strategy of building credibility and reputation with GiveWell-like stuff, and boosting the reputation of AI risk by association.)
Yep agree with all that. (I stand by my comment as mostly arguing directionally against Richard’s summary but seems fine to also argue directionally against mine)
The worldview that generated AI was not (exactly) the same one generating global health, they were just two clusters of worldview that were in conversation with each other from the beginning.
Yes, I agree; my point is that people with the global health worldview ended up being convinced of a bunch of the high-level conclusions of the rationalist worldview, but without updating much away from the generators of the global health worldview.
The terminology is a little tricky here because they’re so entangled but I think it’s reasonable to talk about “EA” as a cluster as opposed to “rationalism” as a cluster even though a lot of people are in both.
E.g. if AI weren’t a big deal then rationalists would probably be doing cryonics or solving aging or something. Whereas if EAs weren’t into AI they’d probably be doing global health, factory farming, etc.
E.g. if AI weren’t a big deal then rationalists would probably be doing cryonics or solving aging or something
Strong disagree. We could have done those things, but the rationality movement didn’t have enough motive force or coordination capacity to do much, beyond AI safety.
Yes, because it funneled all of its best and brightest into AI safety?
We might be evaluating the hypothetical at different points. I’m thinking of the movement coalescing around the sequences except the message underlying the sequence is “you should solve ageing” rather than “you should solve alignment”.
Interesting, I’d never explicitly considered that Peter Singer (you should expand your moral circle and do as much good as you can) and GiveWell (given that you want to do good, how to do it?) started as totally different memeplexes and only merged later on. It makes sense in retrospect.
In general, EA emerged as the convergence from 2008 to 2012 at least 4 distinct but overlapping proto-EA communities, in order of founding:
The Singularity Institute (now known as Machine Intelligence Research Institute; MIRI) and the “rationalist” discussion forum LessWrong, founded by Eliezer Yudkowsky and others in 2000 and 2006
GiveWell, founded by Holden Karnofsky and Elie Hassenfeld in 2007, and Good Ventures, founded by with Dustin Moskovitz and Cari Tuna in 2011, which partnered together in 2014 as GiveWell Labs (now Open Philanthropy)
Felicifia, created by Seth Baum, Ryan Carey, and Sasha Cooper in 2008 as a utilitarianism discussion forum, which is how I got involved as discussed above; these discussions largely moved to other venues such as Facebook in 2012, and Felicifia is no longer active.
Giving What We Can (2009) and 80,000 Hours (2011), founded by Will MacAskill and Toby Ord, philosophers at the University of Oxford, and the umbrella organization Centre for Effective Altruism; Will has written about the early history of EA on the TLYCS blog and the history of the term on the Effective Altruism Forum.
As the EA flag was being planted, there were many effectiveness-focused altruists who came out of the woodwork but did not have formal involvement with one of these 4 groups, especially people inspired by the famous philosopher and utilitarian Peter Singer, particularly his essay “Famine, Affluence, and Morality” (1972)3 and book Animal Liberation (1975). Many were also involved in the evidence-based “randomista” movement in economic development, emphasizing evidence-based strategies to help the world’s poorest people, including academic research on this topic since the 1990s, especially IPA (2002) and JPAL (2003). Additionally, there were other email lists and community forms related to EA such as SL4 on the possibility of a technological singularity, as well as personal blogs, such as Brian Tomasik’s. Some were inspired by famous altruists such as Zell Kravinsky. I met many people in the early days of EA who said they had been thinking along EA lines for years and were so thrilled to find a community centered on this mindset. This is less common in 2022 because the movement is so visible and established that people run across it quickly once they start thinking in these ways.
The need to decide upon a name came from two sources:
First, the Giving What We Can (GWWC) community was growing. 80,000 Hours (80k) had soft-launched in February 2011, moving the focus in Oxford away from just charity and onto ethical life-optimisation more generally. There was also a growing realization among the GWWC and 80k Directors that the best thing for us each to be doing was to encourage more people to use their life to do good as effectively as possible (which is now usually called ‘movement-building’).
Second, GWWC and 80k were planning to incorporate as a charity under an ‘umbrella’ name, so that we could take paid staff (decided approx. Aug 2011; I was Managing Director of GWWC at the time and was pushing for this, with Michelle Hutchinson and Holly Morgan as the first planned staff members). So we needed a name for that umbrella organization (the working title was ‘High Impact Alliance’). We were also just starting to realize the importance of good marketing, and therefore willing to put more time into things like choice of name.
At the time, there were a host of related terms: on 12 March 2012 Jeff Kaufman posted on this, listing ‘smart giving’, ‘efficient charity’, ‘optimal philanthropy’, among others. Most of the terms these referred to charity specifically. The one term that was commonly used to refer to people who were trying to use their lives to do good effectively was the tongue-in-cheek ‘super-hardcore do-gooder’. It was pretty clear we needed a new name! I summarized this in an email to the 80k team (then the ‘High Impact Careers’ team) on 13 October 2011:
We need a name for “someone who pursues a high impact lifestyle”. This has been such an obstacle in the utilitarianesque community - ‘do-gooder’ is the current term, and it sucks.”
What happened, then, is that there was a period of brainstorming—combining different terms like ‘effective’, ‘efficient’, ‘rational’ with ‘altruism’, ‘benevolence’, ‘charity’. Then the Directors of GWWC and 80k decided, in November 2011, to aggregate everyone’s views and make a final decision by vote. This vote would decide both the name of the type of person we wanted to refer to, and for the name of the organization we were setting up. …
And then the vote came down to this shortlist (emphasis mine):
Rational Altruist Community RAC
Effective Utilitarian Community EUC
Evidence-based Charity Association ECA
Alliance for Rational Compassion ARC
Evidence-based Philanthropy Association EPA
High Impact Alliance HIA
Association for Evidence-Based Altruism AEA
Optimal Altruism Network OAN
High Impact Altruist Network HIAN
Rational Altruist Network RAN
Association of Optimal Altruists AON
Centre for Effective Altruism CEA
Centre for Rational Altruism CRA
Big Visions Network BVN
Optimal Altruists Forum OAF
… In the vote, CEA won, by quite a clear margin. Different people had been pushing for different names. I remember that Michelle preferred “Rational Altruism”, the Leverage folks preferred “Strategic Altruism,” and I was pushing for ’”Effective Altruism”. But no-one had terribly strong views, so everyone was happy to go with the name we voted on. …
We hadn’t planned ‘effective altruism’ to take off in the way that it did. ‘Centre for Effective Altruism’ was intended not to have a public presence at all, and just be a legal entity. I had thought that effective altruism was too abstract an idea for it to really catch on, and had a disagreement with Mark Lee and Geoff Anders about this. Time proved them correct on that point!
imo a larger one is something like not rooting the foundations in “build your own models of the world so that you contain within you a stack trace of why you’re doing what you’re doing” + “be willing to be challenges and update major beliefs based on deep-in-the-weeds technical arguments, and do so from a highly truth-seeking stance which knows what it feels like to actually understand something not just have an opinion”.
Lack of this is fineish in global health, but in AI Safety generates a crop of people with only surface deferral flavor understanding of the issues, which is insufficient to orient in a much less straightforward technical domain.
hmm, I like the diagnosis of issues with the EA worldview, but I don’t really buy that they’re downstream of issues with consequentialism and utilitarianism itself.
I would say it’s more like: Effective Altruism has historically embraced a certain flavor of utilitarianism and naive consequentialism that attempts to be compatible with pre-existing vibes and (to some degree) mainstream politics. Concretely, EAs are (to their credit) willing to bite some strange bullets and then act on their conclusions, and are also generally pro-market compared to mainstream Democratic politics. But they’re still very, very Blue Tribe-coded culture-wise, and this causes them to deviate from actually-correct versions of consequentialism and utilitarianism in predictable directions.
Or: in my view, “Utilitarianism and non-naive consequentialism with guardrails” is pretty close to correct philosophy for humans; the issue is that the EA worldview systematically selects for the wrong guardrails[1]. But better ones are available; for example Eliezer wrote this nearly 20 years ago: Ends Don’t Justify Means (Among Humans)
I’d be interested in hearing what kind of criticism you have of the posts in that sequence, and whether your issues with EA are more about a lack of emphasis and embrace of some of those principles, or that the ideas in that sequence are incomplete or even fundamentally mistaken or leading people astray.
Not deeply believing in technological progress (e.g. prioritizing bed nets over vaccines).
I think this is wrong about the history.
I was recently watched this Q&A with Holden in 2013. The picture I got is that GiveWell was focusing on “proven interventions” (e.g. bednets) over “speculative interventions” (e.g. biomedical research), because proven interventions were easier to evaluate.[1] He says he thinks speculative/high-risk stuff is probably better, and GiveWell Labs (i.e. Open Philanthropy, i.e. Coefficient Giving) was the pivot toward finding those opportunities.
Supporting quotations
“We really focused on what I’d call traditional GiveWell, which is looking for charities that are proven and cost effective and scalable… That’s why we picked it, is because we thought we’d be able to get somewhere on that.”
“We kind of bit off this chunk at the beginning of proven effective scalable… that was because it had a shorter feedback loop. Do this analysis, people would check the analysis… we would learn.”
“Proven can be easy to see, it also does tend to attract money from other funders… It allows a level of accountability that you don’t get with other things, it allows a level of honesty and objectivity and transparency you don’t get with other things.”
“You get much better information when you’re taken more seriously, and you’re taken more seriously when there’s more money behind you.”
“I feel like by running an organization and by operating within that framework, I’ve just gotten so much better… knowing what’s going to be accomplishable, knowing what sort of person to listen to.”
“We don’t believe, and didn’t necessarily — didn’t really ever believe — that this kind of charity is the only way to have a positive impact or the best way to have a positive impact.”
“I do think that by taking away the restrictions, I believe we’re going to find much better giving opportunities… Conventional wisdom in philanthropy is that speculative and risky is where all the action is.”
“The ROI on medical research is probably really good… Aging is one of the things that I would look at as being promising.”
Fun fact: Holden mentions that his “informal scientific adviser” for exploring biomedical research as a cause area ia a biology postdoc at Stanford called Dario Amodei.
My own take re rationalization/motivated reasoning is that at the end of the day, no form of ethics can meaningfully slow it down if the person either can’t credibly commit to their future selves, or simply isn’t bound/want to follow ethical rules, so the motivated reasoning critique isn’t EA specific, but rather shows 2 things:
People are more selfish than they think themselves to be, and care less about virtues, so motivated reasoning is very easy.
We can’t credibly commit our future selves to do certain things, especially over long timeframes, and even when people do care about virtues, motivated reasoning still harms their thinking.
Motivated reasoning IMO is a pretty deep-seated problem within our own brains, and is probably unsolvable in the near term.
To clarify, I think your criticism of utilitarianism/consequentialism is of a naive form of it that only looks at first-order effects. Not ‘proper’ utilitarianism. But yes no doubt many are naive like this, and it’s v hard to evaluate second- and higher-order effects (such as exploitation and coordination).
Also, this kind of naivety is particularly common on the left.
However, I feel like traditional virtue-ethical notions e.g. “courage,” “integrity,” have the same adversarial Goodharting problem (3) as in your critique of utilitarianism. “Loyalty is when you obey the Master,” “courage is when you go to war against the Enemy without fearing death,” etc. I suspect utilitarianism is maybe only barely worse than other ethical systems in regards to (2). It’s worthwhile to compare EA against regular humans. I don’t really understand either.
I think of virtue ethics as something like “being a healthy and functional cog in the Humanity machine,” where the Humanity machine is ultimately utilitarian.
Further, I think a lot of arguments for utilitarian behaviors “pass through” to virtue ethics insofar as we think that traits like “ambition” and “scope sensitivity” are virtues. I think they are: seeing their characteristic absence has a similar sliminess to seeing a cowardly or slavish person.
(Sometimes it’s just a lack of underlying numeracy, which I would not consider a lack of virtue but rather of education. I spoke to a man who said he wouldn’t suck a dick for a billion dollars, because he just couldn’t. I walked him through the size of a billion, and he changed his mind.)
A response to someone asking about my criticisms of EA (crossposted from twitter):
EA started off with global health and ended up pivoting hard to AI safety, AI governance, etc. You can think of this as “we started with one cause area and we found another using the same core assumptions” but another way to think about it is “the worldview which generated ‘work on global health’ was wrong about some crucial things”, and the ideology hasn’t been adequately refactored to take those things out.
Some of them include:
Focusing on helping the worst-off instead of empowering the best
Being very scared of political controversy (e.g. not being willing to discuss the real causes of African poverty)
Not deeply believing in technological progress (e.g. prioritizing bed nets over vaccines). Related to point 1.
Trusting the international order to mostly have its shit together (related to point 2)
You can partially learn these lessons from within the EA framework but it’s very unnatural and you won’t learn them well enough. E.g. now EAs are pivoting to politics but again they’re flinching away from anything remotely controversial and so are basically just propping up existing elites.
On a deeper ideological level a lot of this is downstream of utilitarianism/consequentialism being wrong. Again hard to compress but a few quick points:
Utilitarianism primarily treats individuals as the relevant actors. But the whole point of ethics is to allow us to (safely) act as coherent coordinated groups instead.
Utilitarianism exacerbates the natural human tendency to flinch away from fears. It’s far easier to soothe your ego by finding a rationalization for having caused good consequences, than to self-deceive about whether you’re a paragon of courage or honesty.
On a technical level utilitarianism is extremely non-incentive-compatible. You’re basically asking for people to set up systems to exploit you, as long as they do so with enough levels of indirection that you can fool yourself into thinking you’re doing good (e.g. a dictator keeping a population as poor as possible, in plausibly-deniable ways, so utilitarians will keep sending aid which they can steal).
A lot of utilitarians will say “whether or not our strategy is bad, consequences are still the only thing that ultimately matter”. But this is like a Marxist saying to an economist “whether or not our strategy is bad, liberating the workers is the only thing that ultimately matters“ and then using that as an excuse to not learn economics. There *are* deep principles of (internal and external) cooperation, but utilitarianism is very effective in making people look away from them and towards power-seeking strategies.
A second tweet, in response to someone disagreeing with the Marxism analogy because many utilitarians follow principles too:
I think “maximize expected utility while obeying some constraints” looks very different from actually taking non-consequentialist decision procedures seriously.
In principle the utility-maximizing decision procedure might not involve thinking about “impact” at all.
And this is not even an insane hypothetical, IMO thinking about impact is pretty corrosive to one’s ability to do excellent research for example.
It’s hard to engage too deeply here because I think the notion of a “utility-maximizing decision procedure” is very underdefined on an individual level, and most of the action is in correlating one’s decisions with others. But my meta-level point is that it’s these kinds of complexities which utilitarians tend to brush under the rug by focusing their intellectual energy on criteria of rightness and then adding on some principles almost as an afterthought.
It seems like the opposite to me, that it is extremely easy to self-deceive about whether you’re a virtuous person. In fact this seems like a quintessential example of self-deception that one encounters fairly commonly.
It is also quite easy to self-deceive about the consequences of ones actions, I agree, but in that case there is at least some empirical fact about the world which could drag you back from your self-deception of you were open to it. In contrast, it seems to me like if you self-deceive hard enough you can always view yourself as a paragon of virtue essentially regardless of how virtuous you actually are and you can always come up with rationalizations for your own actions. If you care about consequences your performance on those consequences can be contradicted empirically, but there isn’t a similar way of contradicting your virtue if you are willing to twist your impressions of what is virtuous enough.
I find this an interesting line of criticism, it is essentially pointing at the difficulty of finding good evidence and evaluating yourself on that evidence and making a disagreement about how easy it is.
I would like to bring in a perspective of more first principles modelling of how quickly you incorporate evidence pointing against the way you’re thinking.
One thing is the amount of disconfirming evidence you look for. Another thing is your ability to bring that information into your worldview, your openness to being wrong. Thirdly we might also mention is the speed of the feedback, how long time does it take for you to get feedback.
I think you’re saying that when you go into virtue ethics we often find failures in bringing in disconcerting information into a worldview. I don’t think this has to be the case personally as I do think there are ways to actually get feedback on whether you’re acting in a way that is aligned with your virtues, by mentioning examples of them and then having anonymous people give feedback or just normal reflection.
This is a lot easier to do if your loops are shorter which is the exact point that consequentalism and utilitarianism can fail on, it is a target that is quite far away and so the crispness and locality of the feedback is not high enough.
I think that virtue ethics outperforms consequentialism because it is better suited for bringing in information as well as for speed and crispness of that information. I personally think it is because it is a game theory optimal solution to consequentialism in environments where you have little information but that is probably beside the point.
This might just be a difference in terminology though? Would you agree with my 3 point characterisation above?
For clarity, I’m not trying to make the case for or against consequentialism/virtue ethics, I’m just trying to respond to the narrow point that I quoted. I don’t think people should choose an approach to ethical decision making based primarily on this one specific point.
That said, I take your central point to be that virtue is more direct or local compared to consequences that it is easier to have evidence on your virtues than the consequences of your actions.
My argument above is specifically about the robustness of these to self deception. Being more local and in a sense “personal” is what makes virtue more suscetible to self deception in my view. There can be things that a reasonable obsever might consider evidence about your virtue, but these are so closely tied to you personally that self deception will often be easy. It will be easy to dismiss people who decry your lack of virtue as unfair or bias or bad people themselves, precisely because it is the question of your virtue on the line!
In contrast, the evidence you get about the consequences of your actions may be harder to interpret in some cases because you have to analysze the causation and it could be noisier, but this also creates a seperation so that it is not as personal. It gives you the chance to admit that even if you had good intentions and high integrity things didn’t play out how you wanted in actual fact. For virtue ethics you can’t admit you were wrong without also admitting you had bad intentions or lacked integrity.
I think you can do virtue ethics and also work on your tendency to self deceive, but that doesn’t make it is robust if you are self deception (although the degree of delf seception could defintiely be relevant).
That is a fair point, since virtue is tied to your identity and self it is a lot easier to take things personally and therefore distort the truth.
A part of me is like “meh, skill issue, just get good at emotional intelligence and see through your self” but that is probably not a very valid solution at scale if I’m being honest.
There’s still something nice about it leading to repeated games and similar, something about how if you look at our past then cooperation arises from repeated games rather than individual games where you analyse things in detail. This is the specific point that Joshua Greene makes in his Moral Tribes book for example.
Maybe the core point here is not virtue versus utilitarian reasoning, it might more be about the ease of self-deception and how different time limits and how ways of evaluating your own outcomes and outputs should be done in a more impersonal way. Maybe one shouldn’t call this virtue ethics as it carries a large bag and camp, maybe heruistics ethics or something (though that feels stupid).
If you care about the consequences of your crypto philanthropy strategy, your performance on those consequences can be contradicted empirically. But so what if the contradiction arrives in the form of your bankruptcy, along with blemishing the reputation of the movement you were trying to support?
“Basic virtue ethics” would probably prevent this (if not through by making you correct your strategy, at least it would light up more red flags in other people’s heads) (as would “non-naive consequentialism”). Of course, virtue ethics has its own failure modes. Virtue ethics and consequentialism have different failure profiles.
For clarity, I’m not trying to make the case for or against consequentialism/virtue ethics, I’m just trying to respond to the narrow point that I quoted. I don’t think people should choose an approach to ethical decision making based primarily on this one specific point.
I think this is consistent with my point. From what I can tell SBF continues to claim that he was acting with good intentions and high integrity, despite being convicted for fraud, which I think most people would reasonably assume demonstrates a strong lack of those characteristics. This seems like it might be a case of self deception about one’s own virtues. This is the kind of thing I meant when I said it seems like this is a quintisentially example of self deception. From what I can tell it is extremely common that people who aren’t virtuous still think of themselves as virtuous.
It seems to me like a lot of people involved in the SBF scandal admit that it was bad and that they made strategic mistakes by trusting SBF, but they often don’t say that this is related to virtue failures on their part, such as lacking integrity or honesty. In other words, they admit to their actions potentially having bad consequences as the result of evidence about those consequences, but don’t admit to these events shedding light on their virtues or character.
I feel like this is getting at some directionally correct stuff but, feels off.
EA was the union of a few disparate groups, roughly encapsulated by:
Peter Singer / Giving What We Can folk
Early Givewell
Early LessWrong
There are other specific subgroups. But, my guess is early Givewell was most loadbearing in there ending up being an “EA” identity, in that there were concrete recommendations of what to do with the money that stood up to some scrutiny. Otherwise it’d have just been more “A”, or, “weird transhumanists with weird goals.”
Givewell started out with a wider variety of cause areas, including education in America. It just turned out that it seemed way more obviously cost effective to do specific global health interventions than to try to fix education in America. (I realize education in America isn’t particularly “empowering the best”, but, the flow towards “helping worst off” seems to me like it wasn’t actually the initial focus)
I agree some memeplex accreted around that, which had some of the properties you describe.
But meanwhile:
It seems off to say “started off in global health, and pivoted to AI”, when all the AI stuff was there from the beginning at the very first pre-EA-Global events, and just eventually became clear that it was real, and important. The worldview that generated AI was not (exactly) the same one generating global health, they were just two clusters of worldview that were in conversation with each other from the beginning.
I agree with all the facts cited here, but I think it still understates the way that there was an intentional pivot.
The EA brand to broader world emphasized earning to give and effective global poverty charities in particular. That’s what most people who had heard of it associated with “effective altruism”. And most of the people who got involved before 2019 got involved with an EA bearing that brand.
I guess that in 2015, the average EAG-goer was mostly interested in GiveWell style effective charities, and gave a bit of difference to the more speculative x-risk stuff (because smart EAs seem to take it seriously), but mostly didn’t focus on it very much.
And while it’s true that AI risk was part of the discussion from the very beginning, there were explicit top-down pushes from the leadership to prioritize it and to give it more credibility.
(And more than that, I’m told that at least some of the leadership had the explicit strategy of building credibility and reputation with GiveWell-like stuff, and boosting the reputation of AI risk by association.)
Yep agree with all that. (I stand by my comment as mostly arguing directionally against Richard’s summary but seems fine to also argue directionally against mine)
Yes, I agree; my point is that people with the global health worldview ended up being convinced of a bunch of the high-level conclusions of the rationalist worldview, but without updating much away from the generators of the global health worldview.
The terminology is a little tricky here because they’re so entangled but I think it’s reasonable to talk about “EA” as a cluster as opposed to “rationalism” as a cluster even though a lot of people are in both.
E.g. if AI weren’t a big deal then rationalists would probably be doing cryonics or solving aging or something. Whereas if EAs weren’t into AI they’d probably be doing global health, factory farming, etc.
Strong disagree. We could have done those things, but the rationality movement didn’t have enough motive force or coordination capacity to do much, beyond AI safety.
Yes, because it funneled all of its best and brightest into AI safety?
We might be evaluating the hypothetical at different points. I’m thinking of the movement coalescing around the sequences except the message underlying the sequence is “you should solve ageing” rather than “you should solve alignment”.
Maybe I’m missing something. Why are you comparing to the that hypothetical world?
Interesting, I’d never explicitly considered that Peter Singer (you should expand your moral circle and do as much good as you can) and GiveWell (given that you want to do good, how to do it?) started as totally different memeplexes and only merged later on. It makes sense in retrospect.
To add to your point, Jacy Reese Anthis in Some Early History of Effective Altruism wrote
On the history of the term “effective altruism”, Will MacAskill in 2014 dug through old emails and came up with the following stylised summary:
And then the vote came down to this shortlist (emphasis mine):
So predictably you have folks arguing e.g. Effective altruism is no longer the right name for the movement and so on.
imo a larger one is something like not rooting the foundations in “build your own models of the world so that you contain within you a stack trace of why you’re doing what you’re doing” + “be willing to be challenges and update major beliefs based on deep-in-the-weeds technical arguments, and do so from a highly truth-seeking stance which knows what it feels like to actually understand something not just have an opinion”.
Lack of this is fineish in global health, but in AI Safety generates a crop of people with only surface deferral flavor understanding of the issues, which is insufficient to orient in a much less straightforward technical domain.
hmm, I like the diagnosis of issues with the EA worldview, but I don’t really buy that they’re downstream of issues with consequentialism and utilitarianism itself.
I would say it’s more like: Effective Altruism has historically embraced a certain flavor of utilitarianism and naive consequentialism that attempts to be compatible with pre-existing vibes and (to some degree) mainstream politics. Concretely, EAs are (to their credit) willing to bite some strange bullets and then act on their conclusions, and are also generally pro-market compared to mainstream Democratic politics. But they’re still very, very Blue Tribe-coded culture-wise, and this causes them to deviate from actually-correct versions of consequentialism and utilitarianism in predictable directions.
Or: in my view, “Utilitarianism and non-naive consequentialism with guardrails” is pretty close to correct philosophy for humans; the issue is that the EA worldview systematically selects for the wrong guardrails[1]. But better ones are available; for example Eliezer wrote this nearly 20 years ago: Ends Don’t Justify Means (Among Humans)
I’d be interested in hearing what kind of criticism you have of the posts in that sequence, and whether your issues with EA are more about a lack of emphasis and embrace of some of those principles, or that the ideas in that sequence are incomplete or even fundamentally mistaken or leading people astray.
ETA: And IMO this systematic selection is downstream of culture, not utilitarianism
I think this is wrong about the history.
I was recently watched this Q&A with Holden in 2013. The picture I got is that GiveWell was focusing on “proven interventions” (e.g. bednets) over “speculative interventions” (e.g. biomedical research), because proven interventions were easier to evaluate.[1] He says he thinks speculative/high-risk stuff is probably better, and GiveWell Labs (i.e. Open Philanthropy, i.e. Coefficient Giving) was the pivot toward finding those opportunities.
Supporting quotations
“We really focused on what I’d call traditional GiveWell, which is looking for charities that are proven and cost effective and scalable… That’s why we picked it, is because we thought we’d be able to get somewhere on that.”
“We kind of bit off this chunk at the beginning of proven effective scalable… that was because it had a shorter feedback loop. Do this analysis, people would check the analysis… we would learn.”
“Proven can be easy to see, it also does tend to attract money from other funders… It allows a level of accountability that you don’t get with other things, it allows a level of honesty and objectivity and transparency you don’t get with other things.”
“You get much better information when you’re taken more seriously, and you’re taken more seriously when there’s more money behind you.”
“I feel like by running an organization and by operating within that framework, I’ve just gotten so much better… knowing what’s going to be accomplishable, knowing what sort of person to listen to.”
“We don’t believe, and didn’t necessarily — didn’t really ever believe — that this kind of charity is the only way to have a positive impact or the best way to have a positive impact.”
“I do think that by taking away the restrictions, I believe we’re going to find much better giving opportunities… Conventional wisdom in philanthropy is that speculative and risky is where all the action is.”
“The ROI on medical research is probably really good… Aging is one of the things that I would look at as being promising.”
Fun fact: Holden mentions that his “informal scientific adviser” for exploring biomedical research as a cause area ia a biology postdoc at Stanford called Dario Amodei.
My own take re rationalization/motivated reasoning is that at the end of the day, no form of ethics can meaningfully slow it down if the person either can’t credibly commit to their future selves, or simply isn’t bound/want to follow ethical rules, so the motivated reasoning critique isn’t EA specific, but rather shows 2 things:
People are more selfish than they think themselves to be, and care less about virtues, so motivated reasoning is very easy.
We can’t credibly commit our future selves to do certain things, especially over long timeframes, and even when people do care about virtues, motivated reasoning still harms their thinking.
Motivated reasoning IMO is a pretty deep-seated problem within our own brains, and is probably unsolvable in the near term.
To clarify, I think your criticism of utilitarianism/consequentialism is of a naive form of it that only looks at first-order effects. Not ‘proper’ utilitarianism. But yes no doubt many are naive like this, and it’s v hard to evaluate second- and higher-order effects (such as exploitation and coordination).
Also, this kind of naivety is particularly common on the left.
I refactored my thinking similarly a while ago.
However, I feel like traditional virtue-ethical notions e.g. “courage,” “integrity,” have the same adversarial Goodharting problem (3) as in your critique of utilitarianism. “Loyalty is when you obey the Master,” “courage is when you go to war against the Enemy without fearing death,” etc. I suspect utilitarianism is maybe only barely worse than other ethical systems in regards to (2). It’s worthwhile to compare EA against regular humans. I don’t really understand either.
I think of virtue ethics as something like “being a healthy and functional cog in the Humanity machine,” where the Humanity machine is ultimately utilitarian.
Further, I think a lot of arguments for utilitarian behaviors “pass through” to virtue ethics insofar as we think that traits like “ambition” and “scope sensitivity” are virtues. I think they are: seeing their characteristic absence has a similar sliminess to seeing a cowardly or slavish person.
(Sometimes it’s just a lack of underlying numeracy, which I would not consider a lack of virtue but rather of education. I spoke to a man who said he wouldn’t suck a dick for a billion dollars, because he just couldn’t. I walked him through the size of a billion, and he changed his mind.)