$1 billion is not enough; OpenAI Foundation must start spending tens of billions each year
OpenAI is now a public benefit corporation, with a charter that demands they use AGI for the benefit of all, and do so safely. To justify this structure to the Attorneys General of Delaware and California, they split off the nonprofit OpenAI Foundation, and instead of full ownership, gave it 27% equity, worth well over $150 billion—what some have called the largest theft in human history. They said at the time they would donate a small fraction of the value, $25 billion, eventually.
This week, they announced a team, and a commitment; giving “at least $1 billion” over the coming year; in the full post, I argue that given their values and expectations, everyone should agree that this is far too little, moving far too slowly.
I agree locally with “$1billion indicators they are not serious about their mission (which is bad).”
But, I separately think it’s hard-to-the-point-of-impossible to give away $10 billion in a productive way, and it’s not obviously the right argument here.
If you read the full post on the EA Forum, I think I addressed this clearly. They could pick a half dozen ready-to-go paths, obviously including Givewell, with stated room for billions in funding, Givedirectly, with stated room for well over $1b, and Coeff, which is now explicitly set up to support large cause-area-specific commitments. And they can do all of this while ramping up internal capacity.
And all of that is without drastically lowering the bar for funding to, say, $10k/life saved instead of $5k—which I’d bet would mean those orgs have room for closer to $50b/year in funding. Or simply re-reviewing the > $20b in funding cut by USAID and picking their favorite 50%.
So I disagree, and claim it’s not nearly as hard to do this if they care about getting things done ASAP, since they’re actually rich enough to give ten billion this year, and double that next year, and do it again the year after, and double yet again the year after that—while still having capital left even if somehow the equity completely fails to appreciate in value—and they have a magically aligned ASI they’ve created before 2030 to fix everything else, or we’re all dead and at least they made things slightly better for people before that.
This might be true but the arguments here (and on other post in it’s current form) are pretty handwavy and I don’t find compelling and seem like a kind of naive philanthropy argument I’ve long found sus.
I didn’t really present arguments either, so, here’s a rough pass at that. I don’t think a quick back-and-forth will resolve things, this is like multiple posts worth of analysis to figure out what’s true in 2026.
But roughly my understanding is:
1) We don’t have good data on cost effectiveness
The raw data on cost effectiveness is sparse, confusedly measuring different things, doesn’t generalize as much as you’d like.
I would bet against there existing a stream of projects at $10k/life or whatever that are reasonably vetted and you could scale up to meet. Figuring out a half-decent estimate for a given project that you can compare against other projects, is a big endeavor.
The world is full of complex interplays, hidden gotchas, etc.
...
2) The skill of making sense of what data we have (or creating new data) is epistemically difficult
Givewell and OpenPhil/CoefficientGiving took a long time to build up the capacity to give away a billion a year. (Looks like OpenPhil gave away $1 billion for the first time last year, Claude Link with some citations) It involved lots of trial and error. It involved having early founders who really deeply cared about doing a good job with this.
The OpenAI nonprofit is not going to be starting off with a deep epistemic commitment to engaging with the real problem. (or, that’d be very surprising).
If CoefficientGiving isn’t giving away more than $1 billion / year so far, I don’t think you should expect them to be able to absorb $25 billion.
...
3) You can’t just hire experts for this
Because hiring experts is generally hard (because you don’t have the taste to tell the difference, and because few people even have this skill in the first place).
...
4) When you give away billions, you create an ecosystem of distortions and fraud around you that make it even more epistemically difficult.
Citation needed, exactly how bad this is is confusing, we can dig more into it but it’s my default expectation.
(I’m particularly worried about trying to give away billions for AI safety because it will dramatically increase the fraud / confused mediocre research signal/noise ratio. You didn’t particularly advocate that but wanted that as part of the convo)
...
There is not an off-the-shelf “donate $25 billion in way that is pretty-good” action that people can take. It requires a lot of skilled, intense attention.
I’m not arguing that this is easy, I was not trying to make a formal argument that would explain half of what EA is trying to figure out in a single post, and I agree that we won’t resolve all of this, but there is room for converging on some of this.
1) Agree, the data isn’t great. As I always say, the hard apart of decision making under uncertainty is the uncertainty. BUT that doesn’t mean you don’t have good ways of making those decisions anyways.
2&3) You don’t need to start from zero, and you absolutely can hire people; OpenAI hired away Jacob Trefethen from OpenPhil/Coeff, and they could similarly hire folks from Gates Foundation, USAID, etc. The people they hire have track records, and you don’t need nearly as much taste to look at what they actually donated to, or the analyses of lives saved from those programs. And the biggest part of the reason OpenPhil is giving money slowly is because Dustin doesn’t want to spend the money faster; if he did, they could lower the bar for programs, and/or give more to GiveWell, which really is open to getting much more funding..
4) Absolutely. And giving in areas that aren’t well established, like AI safety, or where there are huge information asymmetries, like most academic research, is hard. But doing so in areas like global poverty or public health or infectious disease work is much easier, and there are mature ecosystems for evaluating impact. But that said, yes, there will be fraud and waste, and you should work to minimize it—but the burden of keeping fraud or waste to 0.1% is more than two orders of magnitude larger than what is needed in keeping it below 10%, and given the stakes and timelines, that’s a tradeoff to make.
As to your final comment, I agree there is not an off-the-shelf “donate $25 billion in way that is pretty-good” action, and yes, it requires a lot of skilled, intense attention. But that is a thing you can, in fact, buy, or hire, and as we know, money is the unit of caring—not just caring about the thing, but caring about the ability to do the thing.
But there absolutely is a “donate at least $2-3 billion in way that is pretty-good” action—specifically, fully fund Givewell’s request, and fully fund GiveDirectly’s room for funding. Is this bulletproof? No. Are there perfect actions? No. But delaying is incredibly costly, and yes, if you commit to giving over a hundred billion dollars, you do have a moral, ethical, and legal duty to get off your ass and do it.
(And there WAS a “donate $25 billion in way that is clearly very good if not necessarily as effective as we normally expect in EA” action available a year ago, when they committed the money, which would have been to fund all the global health projects that got dropped by USAID for, say, a 2-year ramp down period to ensure that they don’t get broken or other funders or programs get step in.)
I agree with your argument completely. This subject has been sitting with me for a while, and I’ve been hesitant to raise it for fear of losing newbie karma — but I genuinely can’t understand why the prevailing sentiment toward OAI management (and Altman specifically) across LessWrong seems capped at a kind of tepid disdain rather than outrage.
Consider the conjunction:
The general consensus here has been, and continues to be, that ASI poses existential risk.
We are now living through an arms race toward its development, catalyzed in large part by OAI’s role as first mover.
The decision to initiate this race was a direct consequence of choices Altman made with full visibility into the consequences — in a role he obtained through an explicit promise to steward AI development against exactly the kind of competitive social opportunism he then executed, to near-maximal degree.
So: why don’t we hate this guy?
I’m not sure what additional evidence could more plausibly be obtained to support the claim that he is doing everything conceivable to accelerate the existential risk timeline — all while continuing to serve as the public authority and representative for the very category of expertise and preferences exchanged in this community, and taking every action available to contravene its influence.
I think FDT would literally prescribe more vitriol and “cancelling” in response to defections of this magnitude than I currently see. What am I missing?
I don’t disagree—but none of this is related to the argument made here about charitable giving by the foundation, which is conditional on allowing that the planned surrender of control of the for-profit was an OK decision and accepting their other stated views about charitable mission, AI timelines, etc.
I don’t consider that to be factually the case. It may be true of how you intended the article to be received but the post ends with: “I argue that given their values and expectations, everyone should agree that this is far too little, moving far too slowly” And my reply was, in amended terms: “Yes, I agree with that, everyone should as well, and I am surprised that everyone is not outraged by how little has been done to remain aligned to their initial charter and mission thus far.”
I don’t see how this could be interpreted as maligned or off topic by any stretch to the document itself. Is it prohibited to react to the piece of news received as news, or a claim about how everyone should feel about something with a question about why they don’t feel that way already?
Beyond that, the headlining statements of the document were news to me and may have been to many others. The implied expectation that it’s my responsibility to interpret solely one quality of the news (which is, generally much less significant than the facts you layered about it being the largest heist in modern history) I don’t consider to be a fair expectation of two-way discourse.
You were responding to a post that said specific things, not entering a discussion where you present related ideas which you want to talk about. If you want to talk about the news the post was responding to, or to talk about their general lack of mission alignment, you can do that in a separate post—or at the very least say that you have a separate related point from the topic of article.
I see the problem now. I failed to observe the content as a link post by skimming over the words ‘in the full post’. I then failed to observe you were presenting a link that you wrote and not additional news articles, and then reacted to your summary of the more detailed argument as if it was the entire content.
And then instead of carefully re-examining why I missed the point, I doubled down on the justification within the maligned epistemics. You are right, given that observation it was not the appropriate place for the comment and even worse the pushback. I am sorry for wasting your time.
Socratic question—what does being (especially within the in-group community) publicly being outraged at Altman achieve?
Can follow up with more thoughts later, but I am interested in your views as to the utility of various group postures (we can, for the sake of doing the interesting part of the discussion and not the boring part, assume we don’t need more evidence and that your claim is true).
Thanks for engaging the comment!
I consider the failures of alignment to date almost all a consequence of social and narrative capture and control versus any paper solution worked out by this group. CEV is probably generally correct. FDT is a very good formalization for a hard problem.
When OpenAI was created it was structured, institutionally, to be a bastion of the exact theory preferences and discourse long championed by this forum and MIRI. The challenges in getting LW ‘preferences’ actualized have never been its formalizations or the magnitude and precision of its claims (to date) — they have been the failure of the risk model well understood by this group to propagate outside of it, to the operators of institutions and the social forces that orient them.
The failure to sufficiently mitigate ASI existential risk while knowing it was coming is a problem in influence, not ‘right-ness’. Rather than adopting a more aggressive strategy toward the forces which continue to prevent the risk, it seems that dispassionate resignation or further detached analysis of failure modes becomes the exception handler — as opposed to calling truth to power in such a way that new norms are established memetically, and by osmosis to the broader public narrative which operates much more commonly on arguments of authority.
It is incredibly common that policy is formulated on sentiment, and sentiment is memetically contagious. If we genuinely believe that what Altman is doing is bad, and it is a necessary operationalization of our beliefs to continuously and relentlessly undermine that influence through arguments with emotive uptake, then why are the avenues that may make such an exercise effective not actualized rather than merely studied?
One of the primary motivators in human cognition is status. The activation of status-reducing social mechanisms — ostracism, reputational cost, public moral censure — against those who defect on alignment commitments is an ancient enforcement strategy, and one that remains psychologically powerful precisely where legal frameworks offer no protection. I don’t think any of these claims are at all structurally novel. A direct and labelled antagonist to the preferences of a group has historically been one of the strongest forces for public assembly in history.
Though it is a morally scrupulous strategy in regularity, I would argue this type of operationalization of narrative control is exactly what Altman has mastered, and unless this game is played, formalizations discussed here will forever remain beautiful in principle but prevented in practice the second that the next ‘superalignment’ team gets sidelined from implementing them.
Imagine you solved alignment tomorrow where SI could be built in a way to benefit all beings equally and democratically according to CEV—but the only way to implement it was by getting Altman to sign off on not profiting from its deployment. What are our odds of success and operational tools now? That problem continues to exist so long as people aren’t emotively loaded enough from both reason and psychological friction to go beyond an action threshold to attempt re-arrangement of either Altman’s values or sphere of control directly on alignment’s behalf.
The same argument can be made about whether it is rational to ‘beat someone up’ who defected, even if there is no consequential reward for the retribution. I am not advocating for violence but I am saying that we have rational reason for seeking a reputational re-balancing of public opinion, as being the ‘type’ of person to whom defection is costly, is what FDT can conclude structurally prevents misalignment from opportunism.
The operationalization of aligned or unaligned SI is being determined in a political, institutional, and narrative knife fight by people who all understand this. Is it the preferences of this group to be proven right about the risks or to actually prevent them? Because that is a human alignment problem, which is a value-loading problem, where the only available syntax to write solutions is meme.