Researching donation opportunities. Previously: ailabwatch.org.
Zach Stein-Perlman
Karma: 11,870
I agree.
(To be clear, since OP is ambiguous: generically boosting the Democratic Party might be surprising, but that’s very much not what’s going on. EAs donate to Ds and Rs, although the three most common donation-targets for the 2026 elections are all Ds. And none of these three donations increase D power — they’re all about replacing one D with a different D. Maybe OP misunderstands the situation.)
Better concept imo: games that are monotonic (i.e. anything that makes one player better off makes all others worse off) vs all other games. I don’t think there’s a good definition for “positive-sum,” “negative-sum,” and “zero-sum.”
I agree that the SFF experience is very terrible in many ways, including that it’s like six months. I’m hoping that Oli’s thing will be two months from application deadline to funding announcements.
I am pretty excited about Oli’s plan too. One difference of his plan from Manifund: the Manifund system is regrantors unilaterally allocate their budget (with opportunity for discussion), while in an s-process, after a recommender expresses a desire to fund a project, there’s an opportunity for other recommenders to express views and then the funders get to make a decision (on the project-level and/or the recommender-level). This is slower and more effortful but I expect it will result in better funding decisions, especially avoiding grants that are predictably bad to many people but not the regrantor.
Sorry for the lazy criticism. Thanks for your polite reply. And glancing at the page you just linked, I was pleasantly surprised by the 2025 stuff.
I think some disagreements here are probably intractable/illegible; it’s not worthwhile/helpful for me to try to explain my views on all of your regrantors. And I don’t have regrantor suggestions off the top of my head, sorry (fortunately, many of the non-grantmakers best positioned to be regrantors by my lights are already able to fund the lowest-hanging fruit by their lights, in various different ways).
Yeah, sorry, I’m thinking specifically of AI safety where there’s a bunch of people trying to make good things happen.
Oli: I don’t really believe in professional grantmakers as a role that benefits from specialization — I think it’s deeply harmed by the fact that you don’t get to experience things.
Maybe, but almost all non-professional grantmakers are bad at crucial aspects of grantmaking!
Austin: Yeah, I mostly agree — this is why I still like re-granting a lot. The concept of re-granting is that you give money to people who are very part-time grantmakers and mostly doing their research, field building, or whatever they’re really good at. And then if they find good opportunities, they can apply money towards that, while still being grounded in the actual work.
By my lights, some regrantors are great, but most are substantially worse than the standard professional grantmakers, and it’s apparently nontrivial to tell which are which (and in particular Manifund is not good at this).
Oli: Longview kind of never even tried [to be scrutable]. They’re just trying to be an institution that somehow builds such a strong trust relationship with its funders that it can overcome the principal-agent problem easily, and I’m not that optimistic about it. Historically they’ve been extremely limited in what grants they can make because they really need to select from a subset of grants that look credible and legible in a way that just often gets rid of 99% of the impact.
There have been very few Longview grants I’ve been excited about historically.
Austin: I mostly agree. I’m very confused why it seems like the Anthropic default — or among the set of Anthropic employees with liquidity in the current situation — is that they’re planning on giving it mostly to Longview. I don’t know.
Oli: I mean, what’s the alternative? That’s the big question. Like, what else are they going to do?
But I also think that partially it’s just kind of a con, in the sense that a lot of EAs are giving strong endorsements of Longview because they’re worried that otherwise the money won’t go to charity at all. So they’re willing to say things like, “these are amazing advisors, these are really trustworthy people” — when I think the right answer is to say, hey, this is a really, really cursed problem.
There’s a decent chance these people will be in a really, really difficult position where they’ll be very tempted to recommend grants that sound good to you when you think about it only a tiny bit.
They can get more money from you, use that to establish more of a reputation, and there’s this whole parallel game. These people are probably going to give you better advice than others who are also in this cursed problem class, but you should view them with a lot of suspicion — this is one of the most difficult principal-agent problems that exists. But that’s just not what people are saying. People are saying these people are extremely trustworthy and great and you should give them your money. I think there’s a thing going on where people want to make that recommendation because the money is very dead otherwise — it’s not going to go to charity at all.
I’m currently expecting that a lot of Anthropic employees will be pretty burned in a year or two. When they experience these dynamics, it becomes very clear that a huge amount of grants are never presented to them because people are worried they’ll find them weird. People will see that there’s a lot of politics behind the scenes about which grants get put in front of them. People will see that there’s a lot of weird reputational management — like, ‘I can’t make this grant because another funder would be angry at you if you made this grant.’ That’s really not what I want.
I also think that partially it’s just kind of a con, in the sense that a lot of EAs are giving strong endorsements of Longview because they’re worried that otherwise the money won’t go to charity at all.
Mostly disagree but might be missing some context. I have a generally high opinion of Longview’s AI grantmaking; when I have an opportunity to give money to nonprofits or advise a friend who’s doing so, I’m very interested in takes from Longview. They know about many good opportunities and have lots of information. Maybe Longview also gives less effective recommendations to donors who aren’t bought in to all of the weird AI safety stuff; maybe Oli is thinking about that (idk). And I agree Longview sometimes optimizes for things other than best-donation-opportunities, including in their (tiny) legibility-focused Emerging Challenges Fund. But I think it’s easy for donors to get advice from Longview’s AI grantmakers on the actual best donation opportunities, and I think that’s what relevant Anthropic staff are doing.
Longview doesn’t publish its grants so it’s hard to argue about them. But I think their AI grants are great (outside of digital minds stuff, which I believe is just a disagreement on the merits, not related to them being corrupt and influence-seeking).
My views here are partially based on: like, I know some of the Longview AI grantmakers, I’m similar to them in many ways, we’re all nerds who are way more interested in finding great donation opportunities than doing galaxy-brained stuff to manipulate donors to trust us more in the future. And my impression is that Longview’s AI grantmakers are much less constrained by leadership/funders than CG’s.
Oli: if you join [an s-process], you get to know where your money goes before you spend it — in a way that’s very difficult to achieve by almost any other means — that’s the key message.
Agree. It’s unfortunate that the funder-moving-last idea is coupled to SFF, which has many problems. I feel interested in trying a funder-moving-last system where recommenders include professional grantmakers.
If (1) Alice is good on AI safety, (2) Alice is legibly qualified for the role, (3) the election is tractable, and (4) the role is important or presents a credible path to an important government role by ~2033, then Alice will likely receive donations and maybe other support from AI safety folks (even in the absence of a public blogpost). Unfortunately, this conjunction is rare. E.g. running for state legislature generally fails #4 (especially if you run in 2028 or later), and AI safety people running for higher office out of the blue generally fails #2 and #3. I feel mostly pessimistic about random smart people “thinking about running for office” for the first time in 2028 and beyond, unless their resume gives them an excellent opportunity (but not confident + it depends).
The fact that candidates can expect support from individual donors is currently illegible outside of the AI safety community. I think that’s fine, in part because I’m much more excited about candidates who are actually AI-safety-motivated rather than just think-AI-safety-is-currently-politically-expedient. (Unlike for the NRA, a candidate saying they support/oppose certain regulation or making some pledge doesn’t go super far for AI safety and better futures [although it could be helpful in various ways].) More legible is the existence of AI-safety-ish super PACs like Public First; it’s good for Congress to know that AI safety is a strong political force (and fortunately this is becoming more salient, e.g. NYT yesterday). In general helping powerful incumbents feel safe standing up to the AI industry seems [more-important / higher-leverage] than getting new people to run [there’s also many other goals for AI safety in politics].
Joyce lost the primary 16-84. It’s hard for a Dem challenger to beat a Dem incumbent, especially without a stellar resume or establishment support.
“36cts per vote” is not accurate; Joyce would have gotten almost as many votes if she’d spent $0. “$350K probably does moves her chances of winning by 10%” is big if true but I (tentatively, with little info) think it’s <<1%. I wish her well but I don’t think this is a promising donation opportunity on the basis of flipping the election.
Yayyy!
(based on the summary — haven’t read the paper)
Adverse selection might be pretty brutal. And you can’t get much signal quickly; there would be tons of noise. (But it could still be worthwhile if the upside is high, idk.)
Good post.
the data on ads seems pretty clear
Citation needed. I’m not aware of good data.
To a first approximation, I’m into maximizing EV; I’m only into maxipok insofar as a good heuristic for maximizing EV. Portfolio allocation depends on tractability, but yeah I’m excited about work like https://www.forethought.org/research/viatopia.
the ~zero mode is distributed over +/-10^-10% optimal futures or even +/-10^-20% optimal futures, and the space inbetween from 10^-10% to 0.1% optimal futures has lower cumulative probability mass than either mode. So the EV of the future is basically equal to the probability of getting a future in the high value mode times the average value of a future in that mode.
I agree with this. I meant to say: a strong version of bimodality would imply that the absolute variance within high-value futures is much smaller than the variance between high-value and zero-value, and so nobody should work on moving us between different high-value futures. But that seems false.
Lol, I no longer endorse it! My current view is:
I still feel like almost all goods are ~0 terminal value because you get OOMs more goodness by optimizing hard, and we’re unlikely to produce terminal goods at large scale that are 5-95% of maximum value. (I think some experts agree and some disagree, but not sure.)
That’s before accounting for acausal considerations. I think people getting acausal stuff right but having wrong values is a very plausible way to get ~50% of max value.
And I now think compromise is very plausible, where the lightcone is filled with several different goods (or, if acausal stuff works out, our lightcone is filled with goods-that-people-in-other-universes-like but we’re trading for several different goods in their universes).
The binary view has been argued against by MacAskill and Assadi; I haven’t read that.
No, OpenAI and Anthropic staff currently generally cannot sell their equity. This will change after they go public. Some staff would donate more—not to mention diversify—if they could.
A year ago, people expected specific work outputs from me and they would (helpfully!) point out big ways I could do better. They appreciated my work but I was letting them down all the time by not being better.
Now I have no specific responsibilities and people don’t expect anything specific from me. When I do a good/helpful thing, people are happy/grateful. I do lots of little side projects; people are frequently happy/grateful. Nobody is telling me to update the website or expand my audience or find a cofounder. It’s great!
I don’t necessarily endorse the latter but it sure is way more relaxing.