What funding sources exist? Who are they aimed at—people within academia, independent researchers, early-career researchers, established researchers? What sort of research are they aimed at—MIRI-style deconfusion, ML-style approaches, more theoretical, less theoretical? What quirks do they have, what specific things do they target?
For purposes of this question, I am not interested in funding sources aimed at strategy/infrastructure/coordination/etc, only direct technical safety research.
Some opportunities that I know about:
The Long Term Future Fund is I think currently the most active granter in the space for independent researchers
MIRI has in the past funded some independent researchers, and some programs that try to facilitate research (like funding to MIRIx workshops, or the AI Safety camp, or related projects)
Jaan Tallinn has some budget set aside to fund people doing interesting research, I think partially administered via Alex Zhu. Happy to put anyone interested into contact.
There exist a variety of small donors who I think are open to this. Matt Wage (also on the LTFF) is often open to things like this.
The Open Philanthropy Project has (had?) a scholarship program, here are some examples of people they funded, I do not know whether there is any way to apply to it:
https://www.openphilanthropy.org/focus/global-catastrophic-risks/scholarship-support-2019
https://www.openphilanthropy.org/focus/global-catastrophic-risks/potential-risks-advanced-artificial-intelligence/ai-scholarships-2018
The Open Philanthropy Project also has an AI Policy careers scholarship program:
https://www.openphilanthropy.org/focus/global-catastrophic-risks/potential-risks-advanced-artificial-intelligence/funding-AI-policy-careers
The EA Foundation has given grants for AI Alignment Research:
https://ea-foundation.org/grantmaking/
Being a student at CHAI is one way of getting funding. They might also be open to more independent research arrangements, but not sure:
https://humancompatible.ai/
The FHI research scholars program includes people who work on technical AI Alignment:
https://www.fhi.ox.ac.uk/rsp/
BERI has historically done some grantmaking in this space, though that will likely now be done via the Survival and Flourishing Fund. For that, applications are open for 4 more days, though require some kind of existing institutional affiliation.
I can imagine that Tyler Cowen’s Emergent Ventures is open to funding things in this space
Will edit this if I come up with anything else, or if someone else points out something that I am missing.
Edit: New additions by Issa Rice:
Paul Christiano has done some independent funding in this space (and maybe Open AI is also open to that, but I don’t know)
FLI has made grants, though I don’t know about the current state of their grant program
EA Grants also has given some grants in this space
Potential additions: Paul Christiano, Future of Life Institute, EA Grants.
Indeed, thanks! Added them to the list above