Suggestions for places to publicise this talk would also be greatly appreciated. Thanks!
Sean_o_h
Karma: 700
Cambridge (England) lecture: Existential Risk: Surviving the 21st Century, 26th February
This sounds like a good idea. I’ll either try to do so, or ask a member of CEA if they’d be willing to talk to me and then do so. I think I prefer the latter, as they may be able to give an assessment that is less biased and more informed by how “great” FHI is as a giving opportunity relative to other opportunities.
Unfortunately it got to a very late stage but was not funded. While reviewers ranked it very highly, it was always a long shot—we were asking for a lot of money for what is not a “traditional” academic discipline. However, we think that there are several foundations that might be inclined to support the type of research programme proposed—we aim to submit proposals to these early this year.
(Note: we had the same result with an Arts and Humanities Research Counciul grant submitted this fall—highly-ranked but ultimately unsuccessful. Also should be good material for further applications).
Re: point 1: The bulk of our policy consultations to date have actually been Nick Bostrom, although Anders Sandberg has done quite a bit, Toby has been regularly consulting with the UK government recently, and I’ve been doing some lately (mostly wearing my CSER hat, but drawing on my FHI expertise, so I would give FHI credit there ;) ) and others have also done bits and pieces.
Thanks. At least let me apologise for the formatting. I’m a frequent LW reader but infrequent poster. I’m trying to fix it, but not doing a great job ;)
Ways in which we’re underfunded:
Most of our funding comes from academic sources, which for the most part are quite constrained and not suitable for funding the full range of activities of a centre like the FHI. In order to be successful, we need to produce “lean and tight” funding applications that typically cover researchers’ salaries, perhaps some workshop/conference funds and a bare minimum of admin support. Most academic institutes within philosophy do not do nearly as much public outreach, projects of various types, visitor hosting and policy engagement as FHI does.
The result of this is that:
We’re typically underfunded in “core staff”. E.g. last year I was doubling up for the FHI as staff manager, fundraiser, grantwriter/coordinator, communications+media manager, administrator, researcher, workshop/conference organiser, research strategist, research networks person, and general purpose project manager. It also means that a lot of Nick Bostrom’s time ends up spend on this stuff (as well as on admin for himself that a PA could be doing), whereas ideally he would be able to focus on FHI research strategy and his own research. (In academia the expectation appears to be that a centre Director is a manager/administrator/networker/research strategist, whereas ideally we would like to free up Nick from as much of the admin/management/fundraising/networking burden as possible to allow research time). This year, our acquisition of industry funding has allowed us to get two new core staff members: an office administrator+PA for Nick Bostrom, and a general purpose academic project manager, which will help a lot—however, our overall research staff has also expanded (we currently have 13 staff members + 3 active collaborators, whereas we had 4 or 5 when I arrived in 2011). Therefore I believe core operations funding still represents a bottleneck (though less severely than before). I.e. a minimal (in the overall scheme) increase in our funding here would represent a very cost-effective way to increase our overall output of various types (research, outreach, fundraising, policy engagement, networking). While our “core” to “pure research” staffing ratio is large compared to a traditional academic research unit (e.g. a group in a room researching religious epistemology), it’s still very small compared to the organisations I know that are more directly similar to the FHI.
Many of the projects we would like to do are ones that are difficult to fund through academic means—e.g. the thesis prize competitions, currently-shelved plans for a “top young talent” summer school, some public communications/outreach projects. Also, having a reserve fund to hold onto key researchers between grants is extremely valuable (I’m trying to rebuild this at present after we depleted it last year holding onto Stuart Armstrong and Anders Sandberg. Several current researchers’ funds will run out in late 2014, so this fund may need to be put to use again!). This fund can also double up as funds FHI (as host institute) can commit towards supporting aspects of larger grant applications (e.g. support a conference/academic visitor programme to complement the research programme), which apparently “looks” very good to reviewers and may increase their likelihood of success.
On the research front:
There are several areas of research that we think would be valuable to expand into—some examples include surveillance technology and synthetic biology/genomics/biotech, while maintaining our AI risk research—and the in-house consensus is that doing more work on “core” existential risk concepts would also be very valuable. We are currently writing grants to support work in this area, but given success is not a guarantee, philanthropic support of additional researchers in these areas would be very valuable.
As mentioned elsewhere, at all times there appears to be at least one researcher available who we would love to hire, and who we have not been able to afford to.
As mentioned elsewhere, there are several researchers who are having to spend quite a bit of their time suboptimally (e.g. doing some direct X-risk research, but also producing less X-risk-relevant work in order to satisfy a funder) who it would be valuable to be able to “buy out” and have work full-time on X-risk, were the funds available.
Having funds to hire a research assistant to support the research work of key researchers—e.g. Nick Bostrom, Anders Sandberg, Stuart Armstrong, Toby Ord, Nick Beckstead—would be a very cost-effective way to increase their output and productivity (we did this to good effect for Nick’s Superintelligence book, but don’t currently have the funds to hire someone to do this full-time).
Different amounts of funding allow us to tackle different ones of these. I have a rough priority list based on what I think most cost-effectively increases our output/situation at a given time (and what I think we may not be able to get academic funding for) that currently goes Reserve fund → “hard to fund normally” projects → core activities → research assistant → hire more researchers/sub out researchers (depending on what talent is available at a given time). However, we’re constantly reassessing this as our research and situation evolves. It’s also the case that larger donations can be used in a different way than smaller ones (e.g. a new researcher may need to be funded 100% or not at all depending on situation). It’s also the case that if a donor specifically wants to fund desirable thing X, then that’s what we use the funds for.
Sorry for the essay, hope this is the kind of information you wanted.
I think it’s worth considering the funding landscape and the funding situations of the various organisations—“how much good” is obviously not static, and will depend on factors including this one. I’ve commented in other places in this thread that FHI is underfunded relative to its needs as an organisation and the research we think is important to do. I believe this is true.
I also believe that FHI has funding opportunities available to it that aren’t available to CEA/CFAR/MIRI. For example, at present ~80% of our overall funding comes from a combination of industry and academic grants.
This has its downsides—while this allows us scope to do a large amount of pure existential risk research, some of our research has to be tailored towards satisfying our funders, and so from a pure X-risk reduction point of view we are somewhat constrained. There are also a number of valuable uses of money that we can’t use these funds for.
It means the ~20% that we gain from philanthropic sources has very high utility. These are the funds that allow us to do public outreach, thesis prize competitions, further philanthropic/industry fundraising, extensive networking and visitor programmes, and other important work that doesn’t fit into a traditional academic research project—thus these funds add utility to the other ~80% of funds. Also, we have one researcher currently funded from a philanthropic donation, and he has the freedom to work purely on the Xrisk work he (or Nick Bostrom) considers most important, rather than having to devote some time to e.g. framing research output to fit an academic funder. This unconstrained research capability is of obvious value.
However, it also means that we can (at least at present) get the bulk of the funds needed to run the FHI from sources that may not be available to these other valuable organisations. While we can’t usually get grants to get exactly what we want to do, we can usually get grants to do something reasonably close—enough so that we can use them to get useful work done (even if we lose some efficiency compared to unconstrained funding). Hence, while I think donations sent to FHI will do quite a lot of good, I would be very reluctant to see too much funding diverted away from organisations that may not have the same breadth of funding options available to them if at all possible. (Frankly, I prefer to get funding from companies that sponsor rugby teams when they’re not sponsoring FHI ;) )This is also a reason CSER’s been targeting academic councils in the first instance.
tl;dr: We get quite a lot of our funding from other sources, and so may not always need additional funds as much as organisations with a less broad funding profile. That said, philanthropic funds that come to us play an important role in our running and have high utility.
Please consider this as a placeholder: I can provide some more info and am grateful for the inquiry; however, I’m a little pressed right now so will come back to this shortly. Thanks!
On point 1: I can confirm that members of CEA have done quite a lot of awareness-spreading about existential risks and long-run considerations, as well as bringing FHI, MIRI and other organisations to the attention of potential donors who have concerns in this area. I generally agree with Will’s point, and I think it’s very plausible that CEA’s work will result in more philanthropic funding coming FHI’s way in the future.
On point 2: I also agree. I need to have some discussion with the founders to confirm some points on strategy going forward as soon as the Christmas period’s over, but it’s likely that additional funds could play a big role in CSER’s progress in gaining larger funding streams. I’ll be posting on this shortly.
“FHI strikes me as more underfunded than MIRI, given that they are having to do a collaboration with an insurance company to stay afloat, whereas MIRI has maxed out all of their fundraisers to date. (Hence my decision to give to FHI this year.)”
It’s true that due to a grant ending and a failure to secure a followon grant we were heavily dependent on success with the insurance collaboration in order to hold onto key researchers and core staff needed to run the FHI. However, I would add that this is not my/our only motivation for heavily pursuing this area.
Wherever possible, I think there is a high value in FHI (and CSER) pursuing large pots of funding that are not available to other Xrisk/EA organisations (i.e. not diverting funds from other worthy organisations), and I think this type of funding is a prime example of this. If the current collaboration goes well, I believe there’s the possibility of substantial further funding in this area.
It is true that some staff time is being diverted towards systemic risk research, but it still represents a substantial increase in FHI’s overall Xrisk research output (and depending on availability of unconstrained funds and the development of the project, we may be in a position to “sub out” people as the project goes on, in favour of their doing 100% Xrisk research).
Lastly, I believe there is a value to us producing high-quality research on a more “mainstream” risk topic that gains us additional academic prestige and credibility, and thus lends credibility to the more speculative existential risk work we do. It also introduces new, potentially influential, audiences to our existential risk work.
“On the other hand, I have heard MIRI/FHI/CEA staff claiming they are much more in need of qualified people than money”
“Getting the right people is the most important thing” is a general principle at FHI. However, in my 2.5 years managing FHI there have consistently been people we haven’t been able to hire and research areas we haven’t been able to address, although we’ve been successful in expanding and improving our funding situation quite a lot. If you presented us with another qualified person right now, I don’t see how we would be able to hire them at present (although that may not be the case in some months, depending on grant successes, etc). We’ve also consistently been understaffed and underfunded on core operations, and thus have only been able to avail of a fraction of the opportunities available to us.
Kristian has returned to Sweden, but is still working remotely part-time for FHI.
The organisers certainly are doing so. However, it’s easy to underestimate the sheer amount of work involved in setting up a centre. Huw and Martin have been putting a lot of time into contacting academic advisors, making policy connections, liaising with Cambridge University and the relevant centres within Cambridge, meeting with representatives from various funding bodies, writing funding applications, and doing media and public outreach. Jaan has provided seed funding, is also making relevant connections and doing media outreach, and has been helping with website design.
Regarding money: Martin and Huw don’t possess the kind of wealth to fund a whole research centre, and Jaan in addition to his CSER funding is funding MIRI, FHI and other organisations. Furthermore, an organisation getting its funds from outside sources, particularly respected academic funding bodies, probably looks more reputable than one that is mainly funded by its own organisers. Raising funds is a big part of setting up any new research centre, Xrisk or otherwise.
Regarding time: Jaan helping with the website is generous, but is not the best use of his time, given his earning and influence power—so I’m going to be minimising how much he’s doing that. Martin and Huw manning the general email account and doing the time-consuming legwork involved in writing grant applications and dealing with bureaucracy is not the best use of their time, given their positions of influence. Better to have someone like me (and others like me) doing that, and for them to be making the connections that I can’t.
There’s just a lot of legwork to be done. The founders aren’t in a position to quit their other positions and responsibilities to devote themselves to that legwork, and even if they were it wouldn’t represent the best use of their time—they would lose valuable influence and earning potential.
“By the way most notable work on x-risks was done without any visable funding, I mean articles of Bostrom and Yudkowsky.” I can’t speak for Yudkowsky, but there certainly was visible (if flexible) funding from the Oxford Martin School for Bostrom’s work on existential risk.
I must admit it leaves me in a bit of a quandry. Now that I’ve got a foot in both camps, I have no idea who it’s safe to support in the annual Water Polo and Tiddlywinks Intervarsity Grudge Match.
We’ll make sure to get high-quality recordings.
In the meantime, the FHI’s youtube channel (http://www.youtube.com/user/FHIOxford/videos) may be of interest (talks from Tegmark, Bostrom, Armstrong, Sandberg, Yudkowsky, Omohundro, plus a recent animation of Bostrom’s “Fable of the Dragon Tyrant”).
Update on establishment of Cambridge’s Centre for Study of Existential Risk
Vacancy at the Future of Humanity Institute: Academic Project Manager
That’s true. Adam is kindly updating videos with the correct year, and we’ll reupload shortly. He’s done excellent and professional work for the conference.
As an FYI, the FHI’s not planning to have a Winter Intelligence Conference in winter 2013, as we’re pretty busy getting new projects off the ground. The expectation is that we’ll continue having them at two year intervals for now (the previous WIC was in January 2011), although this isn’t set in stone. This year’s AGI conference will be in Beijing July 31- August 3 (last year’s was hosted in Oxford as part of the WIC)
Excellent point, didn’t even occur to me. Fixed!