I agree that this is the same type of thing as the construction example for Lighthaven, but I also think that you did leave some value on the table there in certain ways (e.g. commercial-grade furniture vs consumer-grade furniture), and I think that a larger total % domain-specific knowledge I’d hope exists at Open Phil is policy knowledge than total % domain-specific knowledge I’d hope exists at Lightcone is hospitality/construction knowledge.
I hear you as saying ‘experts aren’t all that expert’ *‘hiring is hard’ + ‘OpenPhil does actually have access to quite a few experts when they need them’ = ‘OpenPhil’s strategy here is very reasonable.’
I agree in principal here but think that, on the margin, it just is way more valuable to have the skills in-house than to have external people giving you advice (so that they have both sides of the context, so that you can make demands of them rather than requests, so that they’re filtered for a pretty high degree of value alignment, etc). This is why Anthropic and OAI have policy teams staffed with former federal government officials. It just doesn’t get much more effective than that.
I don’t share Mikhail’s bolded-all-caps-shock at the state of things; I just don’t think the effects you’re reporting, while elucidatory, are a knockdown defense of OpenPhil being (seemingly) slow to hire for a vital role. But running orgs is hard and I wouldn’t shackle someone to a chair to demand an explanation.
Separately, a lot of people defer to some discursive thing like ‘The OP Worldview’ when defending or explicating their positions, and I can’t for the life of me hammer out who the keeper of that view is. It certainly seems like a knock against this particular kind of appeal when their access to policy experts is on-par with e.g. MIRI and Lightcone (informal connections and advisors), rather than the ultra-professional, ultra-informed thing it’s often floated as being. OP employees have said furtive things like ‘you wouldn’t believe who my boss is talking to’ and, similarly, they wouldn’t believe who my boss is talking to. That’s hardly the level of access to experts you’d want from a central decision-making hub aiming to address an extinction-level threat!
To be clear, I was a lot more surprised when I was told about some of what OpenPhil did in DC, once starting to facepalm really hard after two sentences and continuing to facepalm very hard for most of a ten-minute-long story. It was so obviously dumb, that even me, with basically zero exposure to American politics or local DC norms and only some tangential experience running political campaigns in a very different context (an authoritarian country), immediately recognized it as obviously very stupid. While listening, I couldn’t think of better explanations than stuff like “maybe Dustin wanted x and OpenPhil didn’t have a way to push back on it”. But not having anyone who could point out how this would be very, very stupid, on the team, is a perfect explanation for the previous cringe over their actions; and it’s also incredibly incompetent, on the level I did not expect.
As Jason correctly noted, it’s not about “policy”. This is very different from writing papers and figuring out what a good policy should be. It is about advocacy: getting a small number of relevant people to make decisions that lead to the implementation of your preferred policies. OpenPhil’s goals are not papers; and some of the moves they’ve made that their impact their utility more than any of the papers they’ve funded more are ridiculously bad.
A smart enough person could figure it out from the first principles, with no experience, or by looking at stuff like how climate change became polarized, but for most people, it’s a set of intuitions, skills, knowledge that are very separate from those that make you a good evaluator of research grants.
It is absolutely obvious to me that someone experienced in advocacy should get to give feedback on a lot of decisions that you plan to make, including because some of them can have strategic implications you didn’t think about.
Instead, OpenPhil are a bunch of individuals who apparently often don’t know the right questions to ask even despite their employer’s magic of everyone wanting to answer their questions.
(I disagree with Jason on how transparent grant evaluations are ought to be; if you’re bottlenecked by time, it seems fine to make handwavy bets. You just need people who are good of making bets. The issue is that they’re not selected for making good bets in politics, and so they fuck up; not with the general idea of having people who make bets.)
I agree that this is the same type of thing as the construction example for Lighthaven, but I also think that you did leave some value on the table there in certain ways (e.g. commercial-grade furniture vs consumer-grade furniture), and I think that a larger total % domain-specific knowledge I’d hope exists at Open Phil is policy knowledge than total % domain-specific knowledge I’d hope exists at Lightcone is hospitality/construction knowledge.
I hear you as saying ‘experts aren’t all that expert’ * ‘hiring is hard’ + ‘OpenPhil does actually have access to quite a few experts when they need them’ = ‘OpenPhil’s strategy here is very reasonable.’
I agree in principal here but think that, on the margin, it just is way more valuable to have the skills in-house than to have external people giving you advice (so that they have both sides of the context, so that you can make demands of them rather than requests, so that they’re filtered for a pretty high degree of value alignment, etc). This is why Anthropic and OAI have policy teams staffed with former federal government officials. It just doesn’t get much more effective than that.
I don’t share Mikhail’s bolded-all-caps-shock at the state of things; I just don’t think the effects you’re reporting, while elucidatory, are a knockdown defense of OpenPhil being (seemingly) slow to hire for a vital role. But running orgs is hard and I wouldn’t shackle someone to a chair to demand an explanation.
Separately, a lot of people defer to some discursive thing like ‘The OP Worldview’ when defending or explicating their positions, and I can’t for the life of me hammer out who the keeper of that view is. It certainly seems like a knock against this particular kind of appeal when their access to policy experts is on-par with e.g. MIRI and Lightcone (informal connections and advisors), rather than the ultra-professional, ultra-informed thing it’s often floated as being. OP employees have said furtive things like ‘you wouldn’t believe who my boss is talking to’ and, similarly, they wouldn’t believe who my boss is talking to. That’s hardly the level of access to experts you’d want from a central decision-making hub aiming to address an extinction-level threat!
To be clear, I was a lot more surprised when I was told about some of what OpenPhil did in DC, once starting to facepalm really hard after two sentences and continuing to facepalm very hard for most of a ten-minute-long story. It was so obviously dumb, that even me, with basically zero exposure to American politics or local DC norms and only some tangential experience running political campaigns in a very different context (an authoritarian country), immediately recognized it as obviously very stupid. While listening, I couldn’t think of better explanations than stuff like “maybe Dustin wanted x and OpenPhil didn’t have a way to push back on it”. But not having anyone who could point out how this would be very, very stupid, on the team, is a perfect explanation for the previous cringe over their actions; and it’s also incredibly incompetent, on the level I did not expect.
As Jason correctly noted, it’s not about “policy”. This is very different from writing papers and figuring out what a good policy should be. It is about advocacy: getting a small number of relevant people to make decisions that lead to the implementation of your preferred policies. OpenPhil’s goals are not papers; and some of the moves they’ve made that their impact their utility more than any of the papers they’ve funded more are ridiculously bad.
A smart enough person could figure it out from the first principles, with no experience, or by looking at stuff like how climate change became polarized, but for most people, it’s a set of intuitions, skills, knowledge that are very separate from those that make you a good evaluator of research grants.
It is absolutely obvious to me that someone experienced in advocacy should get to give feedback on a lot of decisions that you plan to make, including because some of them can have strategic implications you didn’t think about.
Instead, OpenPhil are a bunch of individuals who apparently often don’t know the right questions to ask even despite their employer’s magic of everyone wanting to answer their questions.
(I disagree with Jason on how transparent grant evaluations are ought to be; if you’re bottlenecked by time, it seems fine to make handwavy bets. You just need people who are good of making bets. The issue is that they’re not selected for making good bets in politics, and so they fuck up; not with the general idea of having people who make bets.)