I think the things Rob is saying still have some strawman-y nature to them, but I think they are reasonably accurate descriptors of Anthropic leadership, plus my best guesses of what Alexander (head of Coefficient Giving) and Zach (head of CEA) believe. I feel like almost all of your comment is just running with that misunderstanding.
But aren’t Alexander Berger’s views not very relevant about OpenPhil’s AI strategy decisions from many years ago when their AI strategy and worldview—which I take to be very cose to the things Rob was criticizing—were worked out and started shaping the views of EAs in OpenPhil’s orbit?
Even now, when people criticize things OpenPhil has done in the past in the AI landscape, or criticize their general worldview and takes on AI risk (as it was developed in influential pieces of writing), I am by default automatically viewing it as criticism of Holden, Ajeya Cotra, Tom Davidson, Joe Carlsmith, etc. If people don’t intend me to interpret them that way, please be more clear. 🙂
I’m aware that, separately, OpenPhil/​Coefficient Giving has undergone quite a transition and that you clashed badly with Dustin M. I think that’s very sad and unfortunate, but I think of these as quite distinct things and I never assumed that the thing with Dustin M. had anything to do with OpenPhil’s AI strategy decisions in (say) five years ago (edit: sorry that sounds like a strawman, but I mean something like “I’m not sure the same cause explains why some people who were at OpenPhil in the past found MIRI epistemically off-putting, and why Dustin M finds the rationalists to be a reputation risk & thinks reputation risks are unusually bad compared to other bad things.”) I could be wrong, of course, and maybe you think the org has a general thing of them of valuing “reputability” and “playing politics” too much. I just want to note that it’s not obvious how much these things are connected/​caused by one “OpenPhil culture,” vs being about distinct things. (I think some of these are maybe directionally accurate as criticism, btw.)
I’m sure this is obvious to everyone involved, but I also just want to point out that when a lot of senior people leave, organizations can change really a lot, so it would be weird to speak of OpenPhil/​Coefficient Giving now as though it were obviously still the same entity/​culture.
But aren’t Alexander Berger’s views not very relevant about OpenPhil’s AI strategy decisions from many years ago when their AI strategy and worldview—which I take to be very cose to the things Rob was criticizing—were worked out and started shaping the views of EAs in OpenPhil’s orbit?
I think Holden at the time believed something closer to what Rob says here (though it’s still not an amazing fit), and more generally, I think “the beliefs of the successor CEO” are actually a better proxy for “the vibes of the broader ecosystem you are part of” than “the beliefs of the founder CEO”. I could go into more detail on my beliefs on this, though I think the argument is reasonably intuitive.
but I think of these as quite distinct things and I never assumed that the thing with Dustin M. had anything to do with OpenPhil’s AI strategy decisions in (say) five years ago
Yep, I think they are highly related. Indeed, I was predicting things like the Dustin thing without any knowledge of Dustin’s specific beliefs, and my predictions were primarily downstream of seeing how Anthropic’s position within the ecosystem was changing, and a broader belief-system that I think is shared by many people in leadership, not just Dustin.
I have since updated that more people who are a level below Alexander, Dustin and Dario have more reasonable beliefs, but also updated that those things end up mattering surprisingly little for what actually ends up a strategic priority.
I just want to note that it’s not obvious how much these things are connected/​caused by one “OpenPhil culture,” vs being about distinct things. (I think some of these are maybe directionally accurate as criticism, btw.)
I think the “OpenPhil culture” thing is a distraction. In my model of the world most of this is downstream of people being into power-seeking strategies mostly from a naive-consequentialist lens, which is not that unique to OpenPhil within EA (and if anything OpenPhil has some of the people with the best antibodies to this, though also a lot of people who think very centrally along these lines, more concentrated among current leadership).
I think some of the people who are best at thinking independently about stuff, and are pretty good at not getting swept up in the power-seeking stuff, work at Open Phil. I think Holden genuinely helped with some of the correct cultural pieces, and my current belief is that if he wasn’t under the most pressure that anyone is, that he would probably have a relatively sane relationship to Anthropic as a result of it, though I am not as confident I am about that as I am that he had a bunch of quite good cultural pieces that help people be less naively power-seeking here.
But aren’t Alexander Berger’s views not very relevant about OpenPhil’s AI strategy decisions from many years ago when their AI strategy and worldview—which I take to be very cose to the things Rob was criticizing—were worked out and started shaping the views of EAs in OpenPhil’s orbit?
Even now, when people criticize things OpenPhil has done in the past in the AI landscape, or criticize their general worldview and takes on AI risk (as it was developed in influential pieces of writing), I am by default automatically viewing it as criticism of Holden, Ajeya Cotra, Tom Davidson, Joe Carlsmith, etc. If people don’t intend me to interpret them that way, please be more clear. 🙂
I’m aware that, separately, OpenPhil/​Coefficient Giving has undergone quite a transition and that you clashed badly with Dustin M. I think that’s very sad and unfortunate, but I think of these as quite distinct things and I never assumed that the thing with Dustin M. had anything to do with OpenPhil’s AI strategy decisions in (say) five years ago (edit: sorry that sounds like a strawman, but I mean something like “I’m not sure the same cause explains why some people who were at OpenPhil in the past found MIRI epistemically off-putting, and why Dustin M finds the rationalists to be a reputation risk & thinks reputation risks are unusually bad compared to other bad things.”) I could be wrong, of course, and maybe you think the org has a general thing of them of valuing “reputability” and “playing politics” too much. I just want to note that it’s not obvious how much these things are connected/​caused by one “OpenPhil culture,” vs being about distinct things. (I think some of these are maybe directionally accurate as criticism, btw.)
I’m sure this is obvious to everyone involved, but I also just want to point out that when a lot of senior people leave, organizations can change really a lot, so it would be weird to speak of OpenPhil/​Coefficient Giving now as though it were obviously still the same entity/​culture.
I think Holden at the time believed something closer to what Rob says here (though it’s still not an amazing fit), and more generally, I think “the beliefs of the successor CEO” are actually a better proxy for “the vibes of the broader ecosystem you are part of” than “the beliefs of the founder CEO”. I could go into more detail on my beliefs on this, though I think the argument is reasonably intuitive.
Yep, I think they are highly related. Indeed, I was predicting things like the Dustin thing without any knowledge of Dustin’s specific beliefs, and my predictions were primarily downstream of seeing how Anthropic’s position within the ecosystem was changing, and a broader belief-system that I think is shared by many people in leadership, not just Dustin.
I have since updated that more people who are a level below Alexander, Dustin and Dario have more reasonable beliefs, but also updated that those things end up mattering surprisingly little for what actually ends up a strategic priority.
I think the “OpenPhil culture” thing is a distraction. In my model of the world most of this is downstream of people being into power-seeking strategies mostly from a naive-consequentialist lens, which is not that unique to OpenPhil within EA (and if anything OpenPhil has some of the people with the best antibodies to this, though also a lot of people who think very centrally along these lines, more concentrated among current leadership).
What do you mean by this?
I think some of the people who are best at thinking independently about stuff, and are pretty good at not getting swept up in the power-seeking stuff, work at Open Phil. I think Holden genuinely helped with some of the correct cultural pieces, and my current belief is that if he wasn’t under the most pressure that anyone is, that he would probably have a relatively sane relationship to Anthropic as a result of it, though I am not as confident I am about that as I am that he had a bunch of quite good cultural pieces that help people be less naively power-seeking here.