I think if they sponsored Cotra’s work and cited it, this reflects badly on them.
I find that position weirdly harsh. Sure, if you’re just answering anaguma’s question as a binary (“does it reflect well or poorly, regardless of magnitude?”), that could make sense. (Note to readers: This would mean that the quote I started this comment with should be regarded as taken out of context!) But seeing it as reflecting badly at a high magnitude is the judgment I’d consider weirdly harsh.
I’m saying that as someone who has very little epistemic respect for people who think AI ruin is only about 10% likely—I consider people who think that biased beyond hope.
But back to the timelines point:
It’s not like Bioanchors was claiming high confidence in its modelling assumptions or resultant timelines. At the time, a bunch of people in the broader EA ecosystem had even longer timelines, and Bioanchors IIRC took a somewhat strong stance against assigning significant probability mass for >2100, which some EAs at least considered non-obvious. Seen in that context, it contributed to people updating in the right direction. The report also contained footnotes pointing out that advisors held in high regard by Ajeya had shorter timelines based on specific thoughts on horizon lengths or whatever, so the report was hedging towards shorter timelines. Factoring that in, it aged less poorly than it would have if we weren’t counting those footnotes. Ajeya also posted an update 2 years later where she shortened her timelines a bunch. If it takes orgs only 2 years to update significantly in the right direction, are they really hopelessly broken?
FWIW, I’m leaning towards you having been right about the critique (credit for sticking your neck out). But why is sponsoring or citing work like that such a bad sign? Sure, if they cited it as particularly authoritative, that would be different. But I don’t feel like Open Phil did that. (This seems like a crux based on your questions in the OP and your comments here; my sense from reading other people’s replies, and also my less informed impressions I got from interacting with some Open Phil staff at very few short occasions, is it that you were overestimating the degree to which Open Phil was attached to specific views.)
For comparison, I think Carlsmith’s report on power-seeking was a lot worse in terms of where its predictions landed, so I’d have more sympathy if you pointed to it as an example of what reflects poorly on Open Phil (just want to flag that Carlsmith is my favorite philosophy writer in all of EA). However, also there, I doubt the report was particularly influential within Open Phil, and I don’t remember it being promoted as such. Also, I would guess that the pushback it received from many sides would have changed their evaluation of the report after it was written, if they had initially been more inclined to update on it. I mean, that’s part of the point of writing/publishing reports like that.
Sure, maybe Open Phil was doing a bunch of work directed more towards convincing outside skeptics that what they’re doing is legitimate/okay rather than doing the work “for themselves”? If so, that’s a strategic choice… I can see it leading to biased epistemics, but in a world where things had gone better, maybe it would have gotten further billionaires on board with their mission of giving? And it’s not like doing the insular MIRI thing that you all had been doing before the recent change to get into public comms was risk-free for internal epistemics either. There are risks on both ends of the spectrum, outward-looking and deferring to many experts or at least “caring whether you can convince them”, and inward looking with a small/shrinking circle of people whose research opinions you respect.
On whether some orgs are/were hopelessly broken: it’s possible. I feel sad about many things having aged poorly and I feel like the EA movement has done disappointingly poorly. I also feel like I’ve heard once or twice Open Phil staff saying disappointingly dismissive things about MIRI (even though many of the research directions there didn’t age well either).
I don’t have a strong view on Open Phil anymore—it used to be that I had one (and it was positive), so I have became more skeptical. Maybe you’re picking up a real thing about Open Phil’s x-risk-focused teams having been irredeemably biased or clouded in their approaches. But insofar as you are, I feel like you’ve started with unfortunate examples that, at least to me, don’t ring super true. (I felt prompted to comment because I feel like I should be well-dispositioned to sympathize with your takes given how disappointed I am at the people who still think there’s only a 10% AI ruin chance.)
I find that position weirdly harsh. Sure, if you’re just answering anaguma’s question as a binary (“does it reflect well or poorly, regardless of magnitude?”), that could make sense. (Note to readers: This would mean that the quote I started this comment with should be regarded as taken out of context!) But seeing it as reflecting badly at a high magnitude is the judgment I’d consider weirdly harsh.
I’m saying that as someone who has very little epistemic respect for people who think AI ruin is only about 10% likely—I consider people who think that biased beyond hope.
But back to the timelines point:
It’s not like Bioanchors was claiming high confidence in its modelling assumptions or resultant timelines. At the time, a bunch of people in the broader EA ecosystem had even longer timelines, and Bioanchors IIRC took a somewhat strong stance against assigning significant probability mass for >2100, which some EAs at least considered non-obvious. Seen in that context, it contributed to people updating in the right direction. The report also contained footnotes pointing out that advisors held in high regard by Ajeya had shorter timelines based on specific thoughts on horizon lengths or whatever, so the report was hedging towards shorter timelines. Factoring that in, it aged less poorly than it would have if we weren’t counting those footnotes. Ajeya also posted an update 2 years later where she shortened her timelines a bunch. If it takes orgs only 2 years to update significantly in the right direction, are they really hopelessly broken?
FWIW, I’m leaning towards you having been right about the critique (credit for sticking your neck out). But why is sponsoring or citing work like that such a bad sign? Sure, if they cited it as particularly authoritative, that would be different. But I don’t feel like Open Phil did that. (This seems like a crux based on your questions in the OP and your comments here; my sense from reading other people’s replies, and also my less informed impressions I got from interacting with some Open Phil staff at very few short occasions, is it that you were overestimating the degree to which Open Phil was attached to specific views.)
For comparison, I think Carlsmith’s report on power-seeking was a lot worse in terms of where its predictions landed, so I’d have more sympathy if you pointed to it as an example of what reflects poorly on Open Phil (just want to flag that Carlsmith is my favorite philosophy writer in all of EA). However, also there, I doubt the report was particularly influential within Open Phil, and I don’t remember it being promoted as such. Also, I would guess that the pushback it received from many sides would have changed their evaluation of the report after it was written, if they had initially been more inclined to update on it. I mean, that’s part of the point of writing/publishing reports like that.
Sure, maybe Open Phil was doing a bunch of work directed more towards convincing outside skeptics that what they’re doing is legitimate/okay rather than doing the work “for themselves”? If so, that’s a strategic choice… I can see it leading to biased epistemics, but in a world where things had gone better, maybe it would have gotten further billionaires on board with their mission of giving? And it’s not like doing the insular MIRI thing that you all had been doing before the recent change to get into public comms was risk-free for internal epistemics either. There are risks on both ends of the spectrum, outward-looking and deferring to many experts or at least “caring whether you can convince them”, and inward looking with a small/shrinking circle of people whose research opinions you respect.
On whether some orgs are/were hopelessly broken: it’s possible. I feel sad about many things having aged poorly and I feel like the EA movement has done disappointingly poorly. I also feel like I’ve heard once or twice Open Phil staff saying disappointingly dismissive things about MIRI (even though many of the research directions there didn’t age well either).
I don’t have a strong view on Open Phil anymore—it used to be that I had one (and it was positive), so I have became more skeptical. Maybe you’re picking up a real thing about Open Phil’s x-risk-focused teams having been irredeemably biased or clouded in their approaches. But insofar as you are, I feel like you’ve started with unfortunate examples that, at least to me, don’t ring super true. (I felt prompted to comment because I feel like I should be well-dispositioned to sympathize with your takes given how disappointed I am at the people who still think there’s only a 10% AI ruin chance.)