FYI I am generally good at tracking inside baseball but I understand neither what specific failures[1] you would have wanted to see discussed in an open postmortem nor what things you’d consider to be “improvements” (and why the changes since 2022/04/01 don’t qualify).
I’m sure there were many, but I have no idea what you consider to have been failures, and it seems like you must have an opinion because otherwise you wouldn’t be confident that the changes over the last three years don’t qualify as improvements.
Not sure what Benquo would say, but I think a natural question when a community of people fails at its goal after ~10 years, is to ask why it failed and what could’ve been done differently. It’s a good opportunity to learn, and it’s a good incentive to expect to do it so that you make sensible choices during the initial period that you’re doing the work (to expect to have to justify them if you fail).
I think that the CFAR one is more natural, because it seems to me that MIRI set itself a great scientific challenge with an externally imposed deadline, whereas CFAR did not have the external deadline on developing an art of rationality (which is also a very difficult problem). So it’s more naturally a place where the locus of control was within yourself, and a postmortem will be able to be accurate.
(I’m interested in this topic because I have myself considered trying to put together a public retro on the past ten years of safe-the-world efforts from this scene.)
The main effects of the sort of “AI Safety/Alignment” movement Eliezer was crucial in popularizing have been OpenAI, which Eliezer says was catastrophic, and funding for “AI Safety/Alignment” professionals, whom Eliezer believes to predominantly be dishonest grifters. This doesn’t seem at all like what he or his sincere supporters thought they were trying to do.
I’ve written extensively on this sort of perverse optimization, but I don’t see either serious public engagement with my ideas here, or a serious alternative agenda.
For instance, Ben Pace is giving me approving vibes but it didn’t occur to him to respond to your message by talking about the obvious well-publicized catastrophic failures I mentioned in the OP. And it seems like you forgot about them too by the time you wrote your comment.
I am sympathetic to your takes here, but I am not that sympathetic to statements like this:
but I don’t see either serious public engagement with my ideas here, or a serious alternative agenda.
As it happens I also happen to have written many tens-of-thousands of words about this in many comments across LW and the EA Forum. I also haven’t seen you engage with those things! (and my guess is the way you are phrasing it suggests you are not aware of them)
Like, man, I do feel like I resonate with the things that you are saying, but it just feels particularly weird to have you show up and complain that no one has engaged with your content on this, while having that exact relationship to approximately the people you are talking to. I, the head admin of LessWrong, have actually spent on the order of many hundreds of hours, maybe 1000+ hours on doing postmortem-ish things in the space, or at least calling for them. I don’t know whether you think what I did/do makes any sense, but I think there is a real attempt of the kind of thing you are hoping for (to be clear, mostly ending with a kind of disappointment and resulting distancing from much of the associated community’s, but it’s not like you can claim a better track record here).
And in contrast to your relationship with my content, I have read your content and have engaged with it a good amount. You can read through my EA Forum comments and LW comments on the topic if you want to get a sense of how I think about these things.
I’m aware that you’ve complained about these problems, but I’m specifically calling for the development and evaluation of explanatory models, which is a different activity. If you’ve done much of that in your public writing I missed it—anything you’d like to point me to?
That does seem like it’s overtly concerned with developing an explanation, but it seems concerned with deviance rather than corruption, so it’s on a different topic than the ones I complain about in the OP. I was aware of that one already, as I replied with a comment at the time.
And it seems like you forgot about them too by the time you wrote your comment.
It was not clear from your comment which particular catastrophic failures you meant (and in fact it’s still not clear to me which things from your post you consider to be in that particular class of “catastrophic failures”, which of them you attribute at least partial responsibility for to MIRI/CFAR, by what mechanisms/causal pathways, etc).
ETA: “OpenAI existing at all” is an obvious one, granted. I do not think EY considers SBF to be his responsibility (reasonable, given SBF’s intellectual inheritance from the parts of EA that were least downstream of EY’s thoughts). You don’t mention other grifters in your post.
FYI I am generally good at tracking inside baseball but I understand neither what specific failures[1] you would have wanted to see discussed in an open postmortem nor what things you’d consider to be “improvements” (and why the changes since 2022/04/01 don’t qualify).
I’m sure there were many, but I have no idea what you consider to have been failures, and it seems like you must have an opinion because otherwise you wouldn’t be confident that the changes over the last three years don’t qualify as improvements.
Not sure what Benquo would say, but I think a natural question when a community of people fails at its goal after ~10 years, is to ask why it failed and what could’ve been done differently. It’s a good opportunity to learn, and it’s a good incentive to expect to do it so that you make sensible choices during the initial period that you’re doing the work (to expect to have to justify them if you fail).
I think that the CFAR one is more natural, because it seems to me that MIRI set itself a great scientific challenge with an externally imposed deadline, whereas CFAR did not have the external deadline on developing an art of rationality (which is also a very difficult problem). So it’s more naturally a place where the locus of control was within yourself, and a postmortem will be able to be accurate.
(I’m interested in this topic because I have myself considered trying to put together a public retro on the past ten years of safe-the-world efforts from this scene.)
The main effects of the sort of “AI Safety/Alignment” movement Eliezer was crucial in popularizing have been OpenAI, which Eliezer says was catastrophic, and funding for “AI Safety/Alignment” professionals, whom Eliezer believes to predominantly be dishonest grifters. This doesn’t seem at all like what he or his sincere supporters thought they were trying to do.
I’ve written extensively on this sort of perverse optimization, but I don’t see either serious public engagement with my ideas here, or a serious alternative agenda.
For instance, Ben Pace is giving me approving vibes but it didn’t occur to him to respond to your message by talking about the obvious well-publicized catastrophic failures I mentioned in the OP. And it seems like you forgot about them too by the time you wrote your comment.
I am sympathetic to your takes here, but I am not that sympathetic to statements like this:
As it happens I also happen to have written many tens-of-thousands of words about this in many comments across LW and the EA Forum. I also haven’t seen you engage with those things! (and my guess is the way you are phrasing it suggests you are not aware of them)
Like, man, I do feel like I resonate with the things that you are saying, but it just feels particularly weird to have you show up and complain that no one has engaged with your content on this, while having that exact relationship to approximately the people you are talking to. I, the head admin of LessWrong, have actually spent on the order of many hundreds of hours, maybe 1000+ hours on doing postmortem-ish things in the space, or at least calling for them. I don’t know whether you think what I did/do makes any sense, but I think there is a real attempt of the kind of thing you are hoping for (to be clear, mostly ending with a kind of disappointment and resulting distancing from much of the associated community’s, but it’s not like you can claim a better track record here).
And in contrast to your relationship with my content, I have read your content and have engaged with it a good amount. You can read through my EA Forum comments and LW comments on the topic if you want to get a sense of how I think about these things.
I’m aware that you’ve complained about these problems, but I’m specifically calling for the development and evaluation of explanatory models, which is a different activity. If you’ve done much of that in your public writing I missed it—anything you’d like to point me to?
I have tried to do that, though it’s definitely more dispersed.
Most of it is still in comments and so a bit hard to extract, but one post I did write about this was My tentative best guess on how EAs and Rationalists sometimes turn crazy.
That does seem like it’s overtly concerned with developing an explanation, but it seems concerned with deviance rather than corruption, so it’s on a different topic than the ones I complain about in the OP. I was aware of that one already, as I replied with a comment at the time.
It was not clear from your comment which particular catastrophic failures you meant (and in fact it’s still not clear to me which things from your post you consider to be in that particular class of “catastrophic failures”, which of them you attribute at least partial responsibility for to MIRI/CFAR, by what mechanisms/causal pathways, etc).
ETA: “OpenAI existing at all” is an obvious one, granted. I do not think EY considers SBF to be his responsibility (reasonable, given SBF’s intellectual inheritance from the parts of EA that were least downstream of EY’s thoughts). You don’t mention other grifters in your post.