“if you care about this, here’s a way to get involved”
My understanding is that MIRI expects alignment will be hard, an international treaty will be needed, and believes that a considerable proportion of the work that gets branded as “AI safety” is either unproductive or counterproductive.
MIRI could of course be wrong, and it’s fine to have an ecosystem where people are pursuing different strategies or focusing on different threat models.
But I also think there’s some sort of missing mood here insofar as the post is explicitly about the MIRI book. The ideal pipeline for people who resonate with the MIRI book may look very different than the typical pipelines for people who get interested in AI risk (and indeed, in many ways I suspect the MIRI book is intended to spawn a different kind of community and a different set of projects than the community/projects that dominated the 2020-2024 period, for example.)
Relatedly, I think this is a good opportunity for orgs/people to reassess their culture, strategy, and theories of change. For example, I suspect many groups/individuals would not have predicted that a book making the AI extinction case so explicitly and unapologetically would have succeeded. To the extent that the book does succeed, it suggests that some common models of “how to communicate about risk” or “what solutions are acceptable/reasonable to pursue” may be worth re-examining.
but if what’s actually happening is that people interpret it as cynical dishonesty that does not believe its own doom arguments and thus must be instead whatever the next most likely reason is to make a doom argument, which seems to be a common reaction, then it may be made of backfire. I find it very hard to tell whether this is happening, and I know of many people who think it’s the only thing that happens. I certainly do think it’s a thing that happens ever.
My understanding is that MIRI expects alignment will be hard, an international treaty will be needed, and believes that a considerable proportion of the work that gets branded as “AI safety” is either unproductive or counterproductive.
MIRI could of course be wrong, and it’s fine to have an ecosystem where people are pursuing different strategies or focusing on different threat models.
But I also think there’s some sort of missing mood here insofar as the post is explicitly about the MIRI book. The ideal pipeline for people who resonate with the MIRI book may look very different than the typical pipelines for people who get interested in AI risk (and indeed, in many ways I suspect the MIRI book is intended to spawn a different kind of community and a different set of projects than the community/projects that dominated the 2020-2024 period, for example.)
Relatedly, I think this is a good opportunity for orgs/people to reassess their culture, strategy, and theories of change. For example, I suspect many groups/individuals would not have predicted that a book making the AI extinction case so explicitly and unapologetically would have succeeded. To the extent that the book does succeed, it suggests that some common models of “how to communicate about risk” or “what solutions are acceptable/reasonable to pursue” may be worth re-examining.
but if what’s actually happening is that people interpret it as cynical dishonesty that does not believe its own doom arguments and thus must be instead whatever the next most likely reason is to make a doom argument, which seems to be a common reaction, then it may be made of backfire. I find it very hard to tell whether this is happening, and I know of many people who think it’s the only thing that happens. I certainly do think it’s a thing that happens ever.