Thanks author for this amazing piece, especially bibliography to followup up on some of the direct sources. While I had been introduced to Normal Accident Theory a decade ago, I wasn’t familiar with the HEVCR model and it definitely opens up new—and very practical—doors to thinking about existential risk in the context of AI.
If use the HEVCR what are the actual “exposure” and “vulnerability” issues for humanity? At least, I would assert that the exposure and vulnerability will vary widely between individuals and population groups. That is a stunning insight that in and of itself deserves future work.
I think the broader lesson to be drawn from this is that the EA/LessWrong/x-risk nexus really needs to make an effort to seek out and listen to people who have been working on related things for decades; they have some really useful things to say!
There seems to be a pervasive tendency here to try to reinvent the wheel from first principles, or else exclusively rely on a handful of approaches that fit the community’s highly quantitative/theoretical/high-modernist sensibilities—decision theory, game theory, prediction markets, judgemental forecasting, toy modelling, etc.---at the expense of all others.
Both of these approaches are rarely productive in my view.
(I am aware that this is not a new argument, but it bears making).
Some organisations are much scientifically omnivorous/open to multiple perspectives and approaches; I’m a big fan of both CSER and the GCRI, for instance.[1]
The Boring Apocalypses paper sketches out some ideas about existential exposures and vulnerabilities, but it’s very much opening the conversation rather than offering the final word.
You’ve definitely identified a major research gap, one made shocking by the fact that a clear majority of disaster risk reduction work is focused on reducing vulnerabilities and exposures rather than hazards.
The point on variability of vulnerability and exposure is also well-taken. Vulnerability usually increases along all the lines you would expect: people living in poverty, marginalised minorities, etc. are typically much more at risk (c.f. the Pressure and Release model).
Thanks author for this amazing piece, especially bibliography to followup up on some of the direct sources. While I had been introduced to Normal Accident Theory a decade ago, I wasn’t familiar with the HEVCR model and it definitely opens up new—and very practical—doors to thinking about existential risk in the context of AI.
If use the HEVCR what are the actual “exposure” and “vulnerability” issues for humanity? At least, I would assert that the exposure and vulnerability will vary widely between individuals and population groups. That is a stunning insight that in and of itself deserves future work.
Thanks!
I think the broader lesson to be drawn from this is that the EA/LessWrong/x-risk nexus really needs to make an effort to seek out and listen to people who have been working on related things for decades; they have some really useful things to say!
There seems to be a pervasive tendency here to try to reinvent the wheel from first principles, or else exclusively rely on a handful of approaches that fit the community’s highly quantitative/theoretical/high-modernist sensibilities—decision theory, game theory, prediction markets, judgemental forecasting, toy modelling, etc.---at the expense of all others.
Both of these approaches are rarely productive in my view.
(I am aware that this is not a new argument, but it bears making).
Some organisations are much scientifically omnivorous/open to multiple perspectives and approaches; I’m a big fan of both CSER and the GCRI, for instance.[1]
The Boring Apocalypses paper sketches out some ideas about existential exposures and vulnerabilities, but it’s very much opening the conversation rather than offering the final word.
You’ve definitely identified a major research gap, one made shocking by the fact that a clear majority of disaster risk reduction work is focused on reducing vulnerabilities and exposures rather than hazards.
The point on variability of vulnerability and exposure is also well-taken. Vulnerability usually increases along all the lines you would expect: people living in poverty, marginalised minorities, etc. are typically much more at risk (c.f. the Pressure and Release model).
Disclosure: I have worked at CSER as a Visiting Researcher.