Why I hate the “accident vs. misuse” AI x-risk dichotomy (quick thoughts on “structural risk”)

I expect irresponsible, reckless, negligent deployment of AI systems without proper accounting of externalities. I consider this the default for any technology with potential for significant externalities, absent regulation.

When something bad happens in such a context, calling it “accident risk” absolves those researching, developing, and/​or deploying the technology of responsibility. They should have known better. Some of them almost certainly did. Rationalization, oversight, and misaligned incentives were almost certainly at play. Failing to predict the particular failure mode encountered is no excuse. Having “good intentions” is no excuse.

So… it must be misuse then, right? Well, no. Calling it “misuse” suggests that those researching, developing, and/​or deploying the technology set out with nefarious purposes and the technology achieved precisely what they intended. But ~nobody wants to destroy the world. It’s just that most people are somewhat selfish and so are willing to trade some x-risk for a large personal benefit.

In summary, saying “accident” makes it sounds like an unpredictable effect, instead of painfully obviously risk that was not taken seriously enough. Saying “misuse” makes it sounds like some supervillian or extremist deliberately destroying the world. While some risks may have something more of a flavor or accident or misuse depending on how obvious the risk was, neither of these pictures gives a remotely accurate picture of the nature of the problem.

I think this makes it a harmful meme, and ask that others stop making this distinction (without appropriate caveats), and join me in pointing out how it contributes to a confused and misleading discourse when others do.

EtA: Many people have responded that “accident” does not connote “unforseen” or “not negligent”, etc., and instead it should simply be interpreted as something like “a result that was not deliberately selected for”. While it can be used this way, I basically disagree that this is how it is usually used, see below:

EtA: as an additional clarification: my main objection is not to the use of “accident” and “misuse”, but rather to their use as a dichotomy. Every use of these terms I can recall seeing in writing (other than those that mention structural risk) supports this dichotomy, and it is often made explicitly.