Now, what exactly is that fallacy in the risk assessment? Seems like it is thinking that there is only one very precise path how things could go wrong and then (perhaps correctly) calculating the probability of that one very precise path and reporting the result as the total risk: “This is the only way things could fail, and it’s probability is low, so everything is safe.”
That’s what I thought too, but the people who buy into this sort of assessment, they do acknowledge that it can be a bit off and that there could be other scenarios. They still think that the overall risk is somewhere in the few per million years range, and I never could quite get why. Now my theory is that due to the conjunction fallacy they see those highly detailed plausible paths to failure as the likely way it would fail, and then if the likely way it could fail is so unlikely—then it is safe. They don’t expect that this path to failure may be extremely unlikely in a very unsafe design.
That’s what I thought too, but the people who buy into this sort of assessment, they do acknowledge that it can be a bit off and that there could be other scenarios. They still think that the overall risk is somewhere in the few per million years range, and I never could quite get why. Now my theory is that due to the conjunction fallacy they see those highly detailed plausible paths to failure as the likely way it would fail, and then if the likely way it could fail is so unlikely—then it is safe. They don’t expect that this path to failure may be extremely unlikely in a very unsafe design.