“Conjunction fallacy and probabilistic risk assessment” is enough for a title; you can move the applause lights into the article.
Now, what exactly is that fallacy in the risk assessment? Seems like it is thinking that there is only one very precise path how things could go wrong and then (perhaps correctly) calculating the probability of that one very precise path and reporting the result as the total risk: “This is the only way things could fail, and it’s probability is low, so everything is safe.”
EDIT: After reading the articles, there are many different fallacies combined, many of them probably caused by motivated cognition—we want the project to succeed, and we want it now.
Now, what exactly is that fallacy in the risk assessment? Seems like it is thinking that there is only one very precise path how things could go wrong and then (perhaps correctly) calculating the probability of that one very precise path and reporting the result as the total risk: “This is the only way things could fail, and it’s probability is low, so everything is safe.”
That’s what I thought too, but the people who buy into this sort of assessment, they do acknowledge that it can be a bit off and that there could be other scenarios. They still think that the overall risk is somewhere in the few per million years range, and I never could quite get why. Now my theory is that due to the conjunction fallacy they see those highly detailed plausible paths to failure as the likely way it would fail, and then if the likely way it could fail is so unlikely—then it is safe. They don’t expect that this path to failure may be extremely unlikely in a very unsafe design.
“Conjunction fallacy and probabilistic risk assessment” is enough for a title; you can move the applause lights into the article.
Now, what exactly is that fallacy in the risk assessment? Seems like it is thinking that there is only one very precise path how things could go wrong and then (perhaps correctly) calculating the probability of that one very precise path and reporting the result as the total risk: “This is the only way things could fail, and it’s probability is low, so everything is safe.”
EDIT: After reading the articles, there are many different fallacies combined, many of them probably caused by motivated cognition—we want the project to succeed, and we want it now.
That’s what I thought too, but the people who buy into this sort of assessment, they do acknowledge that it can be a bit off and that there could be other scenarios. They still think that the overall risk is somewhere in the few per million years range, and I never could quite get why. Now my theory is that due to the conjunction fallacy they see those highly detailed plausible paths to failure as the likely way it would fail, and then if the likely way it could fail is so unlikely—then it is safe. They don’t expect that this path to failure may be extremely unlikely in a very unsafe design.