This was produced as a part of the AI Safety Camp 2026 “Assumptions of the Doom Debate” project, led by Sean Herrington, who was also the lead author on this post. The other participants have equal contributions and are listed in no particular order. It is the first in a sequence we intend to publish over the coming weeks.
This breakdown is in a tree format, and we allow people to set their own probabilities for each threat pathway
You can use this to drill down into components of your P(Doom)
You can analyse the sensitivity of your beliefs to changes in your assumptions
You can compare your worldview to others’ and find cruxes automatically
The exercise of creating this sort of structure is valuable as a way to think about the future in a more general manner.
Introduction
Just about everyone in the AI community seems to disagree about the risks of the technology. People disagree on the likelihood (Yann LeCun: <0.01% chance of extinction; Roman Yampolskiy: >99.99%), the worst threat (AI takeover vs concentration of power vs gradual disempowerment vs bioweapons vs …), the timelines and many other things. It seems like a shared framework would be helpful in this capacity.
The main issue that comes with creating such a thing is that it requires everyone to agree on it. We call this requirement “worldview independence”; it significantly narrows down the space of options for the shape this framework can take. In particular, every part of the structure has to be a mathematical statement; those are some of the only things people can agree on.
The structure we eventually ended on looks like this:
Breaking it down:
E: Event that we’re worried about (Nuclear war, Bioweapon is released, Human extinction, etc)
A: Set of worlds, e.g., “AI-driven worlds” = “credence that we live in a world where AI is making E more likely.”
¬A: Set of worlds which are not in A, e.g., “Non-AI worlds” = “credence that we live in a world where AI is not making E more likely”
P(E|A): Probability that event E happens in worlds A.
On the website, we also have a timeframe, such that we are actually talking about event E happening “within 30 years”, for example. Putting real values in looks like this:
In this case, the thing we want to know is our probability of an existential catastrophe occurring within the next 30 years. We split this into AI-caused and not AI-caused.[1] You can then set your probabilities for each of these and calculate your overall final probability that an existential catastrophe happens.
This becomes significantly more interesting once it gets scaled, as it enables you to break the future down into individually visualisable threat paths. We’ve also found as a group that the exercise of creating this tree has helped us see a range of possibilities we had not considered.
For instance, we hadn’t previously thought much about scenarios where powerful and otherwise aligned AIs made mistakes, but after further thought, consider it significantly more likely.[2] This insight was directly generated by the thought that while trying to split worlds apart, we were implicitly assuming that dangerous AI scenarios were caused by AIs which wanted these scenarios to come about.
User Guide
Labels
Labels are simply changes that can be made to the text in the tree, allowing you to change what you’re talking about. They appear in the left sidebar, next to the tree:
We have 4 different labels you can apply to your tree.
Label
Description
Danger
Changing the danger label is equivalent to changing what you are talking about: some people want to ask whether we’re going to have an “existential catastrophe”; for others, the question is whether “everyone dies” or whether “a bioweapon gets deployed”.
Timeframe
One of the biggest debates in the AI community is one of timelines. Our tree is therefore based around the question of whether the danger occurs within the timeframe of “x years”. Use this label to change which timeframe you are discussing.
Author
Put your name here to show people that it’s your worldview rather than anyone else’s.
Perspective
You can use this to change whether you are talking about your inside view (where you are basing your opinion on a gears-level internal model) oryour outside view (where you also include meta-level factors outside that model).
Base Tree
Our base tree has 4 different branching points, which we’ll explain briefly here.
Branch
Description
AI driven/Not
Separates out worlds in which AI worsens the danger from those in which it doesn’t.
Single AI/Multipolar (AI driven world)
Separates out worlds where most of the danger comes from a single AI from worlds where it comes from multiple.
Internal Model/Not (Single AI Danger world)
Separates out worlds where the single AI causing the most danger is an AI with an internal model of the danger it’s causing from one where it doesn’t. An example of a dangerous system without a model could include (for instance) a simple missile detection system entrusted by a nuclear power with retaliation. The main question here is, “How likely is it that the AI knows what it’s doing?”
Expects danger/doesn’t (Internal Model world)
Splits worlds where the single most dangerous AI expects the danger it is causing to occur (such as in traditional misaligned superintelligence narratives) from those in which it does not (for instance, because the AI has itself made a mistake).
Tree features
Pin branch overrides
If you do not want to set all of the nodes in the tree, you can override the leaves by simply moving the probabilities higher up the tree. You will still be able to edit the probabilities further down, but they will no longer affect your root probability.
Clicking on the triangle symbol allows you to switch between the set probability for that node and the propagated value for the nodes further down the tree.
Collapse tree
The +/- icons in the corner of the nodes of the tree can be used to expand and collapse the tree below them.
Sub-branch probabilities
Each node below the top section of the tree has two numbers: The main one is the actual probability of this world existing and causing an AI catastrophe.
P(We are in a single dominant AI world AND Existential Catastrophe occurs)
The number below is the same, but conditioned on us already being in that branch of the tree.
P(We are in a single dominant AI world AND Existential Catastrophe occurs GIVEN THAT we’re in an AI-driven world)
Connector thickness
The lines between nodes get thicker for branches carrying more probability mass through worldspace. This is a representation of how the worlds are split, rather than representing probabilities of danger.
Sensitivity Analysis
The sensitivity analysis is the derivative of your overall probability by each individual value you’ve set. It allows you to see where changing your mind would be the most significant.
Crux Analysis
Different people have different worldviews. Crux analysis is a way to see which facts are the most important sources of disagreement.
We are comparing 2 worldviews, A and B. These can either be preset or user-saved.
For each node, there are 2 bars:
The green bar is how much A’s final probability would move towards B’s if B convinced them of their worldview in that particular node
The amber bar is how much B’s would change were A to convince them.
Bars going to the right close the gap in root probabilities, while those going to the left open them.[3]
For instance, in the picture above, B convincing A that multipolar AI means a 15% probability of existential catastrophe rather than 10% would move A’s overall probability by 2.2 percentage points towards B’s.
A convincing B to lower their probability to 10% rather than 15% would move B’s overall probability by 2.4 percentage points towards A’s.
Ranges
We have added ranges to the site. This allows for a representation of your uncertainty in the final result: if your current probability estimate is 60% for something, but it seems plausible that on another day you might put 50 or 70, it can be useful to know how that would affect the final result.
We currently have 2 range propagation modes:
Worst case: treats the ranges as maximum and minimum possible probabilities
Independent: treats the ranges as 10th and 90th percentiles and performs a Monte Carlo simulation[4] to establish subsequent ranges.
We would generally recommend using independent propagation as ‘better’, but the worst case can also be useful as a view of the full range of possibilities you see as plausible.
Uncertainty Reduction
When using ranges, you get the additional option of using uncertainty reduction.
This shows you how much reducing your uncertainty on individual nodes reduces your overall uncertainty.
Other features
Feature
Description
Worldviews
The tree contains some preset worldviews, which are AI-generated estimates of prominent figures’ worldviews, inspired by their public statements. Please note these are not endorsed by the actual figures and are there for interest purposes only.
Share
You can share worldviews with friends. Click on the button to get a shareable link they can then paste in their browser.
Save
Save this worldview as a JSON file and to local memory. Saved worldviews persist across sessions.
It’s hard to define this precisely, and we’ve had a bunch of debates about this ourselves. The basic intuition is, for the example E=Existential risk, “How likely do you think it is that we live in a timeline where AI will overall contribute to existential risk (for instance, because it is misaligned) vs helping to mitigate it (via, e.g., helping to solve problems associated with climate change)?”
For a brief intuition here, consider this statement from the perspective of a chimpanzee: “Humans are smart enough to go to the moon, change animals’ DNA and communicate instantly across the globe. There’s no way they’d be dumb enough to cause climate change; they live on this planet like we do!”
An example of how agreeing on one node can make your final probabilities diverge: A is broadly more pessimistic than B, except on one node. Agreeing with B on that node would make A even more pessimistic, taking their overall probability away from B’s.
We made a map of the doom debate
This was produced as a part of the AI Safety Camp 2026 “Assumptions of the Doom Debate” project, led by Sean Herrington, who was also the lead author on this post. The other participants have equal contributions and are listed in no particular order. It is the first in a sequence we intend to publish over the coming weeks.
TL;DR:
We have created a breakdown of AI threat pathways, which can be accessed at https://lifeuniversesafety.com/doom-assumptions/index.html
This breakdown is in a tree format, and we allow people to set their own probabilities for each threat pathway
You can use this to drill down into components of your P(Doom)
You can analyse the sensitivity of your beliefs to changes in your assumptions
You can compare your worldview to others’ and find cruxes automatically
The exercise of creating this sort of structure is valuable as a way to think about the future in a more general manner.
Introduction
Just about everyone in the AI community seems to disagree about the risks of the technology. People disagree on the likelihood (Yann LeCun: <0.01% chance of extinction; Roman Yampolskiy: >99.99%), the worst threat (AI takeover vs concentration of power vs gradual disempowerment vs bioweapons vs …), the timelines and many other things. It seems like a shared framework would be helpful in this capacity.
The main issue that comes with creating such a thing is that it requires everyone to agree on it. We call this requirement “worldview independence”; it significantly narrows down the space of options for the shape this framework can take. In particular, every part of the structure has to be a mathematical statement; those are some of the only things people can agree on.
The structure we eventually ended on looks like this:
Breaking it down:
E: Event that we’re worried about (Nuclear war, Bioweapon is released, Human extinction, etc)
A: Set of worlds, e.g., “AI-driven worlds” = “credence that we live in a world where AI is making E more likely.”
¬A: Set of worlds which are not in A, e.g., “Non-AI worlds” = “credence that we live in a world where AI is not making E more likely”
P(E|A): Probability that event E happens in worlds A.
On the website, we also have a timeframe, such that we are actually talking about event E happening “within 30 years”, for example. Putting real values in looks like this:
In this case, the thing we want to know is our probability of an existential catastrophe occurring within the next 30 years. We split this into AI-caused and not AI-caused.[1] You can then set your probabilities for each of these and calculate your overall final probability that an existential catastrophe happens.
This becomes significantly more interesting once it gets scaled, as it enables you to break the future down into individually visualisable threat paths. We’ve also found as a group that the exercise of creating this tree has helped us see a range of possibilities we had not considered.
For instance, we hadn’t previously thought much about scenarios where powerful and otherwise aligned AIs made mistakes, but after further thought, consider it significantly more likely.[2] This insight was directly generated by the thought that while trying to split worlds apart, we were implicitly assuming that dangerous AI scenarios were caused by AIs which wanted these scenarios to come about.
User Guide
Labels
Labels are simply changes that can be made to the text in the tree, allowing you to change what you’re talking about. They appear in the left sidebar, next to the tree:
We have 4 different labels you can apply to your tree.
Label
Description
Danger
Changing the danger label is equivalent to changing what you are talking about: some people want to ask whether we’re going to have an “existential catastrophe”; for others, the question is whether “everyone dies” or whether “a bioweapon gets deployed”.
Timeframe
One of the biggest debates in the AI community is one of timelines. Our tree is therefore based around the question of whether the danger occurs within the timeframe of “x years”. Use this label to change which timeframe you are discussing.
Author
Put your name here to show people that it’s your worldview rather than anyone else’s.
Perspective
You can use this to change whether you are talking about your inside view (where you are basing your opinion on a gears-level internal model) or your outside view (where you also include meta-level factors outside that model).
Base Tree
Our base tree has 4 different branching points, which we’ll explain briefly here.
Branch
Description
AI driven/Not
Separates out worlds in which AI worsens the danger from those in which it doesn’t.
Single AI/Multipolar (AI driven world)
Separates out worlds where most of the danger comes from a single AI from worlds where it comes from multiple.
Internal Model/Not (Single AI Danger world)
Separates out worlds where the single AI causing the most danger is an AI with an internal model of the danger it’s causing from one where it doesn’t. An example of a dangerous system without a model could include (for instance) a simple missile detection system entrusted by a nuclear power with retaliation. The main question here is, “How likely is it that the AI knows what it’s doing?”
Expects danger/doesn’t (Internal Model world)
Splits worlds where the single most dangerous AI expects the danger it is causing to occur (such as in traditional misaligned superintelligence narratives) from those in which it does not (for instance, because the AI has itself made a mistake).
Tree features
Pin branch overrides
If you do not want to set all of the nodes in the tree, you can override the leaves by simply moving the probabilities higher up the tree. You will still be able to edit the probabilities further down, but they will no longer affect your root probability.
Clicking on the triangle symbol allows you to switch between the set probability for that node and the propagated value for the nodes further down the tree.
Collapse tree
The +/- icons in the corner of the nodes of the tree can be used to expand and collapse the tree below them.
Sub-branch probabilities
Each node below the top section of the tree has two numbers: The main one is the actual probability of this world existing and causing an AI catastrophe.
P(We are in a single dominant AI world AND Existential Catastrophe occurs)
The number below is the same, but conditioned on us already being in that branch of the tree.
P(We are in a single dominant AI world AND Existential Catastrophe occurs GIVEN THAT we’re in an AI-driven world)
Connector thickness
The lines between nodes get thicker for branches carrying more probability mass through worldspace. This is a representation of how the worlds are split, rather than representing probabilities of danger.
Sensitivity Analysis
The sensitivity analysis is the derivative of your overall probability by each individual value you’ve set. It allows you to see where changing your mind would be the most significant.
Crux Analysis
Different people have different worldviews. Crux analysis is a way to see which facts are the most important sources of disagreement.
We are comparing 2 worldviews, A and B. These can either be preset or user-saved.
For each node, there are 2 bars:
The green bar is how much A’s final probability would move towards B’s if B convinced them of their worldview in that particular node
The amber bar is how much B’s would change were A to convince them.
Bars going to the right close the gap in root probabilities, while those going to the left open them.[3]
For instance, in the picture above, B convincing A that multipolar AI means a 15% probability of existential catastrophe rather than 10% would move A’s overall probability by 2.2 percentage points towards B’s.
A convincing B to lower their probability to 10% rather than 15% would move B’s overall probability by 2.4 percentage points towards A’s.
Ranges
We have added ranges to the site. This allows for a representation of your uncertainty in the final result: if your current probability estimate is 60% for something, but it seems plausible that on another day you might put 50 or 70, it can be useful to know how that would affect the final result.
We currently have 2 range propagation modes:
Worst case: treats the ranges as maximum and minimum possible probabilities
Independent: treats the ranges as 10th and 90th percentiles and performs a Monte Carlo simulation[4] to establish subsequent ranges.
We would generally recommend using independent propagation as ‘better’, but the worst case can also be useful as a view of the full range of possibilities you see as plausible.
Uncertainty Reduction
When using ranges, you get the additional option of using uncertainty reduction.
This shows you how much reducing your uncertainty on individual nodes reduces your overall uncertainty.
Other features
Feature
Description
Worldviews
The tree contains some preset worldviews, which are AI-generated estimates of prominent figures’ worldviews, inspired by their public statements. Please note these are not endorsed by the actual figures and are there for interest purposes only.
Share
You can share worldviews with friends. Click on the button to get a shareable link they can then paste in their browser.
Save
Save this worldview as a JSON file and to local memory. Saved worldviews persist across sessions.
Import
Import a JSON worldview file.
Reset
Reset all values to 50%
Delete
Delete a saved worldview
It’s hard to define this precisely, and we’ve had a bunch of debates about this ourselves. The basic intuition is, for the example E=Existential risk, “How likely do you think it is that we live in a timeline where AI will overall contribute to existential risk (for instance, because it is misaligned) vs helping to mitigate it (via, e.g., helping to solve problems associated with climate change)?”
For a brief intuition here, consider this statement from the perspective of a chimpanzee: “Humans are smart enough to go to the moon, change animals’ DNA and communicate instantly across the globe. There’s no way they’d be dumb enough to cause climate change; they live on this planet like we do!”
An example of how agreeing on one node can make your final probabilities diverge: A is broadly more pessimistic than B, except on one node. Agreeing with B on that node would make A even more pessimistic, taking their overall probability away from B’s.
The individual probabilities are drawn from a beta distribution fit to the percentile range