“very bad” = “a significant fraction of humanity died, or something comparable to that”, because we’re talking about x-risk, and the key claim is something like “humanity does better at higher-stakes problems”.
I also apply this view mainly to accidents and not to conflict—we didn’t prevent World War 2, which killed ~2% of humans. (The reason for the distinction is simple—to a first approximation, it’s in everyone’s interest to prevent accidents; it may not be in everyone’s interest to prevent conflicts.)
“very bad” = “a significant fraction of humanity died, or something comparable to that”, because we’re talking about x-risk, and the key claim is something like “humanity does better at higher-stakes problems”.
I also apply this view mainly to accidents and not to conflict—we didn’t prevent World War 2, which killed ~2% of humans. (The reason for the distinction is simple—to a first approximation, it’s in everyone’s interest to prevent accidents; it may not be in everyone’s interest to prevent conflicts.)