Yeah, that’s mostly trying to point to evidence that he’s truly non-trivially worried about misuse risks, which necessarily reduces the share of catastrophic risk likelihood he attributes to misalignment.
Ok. Note that in most groups, worry about misuse risks (for a variety of definitions of misuse risks) is positively correlated rather than negatively correlated with your estimate of misalignment risk and how plausible you think it is that qualitative superintelligence is developed!
Of course you can be worried about multiple things.
If he has 25% on (unconditional) x-risk, whatever share of that is derived from worlds in which we die from “misuse” is therefore not derived worlds in which we die from misalignment. I guess it’s possible that he could be modeling 100% of misuse worlds as also dying to misalignment later in their timelines, so “necessarily” was wrong, but in practice...
Here is a metaphor; I can’t tell if this is insightful to anyone else: if one is worried about defeating Orcs and Ogres, the objective is not to bet on which of them is more dangerous; the objective is to simultaneously beat Orcs and Ogres.
Yeah, that’s mostly trying to point to evidence that he’s truly non-trivially worried about misuse risks, which
necessarilyreduces the share of catastrophic risk likelihood he attributes to misalignment.Ok. Note that in most groups, worry about misuse risks (for a variety of definitions of misuse risks) is positively correlated rather than negatively correlated with your estimate of misalignment risk and how plausible you think it is that qualitative superintelligence is developed!
Weird frame. I can worry about more than one thing. Admittedly a lot of others can’t.
Of course you can be worried about multiple things.
If he has 25% on (unconditional) x-risk, whatever share of that is derived from worlds in which we die from “misuse” is therefore not derived worlds in which we die from misalignment. I guess it’s possible that he could be modeling 100% of misuse worlds as also dying to misalignment later in their timelines, so “necessarily” was wrong, but in practice...
Oh, I see the confusion. (My fault for finishing this post at 11 pm.)
I think that bullet point requires the same disclaimer as the third bullet point. Might edit that in the morning.
Here is a metaphor; I can’t tell if this is insightful to anyone else: if one is worried about defeating Orcs and Ogres, the objective is not to bet on which of them is more dangerous; the objective is to simultaneously beat Orcs and Ogres.