“I’m worried about AI risk” is not any less vague than “P(doom) = 50%”.
I think this misses the point. “I am worried about AI risk” doesn’t seem to give itself any undue weight: it doesn’t feign being more confident or certain of what the risk is.
If your actions can change the probability of an outcome, then you can’t meaningfully have a belief about the probability.
I think the problem getting at is that people’s idea of P(doom) is often really P(doom|inaction) not a cumulative estimate of probability over the full set of probable actions and all interactions. It is frequently given implicitly assuming no interactions with people’s discussions or reactions to risk.
I think the problem getting at is that people’s idea of P(doom) is often really P(doom|inaction) not a cumulative estimate of probability over the full set of probable actions and all interactions.
I don’t think it matters for almost anyone? Like the difference between my best-case future choices and worse-case future choices probably changes P(doom) by less than 0.01 percentage points.
Yeah I would also say that, but if I put in the maximum possible effort into causing vs. preventing international action, the change in probability of whether international action happens is pretty small. So my P(doom) given my own actions being maximally good is approximately identical to my P(doom) given my own actions being maximally bad.
I think this misses the point. “I am worried about AI risk” doesn’t seem to give itself any undue weight: it doesn’t feign being more confident or certain of what the risk is.
I think the problem getting at is that people’s idea of P(doom) is often really P(doom|inaction) not a cumulative estimate of probability over the full set of probable actions and all interactions. It is frequently given implicitly assuming no interactions with people’s discussions or reactions to risk.
I don’t think it matters for almost anyone? Like the difference between my best-case future choices and worse-case future choices probably changes P(doom) by less than 0.01 percentage points.
I think it matters for a lot of people. A lot of people would say the odds are substantially lower given international action.
Yeah I would also say that, but if I put in the maximum possible effort into causing vs. preventing international action, the change in probability of whether international action happens is pretty small. So my P(doom) given my own actions being maximally good is approximately identical to my P(doom) given my own actions being maximally bad.