unconditional P(doom) is 10%?! no wonder your numbers are so weird!
my unconditional P(doom) is more like 60% at the moment. it’s down from P(doom|no ai safety community) which is around 99%. Seems like a bit much to expect that safety acceleration could pull doom down to 10%.
My p(doom) is around 10% probably in part because I imagine a pretty slow takeoff. I don’t think 60% is necessarily unreasonable, nor 1% nor 99%. It’s hard to estimate from essentially zero actual information.
Can I ask why you think the AI safety community has been/will be so impactful (99% → 60%)? I think you believe the community has much more reach and power than it actually does.
Technical research trajectory, mostly; I see paths through current technical alignment research which might be able to pull a rabbit out of a hat, camp A style. Also some chance of slowdown but most of my success probability comes from the possible futures where current technical research hunches pan out and let us know important attributes of a learning system that let us be sure that running it results in mostly-good outcomes for most minds’ preferences, in some cev-ish sense. Mostly this depends on wizard power, not command power.
unconditional P(doom) is 10%?! no wonder your numbers are so weird!
my unconditional P(doom) is more like 60% at the moment. it’s down from P(doom|no ai safety community) which is around 99%. Seems like a bit much to expect that safety acceleration could pull doom down to 10%.
My p(doom) is around 10% probably in part because I imagine a pretty slow takeoff. I don’t think 60% is necessarily unreasonable, nor 1% nor 99%. It’s hard to estimate from essentially zero actual information.
Can I ask why you think the AI safety community has been/will be so impactful (99% → 60%)? I think you believe the community has much more reach and power than it actually does.
Technical research trajectory, mostly; I see paths through current technical alignment research which might be able to pull a rabbit out of a hat, camp A style. Also some chance of slowdown but most of my success probability comes from the possible futures where current technical research hunches pan out and let us know important attributes of a learning system that let us be sure that running it results in mostly-good outcomes for most minds’ preferences, in some cev-ish sense. Mostly this depends on wizard power, not command power.