I don’t really track p(doom) that closely (or think that much about it) compared to timelines and takeoff speeds. Overall over the past few years it’s been one piece of bad news after another on the governance side, mostly, but on average good news on the technical side. At least that’s my opinion. I usually say it roughly cancels out in my mind. Basically, AGI is going to be built in more haste, by less competent people, with less wiggle room to make it safe, than I expected in 2022, but on the bright side, we’ve made steady progress in various forms of interpretability and it seems like we have a decent shot at getting a roughly human-level automated alignment researcher that’s nonscheming / honest / etc. and hopefully they’ll be able to solve the rest of the problem for us fast enough.
Do you think it would be useful to contrast your views with those of Yudkowksy/Soares/MIRI? Both parties seem largely aligned on the direction of the efforts that need to be made, with the MIRI folks making a more fundamental conditional statement about alignment, and yours seeming less pessimistic about its tractability given the potential for bootstrapping alignment via automated researchers, which is the crux of MIRI’s catch-22 view on the matter
I don’t really track p(doom) that closely (or think that much about it) compared to timelines and takeoff speeds. Overall over the past few years it’s been one piece of bad news after another on the governance side, mostly, but on average good news on the technical side. At least that’s my opinion. I usually say it roughly cancels out in my mind. Basically, AGI is going to be built in more haste, by less competent people, with less wiggle room to make it safe, than I expected in 2022, but on the bright side, we’ve made steady progress in various forms of interpretability and it seems like we have a decent shot at getting a roughly human-level automated alignment researcher that’s nonscheming / honest / etc. and hopefully they’ll be able to solve the rest of the problem for us fast enough.
Do you think it would be useful to contrast your views with those of Yudkowksy/Soares/MIRI? Both parties seem largely aligned on the direction of the efforts that need to be made, with the MIRI folks making a more fundamental conditional statement about alignment, and yours seeming less pessimistic about its tractability given the potential for bootstrapping alignment via automated researchers, which is the crux of MIRI’s catch-22 view on the matter
That seems like a reasonable summary to me of the contrast.