I think this would be very interesting to follow-up (at a more reasonable hour of the day).
So, Dario indeed seems not to believe in a quick ASI takeover, and in this sense his definition does seem to differ from yours.
But the question is how does this decompose into differences on:
inherently achievable levels of intelligence
inherent resistance of the world to changes induced by super-high levels of intelligence
ability to have those super-high levels of intelligence and the presence of necessary affordances for radical changes (including the ASI takeover), but also the ability to agree to voluntary curtail the extent of those changes (including refraining from a “true takeover”)
My guess (which might be incorrect) is that your main differences with Dario’s viewpoint are on 2), and to some extent perhaps on 3), but less so on 1). So I think it’s worth a follow-up.
I think this would be very interesting to follow-up (at a more reasonable hour of the day).
So, Dario indeed seems not to believe in a quick ASI takeover, and in this sense his definition does seem to differ from yours.
But the question is how does this decompose into differences on:
inherently achievable levels of intelligence
inherent resistance of the world to changes induced by super-high levels of intelligence
ability to have those super-high levels of intelligence and the presence of necessary affordances for radical changes (including the ASI takeover), but also the ability to agree to voluntary curtail the extent of those changes (including refraining from a “true takeover”)
My guess (which might be incorrect) is that your main differences with Dario’s viewpoint are on 2), and to some extent perhaps on 3), but less so on 1). So I think it’s worth a follow-up.
(Thanks for the post, it’s very interesting.)