I’m not sure why we’re arguing about whether a country of geniuses in a data center, as described by Dario, would qualify as superintelligence by my definition of it, when I am telling you that it wouldn’t. I agree that the definition I gave in my post does not conclusively rule that out, but 1) words are hard, man, 2) the rest of my post really clearly implies that I don’t think the thing he’s describing counts. So if you have a gripe, it should probably be with my definition. Sorry for being snippy.
Anyways, Dario doesn’t believe in my pointer to superintelligence, which I will decline to define further at this hour of the night, but I expect most people here to understand what I mean.
I think this would be very interesting to follow-up (at a more reasonable hour of the day).
So, Dario indeed seems not to believe in a quick ASI takeover, and in this sense his definition does seem to differ from yours.
But the question is how does this decompose into differences on:
inherently achievable levels of intelligence
inherent resistance of the world to changes induced by super-high levels of intelligence
ability to have those super-high levels of intelligence and the presence of necessary affordances for radical changes (including the ASI takeover), but also the ability to agree to voluntary curtail the extent of those changes (including refraining from a “true takeover”)
My guess (which might be incorrect) is that your main differences with Dario’s viewpoint are on 2), and to some extent perhaps on 3), but less so on 1). So I think it’s worth a follow-up.
I’m not sure why we’re arguing about whether a country of geniuses in a data center, as described by Dario, would qualify as superintelligence by my definition of it, when I am telling you that it wouldn’t. I agree that the definition I gave in my post does not conclusively rule that out, but 1) words are hard, man, 2) the rest of my post really clearly implies that I don’t think the thing he’s describing counts. So if you have a gripe, it should probably be with my definition. Sorry for being snippy.
Anyways, Dario doesn’t believe in my pointer to superintelligence, which I will decline to define further at this hour of the night, but I expect most people here to understand what I mean.
I think this would be very interesting to follow-up (at a more reasonable hour of the day).
So, Dario indeed seems not to believe in a quick ASI takeover, and in this sense his definition does seem to differ from yours.
But the question is how does this decompose into differences on:
inherently achievable levels of intelligence
inherent resistance of the world to changes induced by super-high levels of intelligence
ability to have those super-high levels of intelligence and the presence of necessary affordances for radical changes (including the ASI takeover), but also the ability to agree to voluntary curtail the extent of those changes (including refraining from a “true takeover”)
My guess (which might be incorrect) is that your main differences with Dario’s viewpoint are on 2), and to some extent perhaps on 3), but less so on 1). So I think it’s worth a follow-up.
(Thanks for the post, it’s very interesting.)