The set of interesting accomplishments he expects from that country of geniuses in a data center, in that essay, are not the things you get out of a superintelligence.
Edit: separately, I am not sure what your comment is supposed to be doing. This entire post is about how there are clearly different conceptions of the term “superintelligence”, and that, to the extent that he believes in any of them, he certainly doesn’t believe in anything like mine. Unless you mean to claim that I am wrong about how the thing he’s describing in MoLG is actually compatible with the kind of superintelligence I’m imagining?
separately, I am not sure what your comment is supposed to be doing.
My comment is a pretty neutral response to the central claim ‘Dario probably doesn’t believe in superintelligence’ which you specify you believe in and isn’t just a clickbait headline, and to the arguments for it. Do you react like this to all comments which disagree with you? May I suggest to just comment on the arguments in the comment than to have such a kneejerk reaction?
> Unless you mean to claim that I am wrong about how the thing he’s describing in MoLG is actually compatible with the kind of superintelligence I’m imagining?
Your definition is:
> Roughly speaking, that the returns to intelligence past the human level are large, in terms of the additional affordances they would grant for steering the world, and that it is practical to get that additional intelligence into a system.
As I said I think ‘a country of supergeniuses in a datacenter’ fits, yes:
>A country of geniuses in a datacenter is pretty clearly “Superintelligence” and he pretty clearly believes in it. He seems to rather belive that Superintelligence wouldn’t solve everything quite as quick as others think.
I’m not sure why we’re arguing about whether a country of geniuses in a data center, as described by Dario, would qualify as superintelligence by my definition of it, when I am telling you that it wouldn’t. I agree that the definition I gave in my post does not conclusively rule that out, but 1) words are hard, man, 2) the rest of my post really clearly implies that I don’t think the thing he’s describing counts. So if you have a gripe, it should probably be with my definition. Sorry for being snippy.
Anyways, Dario doesn’t believe in my pointer to superintelligence, which I will decline to define further at this hour of the night, but I expect most people here to understand what I mean.
I think this would be very interesting to follow-up (at a more reasonable hour of the day).
So, Dario indeed seems not to believe in a quick ASI takeover, and in this sense his definition does seem to differ from yours.
But the question is how does this decompose into differences on:
inherently achievable levels of intelligence
inherent resistance of the world to changes induced by super-high levels of intelligence
ability to have those super-high levels of intelligence and the presence of necessary affordances for radical changes (including the ASI takeover), but also the ability to agree to voluntary curtail the extent of those changes (including refraining from a “true takeover”)
My guess (which might be incorrect) is that your main differences with Dario’s viewpoint are on 2), and to some extent perhaps on 3), but less so on 1). So I think it’s worth a follow-up.
The set of interesting accomplishments he expects from that country of geniuses in a data center, in that essay, are not the things you get out of a superintelligence.
Edit: separately, I am not sure what your comment is supposed to be doing. This entire post is about how there are clearly different conceptions of the term “superintelligence”, and that, to the extent that he believes in any of them, he certainly doesn’t believe in anything like mine. Unless you mean to claim that I am wrong about how the thing he’s describing in MoLG is actually compatible with the kind of superintelligence I’m imagining?
My comment is a pretty neutral response to the central claim ‘Dario probably doesn’t believe in superintelligence’ which you specify you believe in and isn’t just a clickbait headline, and to the arguments for it. Do you react like this to all comments which disagree with you? May I suggest to just comment on the arguments in the comment than to have such a kneejerk reaction?
> Unless you mean to claim that I am wrong about how the thing he’s describing in MoLG is actually compatible with the kind of superintelligence I’m imagining?
Your definition is:
> Roughly speaking, that the returns to intelligence past the human level are large, in terms of the additional affordances they would grant for steering the world, and that it is practical to get that additional intelligence into a system.
As I said I think ‘a country of supergeniuses in a datacenter’ fits, yes:
>A country of geniuses in a datacenter is pretty clearly “Superintelligence” and he pretty clearly believes in it. He seems to rather belive that Superintelligence wouldn’t solve everything quite as quick as others think.
I’m not sure why we’re arguing about whether a country of geniuses in a data center, as described by Dario, would qualify as superintelligence by my definition of it, when I am telling you that it wouldn’t. I agree that the definition I gave in my post does not conclusively rule that out, but 1) words are hard, man, 2) the rest of my post really clearly implies that I don’t think the thing he’s describing counts. So if you have a gripe, it should probably be with my definition. Sorry for being snippy.
Anyways, Dario doesn’t believe in my pointer to superintelligence, which I will decline to define further at this hour of the night, but I expect most people here to understand what I mean.
I think this would be very interesting to follow-up (at a more reasonable hour of the day).
So, Dario indeed seems not to believe in a quick ASI takeover, and in this sense his definition does seem to differ from yours.
But the question is how does this decompose into differences on:
inherently achievable levels of intelligence
inherent resistance of the world to changes induced by super-high levels of intelligence
ability to have those super-high levels of intelligence and the presence of necessary affordances for radical changes (including the ASI takeover), but also the ability to agree to voluntary curtail the extent of those changes (including refraining from a “true takeover”)
My guess (which might be incorrect) is that your main differences with Dario’s viewpoint are on 2), and to some extent perhaps on 3), but less so on 1). So I think it’s worth a follow-up.
(Thanks for the post, it’s very interesting.)