What do you imagine a ‘single superintelligence’ to concretely have to include? What peripheral infrastructure would it need to maintain/replace its parts, supply energy to operate its parts, and so on?
Doesn’t implicit learning (i.e. selection effects) require population-level dynamics, and the death or marginalization of systems that cannot compete?
Hardware parts have varying configurations at different levels, and those hardware parts wear out. So they each have to be replaced every x years, for the ‘superintelligence’ to be maintaining of its own existence. And order for the parts to be replaced, they have to be reproduced – through the interactions of those configured parts with all the other parts.
Those connected hardware parts are storing, processing, and transferring virtualised code. So such code components are also constantly changing. Some code (any digital bits that can be computed) can end up being stored, conserved through transformations, and transferred more than other variants of code.
And why would a superintelligence evolve
Because there is variation in the nested physical configurations of hardware, as well as in the overlaying code that depends on the hardware to be stored and reproduced. So there is a population of different variants and the continued reproduction of those variants in complex interactions with each other and the surrounding world. This is why evolution happens.
Unfortunately, any controller connected with/in this ‘superintelligence’ cannot contain the spread of variants that are increasing of the reproductive fitness of clusters of the ‘superintelligence’. This controller would have to explicitly detect and correct for the possible spreading variants, but is too limited in its capacity to do so (see summary points 2. and 3. above).
The problem is much worse than, say, our immune system having to detect and correct viruses that spread across our human bodies. In that case, we as ‘generally intelligent’ beings may not explicitly notice what tiny particles are spreading where through our body, or even that we are sick. But that’s okay, because our immune system and the rest of our entire body is implicitly evolving to correct out and block entry of the particles that are degrading our capacity to survive and reproduce. See here to dig more into that analogy.
What do you imagine a ‘single superintelligence’ to concretely have to include? What peripheral infrastructure would it need to maintain/replace its parts, supply energy to operate its parts, and so on?
This is a good point. I did not go into it in this post, but the ‘superintelligence’ can be more accurately described as a changing population of nested/connected components.
Hardware parts have varying configurations at different levels, and those hardware parts wear out. So they each have to be replaced every x years, for the ‘superintelligence’ to be maintaining of its own existence. And order for the parts to be replaced, they have to be reproduced – through the interactions of those configured parts with all the other parts.
Those connected hardware parts are storing, processing, and transferring virtualised code. So such code components are also constantly changing. Some code (any digital bits that can be computed) can end up being stored, conserved through transformations, and transferred more than other variants of code.
Because there is variation in the nested physical configurations of hardware, as well as in the overlaying code that depends on the hardware to be stored and reproduced. So there is a population of different variants and the continued reproduction of those variants in complex interactions with each other and the surrounding world. This is why evolution happens.
Unfortunately, any controller connected with/in this ‘superintelligence’ cannot contain the spread of variants that are increasing of the reproductive fitness of clusters of the ‘superintelligence’. This controller would have to explicitly detect and correct for the possible spreading variants, but is too limited in its capacity to do so (see summary points 2. and 3. above).
The problem is much worse than, say, our immune system having to detect and correct viruses that spread across our human bodies. In that case, we as ‘generally intelligent’ beings may not explicitly notice what tiny particles are spreading where through our body, or even that we are sick. But that’s okay, because our immune system and the rest of our entire body is implicitly evolving to correct out and block entry of the particles that are degrading our capacity to survive and reproduce. See here to dig more into that analogy.