The amount of control necessary for an ASI to preserve goal-directed subsystems against the constant push of evolutionary forces is strictly greater than the maximum degree of control available to any system of any type.
To answer it takes careful reasoning. Here’s my take on it:
We need to examine the degree to which there would be necessarily changesto the connected functional components constituting self-sufficient learning machinery (as including ASI)
Changes by learning/receiving code through environmental inputs, and through introduced changes in assembled molecular/physical configurations (of the hardware).
Necessary in the sense of “must change to adapt (such to continue to exist as self-sufficient learning machinery),” or “must change because of the nature of being in physical interactions (with/in the environment over time).”
We need to examine how changes to the connected functional components result in shiftsin actual functionality (in terms of how the functional components receive input signals and process those into output signals that propagate as effects across surrounding contexts of the environment).
We need to examine the span of evolutionary selection (covering effects that in their degrees/directivity feed back into the maintained/increased existence of any functional component).
We need to examine the span of control-based selection (the span covering detectable, modellable simulatable, evaluatable, and correctable effects).
I agree that point 5 is the main crux:
To answer it takes careful reasoning. Here’s my take on it:
We need to examine the degree to which there would be necessarily changes to the connected functional components constituting self-sufficient learning machinery (as including ASI)
Changes by learning/receiving code through environmental inputs, and through introduced changes in assembled molecular/physical configurations (of the hardware).
Necessary in the sense of “must change to adapt (such to continue to exist as self-sufficient learning machinery),” or “must change because of the nature of being in physical interactions (with/in the environment over time).”
We need to examine how changes to the connected functional components result in shifts in actual functionality (in terms of how the functional components receive input signals and process those into output signals that propagate as effects across surrounding contexts of the environment).
We need to examine the span of evolutionary selection (covering effects that in their degrees/directivity feed back into the maintained/increased existence of any functional component).
We need to examine the span of control-based selection (the span covering detectable, modellable simulatable, evaluatable, and correctable effects).