Doesn’t the choice of a perfect external regulator amount to the same thing as directly imposing restrictions on yourself, thereby going back to the original problem? I suppose such a regulator, on indeed any stabilizing self-modification, could have the advantage of being publicly available and widely used, and therefore be well-tested and having thoroughly understood operations and consequences.
Another way to do it might be to create many copies of yourself (I’m assuming this scenario takes place inside a computer) and let majority (or 2/3s majority or etc) rule when it comes to “rescuing” copies that have made un-self-recoverable errors.
Anyway I suppose this is all somewhat beside the point since such a scenario was chosen as an example of what Eliezer expects a successful future to not look like.
@James:
Doesn’t the choice of a perfect external regulator amount to the same thing as directly imposing restrictions on yourself, thereby going back to the original problem? I suppose such a regulator, on indeed any stabilizing self-modification, could have the advantage of being publicly available and widely used, and therefore be well-tested and having thoroughly understood operations and consequences.
Another way to do it might be to create many copies of yourself (I’m assuming this scenario takes place inside a computer) and let majority (or 2/3s majority or etc) rule when it comes to “rescuing” copies that have made un-self-recoverable errors.
Anyway I suppose this is all somewhat beside the point since such a scenario was chosen as an example of what Eliezer expects a successful future to not look like.