Who says that the ability to modify oneself is also the ability to modify oneself arbitrarily? What’s the difference between an AI knowing what its source code is, and being able to execute code that it writes, and an AI that is able to modify its own code?
If we create an AI that is as smart as us and has all of our knowledge, then we have created an AI with the power to develop at least an equally powerful AI. Why should we think that modifying such an AI would be better if done by us than by itself?
Who says that the ability to modify oneself is also the ability to modify oneself arbitrarily? What’s the difference between an AI knowing what its source code is, and being able to execute code that it writes, and an AI that is able to modify its own code?
If we create an AI that is as smart as us and has all of our knowledge, then we have created an AI with the power to develop at least an equally powerful AI. Why should we think that modifying such an AI would be better if done by us than by itself?