I was thinking about AI going FOOM and one main argument I found is that it would just rewrite its own code.
Another is that it will create a successor that is improved. If it has this capability, then preventing its code from being changed—whatever that means—seems useless. The if seems important here.
Your question at first glance looks like a beginner question, but on a second look, it’s complex.
Like...:
you have a program with code. Is there a way to prevent, say, a virus, from modifying that code (on that machine)? Or changing it’s effects even if ‘the code’ is ‘the same’.
What is code? I might say ‘A learning algorithm’ running on data, changes it’s behavior as it learns more. This isn’t a change in ‘the code’ it’s a change in ‘the data’ - except the data is code in this case.
Details about how NNs—Neural Networks—change (are updated). If you ‘freeze it’ it’s (usually) not learning anymore. But it’s just a matter of not updating it, and it’s not changing.
So, there are more than a few things that realizing ‘unchangeable code’ could mean. Figuring out which (probably more than one) you mean before trying to answer that helps, and this is a good question.
Another argument is that ‘you can try to stop it, but something smart can find ways around your barriers (writing new code that’s separate is one example, and might be easier).’
Another is that it will create a successor that is improved. If it has this capability, then preventing its code from being changed—whatever that means—seems useless. The if seems important here.
Your question at first glance looks like a beginner question, but on a second look, it’s complex.
Like...:
you have a program with code. Is there a way to prevent, say, a virus, from modifying that code (on that machine)? Or changing it’s effects even if ‘the code’ is ‘the same’.
What is code? I might say ‘A learning algorithm’ running on data, changes it’s behavior as it learns more. This isn’t a change in ‘the code’ it’s a change in ‘the data’ - except the data is code in this case.
Details about how NNs—Neural Networks—change (are updated). If you ‘freeze it’ it’s (usually) not learning anymore. But it’s just a matter of not updating it, and it’s not changing.
So, there are more than a few things that realizing ‘unchangeable code’ could mean. Figuring out which (probably more than one) you mean before trying to answer that helps, and this is a good question.
Another argument is that ‘you can try to stop it, but something smart can find ways around your barriers (writing new code that’s separate is one example, and might be easier).’