Given that read-only hardware exists, yes, a clean border can be drawn, with the caveat that nothing is stopping the intelligence from emulating itself as if it were modified.
However—and it’s an important however—emulating your own modified code isn’t the same as modifying yourself. Just because you can imagine what your thought processes might be if you were sociopathic doesn’t make you sociopathic; just because an AI can emulate a process to arrive at a different answer than it would have doesn’t necessarily give it the power to -act- on that answer.
Which is to say, emulation can allow an AI to move past blocks on what it is permitted to think, but doesn’t necessarily permit it to move past blocks on what it is permitted to do.
This is particularly important in the case of something like a goal system; if a bug would result in an AI breaking its own goal system on a self-modification, this bug becomes less significant if the goal system is read-only. It could emulate what it would do with a different goal system, but it would be evaluating solutions from that emulation within its original goal system.
Given that read-only hardware exists, yes, a clean border can be drawn, with the caveat that nothing is stopping the intelligence from emulating itself as if it were modified.
However—and it’s an important however—emulating your own modified code isn’t the same as modifying yourself. Just because you can imagine what your thought processes might be if you were sociopathic doesn’t make you sociopathic; just because an AI can emulate a process to arrive at a different answer than it would have doesn’t necessarily give it the power to -act- on that answer.
Which is to say, emulation can allow an AI to move past blocks on what it is permitted to think, but doesn’t necessarily permit it to move past blocks on what it is permitted to do.
This is particularly important in the case of something like a goal system; if a bug would result in an AI breaking its own goal system on a self-modification, this bug becomes less significant if the goal system is read-only. It could emulate what it would do with a different goal system, but it would be evaluating solutions from that emulation within its original goal system.