Unknown, If you showed me a proof that I was fully deterministic, that wouldn’t remove my perception that I didn’t know what my future choices were. My lack of knowledge about my future choices doesn’t make the proof invalid, even to me. Why wouldn’t the AI understand such a proof?
If I show an oncologist an MRI of his own head and he sees a brain tumor and knows he has a week to live, it doesn’t kill him. He has clear evidence of a process going on inside him, and he knows, generally, what the outcome of that process will be.
A GAI could look at a piece of source code and determine, within certain bounds, what the code will do, and what if anything it will do reliably. If the GAI determines that the source code will reliably ‘be friendly’, and then later discovers that it is examining its own source code, then it will have discovered that it is itself reliably friendly.
Note that it’s not required for an AI to judge friendliness before it knows it’s looking at itself, but putting it that way prevents us from expecting the AI to latch on to a sense od it own decisiveness the way a human would.
Unknown, If you showed me a proof that I was fully deterministic, that wouldn’t remove my perception that I didn’t know what my future choices were. My lack of knowledge about my future choices doesn’t make the proof invalid, even to me. Why wouldn’t the AI understand such a proof?
If I show an oncologist an MRI of his own head and he sees a brain tumor and knows he has a week to live, it doesn’t kill him. He has clear evidence of a process going on inside him, and he knows, generally, what the outcome of that process will be.
A GAI could look at a piece of source code and determine, within certain bounds, what the code will do, and what if anything it will do reliably. If the GAI determines that the source code will reliably ‘be friendly’, and then later discovers that it is examining its own source code, then it will have discovered that it is itself reliably friendly.
Note that it’s not required for an AI to judge friendliness before it knows it’s looking at itself, but putting it that way prevents us from expecting the AI to latch on to a sense od it own decisiveness the way a human would.