Every act of communication does omit information. You can’t translate an English book into French without at least adding or omitting some information because different languages map the semantic space differently.
Any translating from the internal way the AGI represents it’s knowledge towards English or another human readable format is going to omit some information.
Even in cases where most information is there such as in cases where a security researcher audits the source code of a computer program the researcher usually doesn’t find all bugs. That even true without the original programmer putting any intentional backdoors into the code.
That’s why I said “another human readable format”. Of course the AGI can also answer in a non human readable format but that would defeat the point of the exercise.
If you ask a contemporary Chess computer that suddenly gets self awareness why it chooses one move over another it might tell you: Well given heuristics X and Y that’s the outcome of my monte-carlo simulation, if you want here’s the machine code and the whole stack trace.
For practical purposes that doesn’t mean that the human understands the reasoning fully.
Every act of communication does omit information. You can’t translate an English book into French without at least adding or omitting some information because different languages map the semantic space differently.
Any translating from the internal way the AGI represents it’s knowledge towards English or another human readable format is going to omit some information.
Even in cases where most information is there such as in cases where a security researcher audits the source code of a computer program the researcher usually doesn’t find all bugs. That even true without the original programmer putting any intentional backdoors into the code.
I didn’t specify that questions or answers would be in English ;)
That’s why I said “another human readable format”. Of course the AGI can also answer in a non human readable format but that would defeat the point of the exercise.
If you ask a contemporary Chess computer that suddenly gets self awareness why it chooses one move over another it might tell you: Well given heuristics X and Y that’s the outcome of my monte-carlo simulation, if you want here’s the machine code and the whole stack trace.
For practical purposes that doesn’t mean that the human understands the reasoning fully.