The AI could propose the following experiment for the GK to carry out. AI provides source code. GK puts code in a simulated box in a simulated world in a separate real box. GK allows sim-AI to convince him to let it out of the sim-box, and observes what it does in the sim-world. Since AI determines the results, AI can assert that the results are positive and to the GK’s liking.
Smallwood
Karma: 8
“how could you determine that the AI provided the actual source code...”
You don’t, you release the source code that you tested, and if the AI is playing nice then the code you tested IS the AI.