Somewhat. If it is known that the AI actually does not go into infinite loops, then this isn’t a problem—but this creates an interesting question as to how the AI is reasoning about the human’s behavior in a way that doesn’t lead to an infinite loop. One sort of answer we can give is that they’re doing logical reasoning about each other, rather than trying to run each other’s code. This could run into incompleteness problems, but not always:
Somewhat. If it is known that the AI actually does not go into infinite loops, then this isn’t a problem—but this creates an interesting question as to how the AI is reasoning about the human’s behavior in a way that doesn’t lead to an infinite loop. One sort of answer we can give is that they’re doing logical reasoning about each other, rather than trying to run each other’s code. This could run into incompleteness problems, but not always:
http://intelligence.org/files/ParametricBoundedLobsTheorem.pdf