Sure, but that’s equivalent to just not showing him the GLUT in the first place. All you’ve done is move the problem one level up. There is still a circumstance in which the GLUT cannot reliably predict his action, namely that where you show him an accurate GLUT.
Where did you get the accurate GLUT? Before you had an accurate GLUT , you couldn’t show him the accurate GLUT which would be needed to calculate the accurate GLUT.
Sure, you can calculate the GLUT for all possible inputs, some of which may be accurate GLUT inputs, but then you have to solve the problem of finding which of all possible inputs are accurate GLUT inputs. To answer the question, someone has to do calculations above and beyond just simulating a human.
To be concrete suppose the input you give to the subject is a finite string of bits of length N. The GLUT is computed by simulating the output produced by each of the 2^N possible inputs, observing the result, and writing it down. Now we want to know what happens when we encode the GLUT as a bit string of length N and giving it to the subject as an input. I think the answer is that in general we can’t do this, because to encode all 2^N entries of the GLUT clearly requires far more than N bits, so the GLUT is too big to be a valid input to the simulation. So it seems an accurate GLUT is too big for us to be able to show it to the subject. Maybe this is one answer to the question.
But maybe for some subjects we can do enough compression on the GLUT that we can fit it into N bits. As a trivial example, if the input string is large but the subject’s behavior is simply to say “yes” for any input, then we can show him the accurate compressed GLUT: “For any input, the subject will say ‘yes.’”
Less trivially, we could think of the simulator program itself as a highly compressed GLUT. And maybe the simulator executable is a mere 100 MB file, while the simulation accepts 100 GB of input. Then we could supply the simulated subject with the executable that is running him as an accurate compressed GLUT.
Suppose the simulated subject is a contrarian and tries to prove that the compressed GLUT we gave him is not accurate. He loads the executable onto a simulated computer and runs the executable on itself to see what it will output. Of course, it creates a simulated simulated subject who takes the executable and loads it onto a simulated simulated computer to see what it will output. This creates a simulated simulated simulated subject...
So this simulation will never halt when given this input. Presumably an accurate uncompressed GLUT would have an entry reading “Subject is supplied with his own source code → simulation does not halt.”
So maybe we can indeed connect to the halting problem?
So it seems an accurate GLUT is too big for us to be able to show it to the subject. Maybe this is one answer to the question.
Very likely; but I was only going to show him the bits predicting his response to his immediate situation + being shown those bits. We can save a bit of paper by not printing out his predicted reaction to chocolate elephants falling from the sky. :)
So maybe we can indeed connect to the halting problem?
It seems to me that your construction relies on the subject having access to the executable. If we don’t give him that access, he cannot use this method of attempted disproof, no matter how contrarian he is.
I was only going to show him the bits predicting his response to his immediate situation + being shown those bits.
OK, but again this simply may not be possible even if you have an accurate GLUT. If you give him anything that lets him compute your true prediction in finite time, then he can compute your true prediction and then do the opposite. Even if we have a complete and accurate GLUT, we can never supply him with a true prediction if the accurate GLUT contains no entries of the form “Subject is told he is predicted to do X” → subject does X.
Well, that’s precisely my point. But see prase’s comment below, with the very interesting point that every sufficiently-nice function f(x) has some x for which f(x)=x. The question is whether the human brain is sufficiently nice.
Sure, but that’s equivalent to just not showing him the GLUT in the first place. All you’ve done is move the problem one level up. There is still a circumstance in which the GLUT cannot reliably predict his action, namely that where you show him an accurate GLUT.
Where did you get the accurate GLUT? Before you had an accurate GLUT , you couldn’t show him the accurate GLUT which would be needed to calculate the accurate GLUT.
Sure, you can calculate the GLUT for all possible inputs, some of which may be accurate GLUT inputs, but then you have to solve the problem of finding which of all possible inputs are accurate GLUT inputs. To answer the question, someone has to do calculations above and beyond just simulating a human.
OK, that’s fair. Here are some more thoughts.
To be concrete suppose the input you give to the subject is a finite string of bits of length N. The GLUT is computed by simulating the output produced by each of the 2^N possible inputs, observing the result, and writing it down. Now we want to know what happens when we encode the GLUT as a bit string of length N and giving it to the subject as an input. I think the answer is that in general we can’t do this, because to encode all 2^N entries of the GLUT clearly requires far more than N bits, so the GLUT is too big to be a valid input to the simulation. So it seems an accurate GLUT is too big for us to be able to show it to the subject. Maybe this is one answer to the question.
But maybe for some subjects we can do enough compression on the GLUT that we can fit it into N bits. As a trivial example, if the input string is large but the subject’s behavior is simply to say “yes” for any input, then we can show him the accurate compressed GLUT: “For any input, the subject will say ‘yes.’”
Less trivially, we could think of the simulator program itself as a highly compressed GLUT. And maybe the simulator executable is a mere 100 MB file, while the simulation accepts 100 GB of input. Then we could supply the simulated subject with the executable that is running him as an accurate compressed GLUT.
Suppose the simulated subject is a contrarian and tries to prove that the compressed GLUT we gave him is not accurate. He loads the executable onto a simulated computer and runs the executable on itself to see what it will output. Of course, it creates a simulated simulated subject who takes the executable and loads it onto a simulated simulated computer to see what it will output. This creates a simulated simulated simulated subject...
So this simulation will never halt when given this input. Presumably an accurate uncompressed GLUT would have an entry reading “Subject is supplied with his own source code → simulation does not halt.”
So maybe we can indeed connect to the halting problem?
Very likely; but I was only going to show him the bits predicting his response to his immediate situation + being shown those bits. We can save a bit of paper by not printing out his predicted reaction to chocolate elephants falling from the sky. :)
It seems to me that your construction relies on the subject having access to the executable. If we don’t give him that access, he cannot use this method of attempted disproof, no matter how contrarian he is.
OK, but again this simply may not be possible even if you have an accurate GLUT. If you give him anything that lets him compute your true prediction in finite time, then he can compute your true prediction and then do the opposite. Even if we have a complete and accurate GLUT, we can never supply him with a true prediction if the accurate GLUT contains no entries of the form “Subject is told he is predicted to do X” → subject does X.
Well, that’s precisely my point. But see prase’s comment below, with the very interesting point that every sufficiently-nice function f(x) has some x for which f(x)=x. The question is whether the human brain is sufficiently nice.