“Oh but physical devices can’t run an arbitrarily long tape”
This is not the actual issue.
The actual issue is that even with unbounded resources, you still couldn’t simulate an unbounded tape because you can’t get enough space for positional encodings.
Humans are not Turing-complete in some narrow sense;
Note that for the purpose of Turing-completeness, we only need to show that if we gave it unbounded resources, it could solve any computable problem without having to change the code, and we haven’t actually proven that humans aren’t Turing complete (indeed my big guess is that humans are Turing completeness).
If you give a physical computer a large enough tape, or make a human brain large enough without changing its density, it collapses into a black hole. It is really not relevant to any of the points made in the post.
For any set of inputs and outputs to an algorithm, we can make a neural network that approximates with arbitrary precision these inputs and outputs, in a single forward pass, without even having to simulate a tape.
I sincerely ask people to engage with the actual contents of the post related to sharp left turn, goal crystallization, etc. and not with a technicality that doesn’t affect any of the points raised who they’re not an intended audience of.
To clarify a point here:
This is not the actual issue.
The actual issue is that even with unbounded resources, you still couldn’t simulate an unbounded tape because you can’t get enough space for positional encodings.
Note that for the purpose of Turing-completeness, we only need to show that if we gave it unbounded resources, it could solve any computable problem without having to change the code, and we haven’t actually proven that humans aren’t Turing complete (indeed my big guess is that humans are Turing completeness).
If you give a physical computer a large enough tape, or make a human brain large enough without changing its density, it collapses into a black hole. It is really not relevant to any of the points made in the post.
For any set of inputs and outputs to an algorithm, we can make a neural network that approximates with arbitrary precision these inputs and outputs, in a single forward pass, without even having to simulate a tape.
I sincerely ask people to engage with the actual contents of the post related to sharp left turn, goal crystallization, etc. and not with a technicality that doesn’t affect any of the points raised who they’re not an intended audience of.