Am I missing something or does this piece never actually make the argument it promises? It seems roughly comparable to:
A Turing machine & its starting tape are a joint specification; neither means or can do anything without the other, and the same functions can be differently allocated between them.
I wouldn’t suggest this breaks the orthogonality of Turing machines. Nor does the fact that a small machine with a small tape can’t compute large functions, and a small enough machine isn’t Turing complete, break the relevant orthogonality claim.
Similarly, I don’t understand why you think that beliefs and values being only jointly predictive of actions (and therefore jointly selected) restricts the space of values or propositional beliefs implementable in an AI within the space of expressible ones.
Am I missing something or does this piece never actually make the argument it promises? It seems roughly comparable to:
A Turing machine & its starting tape are a joint specification; neither means or can do anything without the other, and the same functions can be differently allocated between them.
I wouldn’t suggest this breaks the orthogonality of Turing machines. Nor does the fact that a small machine with a small tape can’t compute large functions, and a small enough machine isn’t Turing complete, break the relevant orthogonality claim.
Similarly, I don’t understand why you think that beliefs and values being only jointly predictive of actions (and therefore jointly selected) restricts the space of values or propositional beliefs implementable in an AI within the space of expressible ones.