if the free energy principle were right, would it disprove the orthogonality thesis? Might it be impossible to design a working brain with any goal besides free energy reduction? Would anything – even a paperclip maximizer – have to start by minimizing uncertainty, and then add paperclip maximization in later as a hack? Would it change anything if it did?
My own take on this is “kinda no”, but mostly because I already see the orthogonality thesis as holding only for a sufficiently general intelligence. That is, there are probably lots of things that are not orthogonal to what we think of as general intelligence, and it’s only after you get those basics up to a certain level that you get an intelligence general enough for orthogonality to kick in such that capabilities can be orthogonal to telos for capabilities and telos above a certain generality threshold but below that threshold you get capabilities and telos that are correlated.
For example, I suspect phenomenal consciousness to be a property that correlates capabilities and telos around the intentional relation, and free energy may well be looking at a related notion from the perspective of how telos is achieved, and to a certain extent it looks to me like anything that does anything “intelligent” will follow these patterns if it can be called “general”. We still have to worry about orthogonality, though, because these are very low level kinds of telos that shape an agents goals in correlation to its capabilities but not so strongly as to allow steering away from dangerous territory.
In short, orthogonality is probably not true all the way down, but it doesn’t matter because the orthogonality thesis is more about orthogonality after generality of intelligence rather than before.
My own take on this is “kinda no”, but mostly because I already see the orthogonality thesis as holding only for a sufficiently general intelligence. That is, there are probably lots of things that are not orthogonal to what we think of as general intelligence, and it’s only after you get those basics up to a certain level that you get an intelligence general enough for orthogonality to kick in such that capabilities can be orthogonal to telos for capabilities and telos above a certain generality threshold but below that threshold you get capabilities and telos that are correlated.
For example, I suspect phenomenal consciousness to be a property that correlates capabilities and telos around the intentional relation, and free energy may well be looking at a related notion from the perspective of how telos is achieved, and to a certain extent it looks to me like anything that does anything “intelligent” will follow these patterns if it can be called “general”. We still have to worry about orthogonality, though, because these are very low level kinds of telos that shape an agents goals in correlation to its capabilities but not so strongly as to allow steering away from dangerous territory.
In short, orthogonality is probably not true all the way down, but it doesn’t matter because the orthogonality thesis is more about orthogonality after generality of intelligence rather than before.