I think the correct answer is going to separate different notions of ‘goal’ (I think Aristotle might have done this; someone more erudite than I is welcome to pull that in).
One possible notion is the ‘design’ goal: in the case of a man-made machine, the designer’s intent; in the case of a standard machine learner, the training function; in the case of a biological entity, reproductive fitness. There’s also a sense in which the behavior itself can be thought of as the goal; that is, an entity’s goal is to produce the outputs that it in fact produces.
There can also be internal structures that we might call ‘deliberate goals’; this is what human self-help materials tell you to set. I’m not sure if there’s a good general definition of this that’s not parochial to human intelligence.
I’m not sure if there’s a fourth kind, but I have an inkling that there might be: an approximate goal. If we say “Intelligence A maximizes function X”, we can quantify how much simpler this is than the true description of A and how much error it introduces into our predictions. If the simplification is high and the error is low it might make sense to call X an approximate goal of A.
I think the correct answer is going to separate different notions of ‘goal’ (I think Aristotle might have done this; someone more erudite than I is welcome to pull that in).
One possible notion is the ‘design’ goal: in the case of a man-made machine, the designer’s intent; in the case of a standard machine learner, the training function; in the case of a biological entity, reproductive fitness. There’s also a sense in which the behavior itself can be thought of as the goal; that is, an entity’s goal is to produce the outputs that it in fact produces.
There can also be internal structures that we might call ‘deliberate goals’; this is what human self-help materials tell you to set. I’m not sure if there’s a good general definition of this that’s not parochial to human intelligence.
I’m not sure if there’s a fourth kind, but I have an inkling that there might be: an approximate goal. If we say “Intelligence A maximizes function X”, we can quantify how much simpler this is than the true description of A and how much error it introduces into our predictions. If the simplification is high and the error is low it might make sense to call X an approximate goal of A.