Definitions and justifications have to be circular at some point, or else terminate in some unexplained things, or else create an infinite chain.
If I’m understanding your point correctly, I think I disagree completely. A chain of instrumental goals terminates in a terminal goal, which is a very different kind of thing from an instrumental goal in that assigning properties like “unjustified” or “useless” to it is a category error. Instrumental goals either promote higher goals or are unjustified, but that’s not true of all goals- it’s just something particular to that one type of goal.
I’d also argue that a chain of definitions terminates in qualia- things like sense data and instincts determine the structure of our most basic concepts, which define higher concepts, but calling qualia “undefined” would be a category error.
There is no fundamental physical structure which constitutes agency
I also don’t think I agree with this. A given slice of objective reality will only have so much structure- only so many ways of compressing it down with symbols and concepts. It’s true that we’re only interested in a narrow subset of that structure that’s useful to us, but the structure nevertheless exists prior to us. When we come up with a useful concept that objectively predicts part of reality, we’ve, in a very biased way, discovered an objective part of the structure of reality- and I think that’s true of the concept of agency.
Granted, maybe there’s a strange loop in the way that cognitive reduction can be further reduced to physical reduction, while physical reduction can be further reduced to cognitive reduction- objective structure defines qualia, which defines objective structure. If that’s what you’re getting at, you may be on to something.
There seems to be a strong coalition around consciousness
One further objection, however: given that we don’t really understand consciousness, I think the cultural push to base our morality around it is a really bad idea.
If it were up to me, we’d split morality up into stuff meant to solve coordination problems by getting people to pre-commit to not defecting, stuff meant to promote compassionate ends for their own sake, and stuff that’s just traditional. Doing that instead of conflating everything into a single universal imperative would get rid of the deontology/consequentialism confusion, since deontology would explain the first thing and consequentialism the second, and by not founding our morality on poorly understood philosophy concepts, we wouldn’t risk damaging useful social technologies or justifying horrifying atrocities if Dennettian illusionism turns out to be true or something.
I’m hoping to reply more thoroughly later, but here’s a quick one: I am curious how you would further clarify the distinction between terminal goals vs instrumental goals. Naively, a terminal goal is one which does not require justification. However, you appear to be marking this way of speaking as a “category error”. Could you spell out your preferred system in more detail? Personally, I find the distinction between terminal and instrumental goals to be tenable but unnecessary; details of my way of thinking are in my post An Orthodox Case Against Utility Functions.
I’m certain your model of what purpose is is a lot more detailed than mine. My take, however, is that animal brains don’t exactly have a utility function, but probably do have something functionally similar to a reward function in machine learning. A well-defined set of instrumental goals terminating in terminal goals would be a very effective way of maximizing that reward, so the behaviors reinforced will often converge on an approximation of that structure. However, the biological learning algorithm is very bad at consistently finding the structure, and so the approximations will tend to shift around and conflict a lot- behaviors that approximate a terminal goal one year might approximate an instrumental goal later on, or cease to approximate any goal at all. Imagine a primitive image diffusion model with a training set of face photos- you run it on a set of random pixels, and it starts producing eyes and mouths and so on in random places, then gradually shifts those around into a slightly more coherent image as the remaining noise decreases.
So, instrumental and terminal goals in my model aren’t so much things agents actually have as a sort of logical structure that influences how our behaviors develop. It’s sort of like the structure of “if A implies B which implies C, then A implies C”- that’s something that exists prior to us, but we tend to adopt behaviors approximating it because doing so produces a lot of reward. Note, though, that comparing the structure of goals to logic can be confusing, since logic can help promote terminal goals- so when we’re approximating having goals, we want to be logical, but we have no reason to want to have terminal goals. That just something our biological reward function tends to reinforce.
Regarding my use of the term “category error”, I used that term rather than saying “terminal goals don’t require justification” because, while technically accurate, the use of the word “require” there sounds very strange to me. To “require” something means that it’s necessary to promote some terminal goal. So, the phrase reads to me a bit like “a king is the rank which doesn’t follow the king’s orders”- accurate, technically, but odd. More sensible to say instead that following the king’s orders is something having to do with subjects, and a category error when applied to a king.
If I’m understanding your point correctly, I think I disagree completely. A chain of instrumental goals terminates in a terminal goal, which is a very different kind of thing from an instrumental goal in that assigning properties like “unjustified” or “useless” to it is a category error. Instrumental goals either promote higher goals or are unjustified, but that’s not true of all goals- it’s just something particular to that one type of goal.
I’d also argue that a chain of definitions terminates in qualia- things like sense data and instincts determine the structure of our most basic concepts, which define higher concepts, but calling qualia “undefined” would be a category error.
I also don’t think I agree with this. A given slice of objective reality will only have so much structure- only so many ways of compressing it down with symbols and concepts. It’s true that we’re only interested in a narrow subset of that structure that’s useful to us, but the structure nevertheless exists prior to us. When we come up with a useful concept that objectively predicts part of reality, we’ve, in a very biased way, discovered an objective part of the structure of reality- and I think that’s true of the concept of agency.
Granted, maybe there’s a strange loop in the way that cognitive reduction can be further reduced to physical reduction, while physical reduction can be further reduced to cognitive reduction- objective structure defines qualia, which defines objective structure. If that’s what you’re getting at, you may be on to something.
One further objection, however: given that we don’t really understand consciousness, I think the cultural push to base our morality around it is a really bad idea.
If it were up to me, we’d split morality up into stuff meant to solve coordination problems by getting people to pre-commit to not defecting, stuff meant to promote compassionate ends for their own sake, and stuff that’s just traditional. Doing that instead of conflating everything into a single universal imperative would get rid of the deontology/consequentialism confusion, since deontology would explain the first thing and consequentialism the second, and by not founding our morality on poorly understood philosophy concepts, we wouldn’t risk damaging useful social technologies or justifying horrifying atrocities if Dennettian illusionism turns out to be true or something.
I’m hoping to reply more thoroughly later, but here’s a quick one: I am curious how you would further clarify the distinction between terminal goals vs instrumental goals. Naively, a terminal goal is one which does not require justification. However, you appear to be marking this way of speaking as a “category error”. Could you spell out your preferred system in more detail? Personally, I find the distinction between terminal and instrumental goals to be tenable but unnecessary; details of my way of thinking are in my post An Orthodox Case Against Utility Functions.
I’m certain your model of what purpose is is a lot more detailed than mine. My take, however, is that animal brains don’t exactly have a utility function, but probably do have something functionally similar to a reward function in machine learning. A well-defined set of instrumental goals terminating in terminal goals would be a very effective way of maximizing that reward, so the behaviors reinforced will often converge on an approximation of that structure. However, the biological learning algorithm is very bad at consistently finding the structure, and so the approximations will tend to shift around and conflict a lot- behaviors that approximate a terminal goal one year might approximate an instrumental goal later on, or cease to approximate any goal at all. Imagine a primitive image diffusion model with a training set of face photos- you run it on a set of random pixels, and it starts producing eyes and mouths and so on in random places, then gradually shifts those around into a slightly more coherent image as the remaining noise decreases.
So, instrumental and terminal goals in my model aren’t so much things agents actually have as a sort of logical structure that influences how our behaviors develop. It’s sort of like the structure of “if A implies B which implies C, then A implies C”- that’s something that exists prior to us, but we tend to adopt behaviors approximating it because doing so produces a lot of reward. Note, though, that comparing the structure of goals to logic can be confusing, since logic can help promote terminal goals- so when we’re approximating having goals, we want to be logical, but we have no reason to want to have terminal goals. That just something our biological reward function tends to reinforce.
Regarding my use of the term “category error”, I used that term rather than saying “terminal goals don’t require justification” because, while technically accurate, the use of the word “require” there sounds very strange to me. To “require” something means that it’s necessary to promote some terminal goal. So, the phrase reads to me a bit like “a king is the rank which doesn’t follow the king’s orders”- accurate, technically, but odd. More sensible to say instead that following the king’s orders is something having to do with subjects, and a category error when applied to a king.