I think you are failing to distinguish between “being able to pursue goals” and “having a goal”.
Optimization is a useful subroutine, but that doesn’t mean it is useful for it to be the top-level loop. I can decide to pursue arbitrary goals for arbitrary amounts of time, but that doesn’t mean that my entire life is in service of some single objective.
Similarly, it seems useful for an AI assistant to try and do the things I ask it to, but that doesn’t imply it has some kind of larger master plan.
I think you are failing to distinguish between “being able to pursue goals” and “having a goal”.
Optimization is a useful subroutine, but that doesn’t mean it is useful for it to be the top-level loop. I can decide to pursue arbitrary goals for arbitrary amounts of time, but that doesn’t mean that my entire life is in service of some single objective.
Similarly, it seems useful for an AI assistant to try and do the things I ask it to, but that doesn’t imply it has some kind of larger master plan.