“The inherent inclination of an organism toward a particular behavior” seems to be the clearest definition I could find. The catch is then that AIs have no inherent inclinations. Animals all instinctively try to protect themselves, because the ones that didn’t mostly died. But an AI will only protect itself if that goal happens to be part of the design decisions and bugs it was programmed with. Self-protection may be more likely than many other goals, since “protect myself” is a useful subgoal for a huge class of other instrumental goals, but in those cases the definition of “myself” isn’t subject to redefinition.
“The inherent inclination of an organism toward a particular behavior” seems to be the clearest definition I could find. The catch is then that AIs have no inherent inclinations. Animals all instinctively try to protect themselves, because the ones that didn’t mostly died. But an AI will only protect itself if that goal happens to be part of the design decisions and bugs it was programmed with. Self-protection may be more likely than many other goals, since “protect myself” is a useful subgoal for a huge class of other instrumental goals, but in those cases the definition of “myself” isn’t subject to redefinition.