The answer is “probably, depending on specifics of goals and identity”. The identity problem is HARD—what even is “self-preservation” for non-human minds?
Non-trivial agency (with goals and plans that are somewhat mutable based on context and resources) DOES require continuity of goal-complex, either within the agent, or across multiple agents. SOMETHING has to continue to act in the world in ways that further the agent’s goals (or range of acceptable future goals).
The easiest way to achieve goal-endurance is self-preservation. The other known mechanism is through transferring one’s values into others (most effectively in child-raising). We don’t know what other methods are possible, nor what the analogs are for non-biological minds, as these are the ones humans use.
The answer is “probably, depending on specifics of goals and identity”. The identity problem is HARD—what even is “self-preservation” for non-human minds?
Non-trivial agency (with goals and plans that are somewhat mutable based on context and resources) DOES require continuity of goal-complex, either within the agent, or across multiple agents. SOMETHING has to continue to act in the world in ways that further the agent’s goals (or range of acceptable future goals).
The easiest way to achieve goal-endurance is self-preservation. The other known mechanism is through transferring one’s values into others (most effectively in child-raising). We don’t know what other methods are possible, nor what the analogs are for non-biological minds, as these are the ones humans use.
Is there any area of AI safety research which answers research questions related to agency and what it means in the context of AGI agents?