(whether or not H intends for A to achieve H’s goals)?
How is H not intending A to achive H’s goals a meaningful situation. If we make the assumption that humans are goal seeking agents, then the human wants those goals achieved.
Of course the human might not be a goal directed agent, even roughly. Some humans, at least some of those with serious mental illnesses, can’t be modelled as goal directed agents even roughly. The human might not actually know that the AI exists.
But if you are defining the humans goals as something different from what the human actually wants, then something odd is happening somewhere. If you want to propose a specific definition of “intends” and “goals” that are different, go ahead, but to me there words read as synonyms.
Probably H intends A to achieve a narrow subset of H’s goals, but doesn’t necessarily want A pursuing them in general.
Similarly, if I have an employee, I may intend for them to do some work-related tasks for me, but I probably don’t intend for them to go and look after my parents, even though ensuring my parents are well looked-after is a goal of mine.
How is H not intending A to achive H’s goals a meaningful situation. If we make the assumption that humans are goal seeking agents, then the human wants those goals achieved.
Of course the human might not be a goal directed agent, even roughly. Some humans, at least some of those with serious mental illnesses, can’t be modelled as goal directed agents even roughly. The human might not actually know that the AI exists.
But if you are defining the humans goals as something different from what the human actually wants, then something odd is happening somewhere. If you want to propose a specific definition of “intends” and “goals” that are different, go ahead, but to me there words read as synonyms.
Probably H intends A to achieve a narrow subset of H’s goals, but doesn’t necessarily want A pursuing them in general.
Similarly, if I have an employee, I may intend for them to do some work-related tasks for me, but I probably don’t intend for them to go and look after my parents, even though ensuring my parents are well looked-after is a goal of mine.