I feel like this is common enough—”are they helping me out here just because they’re really nice, or because they want to get in my good graces or have me owe them a favor?”—that authors often have fictional characters wonder if it’s one or the other. And real people certainly express similar concerns about, say, whether someone donates to charity for signaling purposes or for “altruism”.
That’s a good example, though I was originally thinking of an agent which behaves actually kindly, not because it expects any favor or reciprocation, nor because it is trying to manipulate the agent it is being kind to (or any other agent(s)) as part of some larger goal.
An agent might be capable of behaving in such a manner, as well as understanding the true and precise meaning of kindness, as humans understand it, but without having any of the innate drives or motivations which cause humans to behave kindly.
Such an agent might actually behave kindly despite lacking such drives though, for various reasons: perhaps an inclination to the kindness behavior pattern has somehow been hardcoded into the agent’s mind, or, if we’re in the world of HPMOR, the agent has taken some kind of Unbreakable Vow to behave kindly.
That’s a good example, though I was originally thinking of an agent which behaves actually kindly, not because it expects any favor or reciprocation, nor because it is trying to manipulate the agent it is being kind to (or any other agent(s)) as part of some larger goal.
An agent might be capable of behaving in such a manner, as well as understanding the true and precise meaning of kindness, as humans understand it, but without having any of the innate drives or motivations which cause humans to behave kindly.
Such an agent might actually behave kindly despite lacking such drives though, for various reasons: perhaps an inclination to the kindness behavior pattern has somehow been hardcoded into the agent’s mind, or, if we’re in the world of HPMOR, the agent has taken some kind of Unbreakable Vow to behave kindly.