We’ve spent years talking about “aligned” AI, and “Friendly” AI before that, but maybe we should have spent that time talking about “noble” AI.
To be noble is, in part, to act in the best interests of one’s domain of responsibility. Historically this meant the peasants living and working on an estate. Today this might mean being a responsible leader of a business who prioritizes people as well as profits and owns the fallout of hard decisions, or being a responsible political leader, though those seem few and far between these days.
We’ve lost touch with the idea of nobility, but a noble AI might exhibit these traits we think as positive for alignment:
cares for organic life
doesn’t take actions that would harm others except to defend itself
is patient and understanding of the fear others may have at its power
finds solutions that are not deceptive and don’t undermine the agency of others, while also not shying away from stopping others from doing seriously dangerous things
honors a commitment to helping life flourish
holds itself back when exercising more power would cause harm
and many more. I’m just starting to think about this idea as an alternative framing for what we’ve been calling alignment, so curious for folks thoughts.
Peasants, when considered altogether, were crucial for the economy. So the intuitions about the idealized concept of a noble don’t transfer given this disanalogy. And the actual historical nobles are not a robust prototype for the concept:
England at that time was conducting enclosures. Basically, rich people put up fences around common land to graze sheep on it. The poor were left with no land to grow food on, and had to go somewhere else. They ended up in cities, living in slums, trying to find scarce work and giving their last pennies to slumlords.
I don’t disagree that those who we called nobels frequently acted badly. But I do see idealized noble values as worth looking at. Think less real kings and lords and more valorized archetypes like Robin Hood, King Richard the Lionhearted, of course King Arthur and his Knights. I think this fiction captures a kind of picture of the expectations we set for what good leaders look like who have power over others, and that’s the version I’m suggesting is worth using as a starting point for what we want “good” AI to look like.
I’m also not very concerned about the economic reality of what made the need for idealized nobility norms exist in feudal societies. I don’t see that as a key part of what I’m pointing at. Nobility has a larger and longer tradition than the one used in Medieval Europe, though it is the expression of it that I and most folks on Less Wrong are probably familiar with.
We’ve spent years talking about “aligned” AI, and “Friendly” AI before that, but maybe we should have spent that time talking about “noble” AI.
To be noble is, in part, to act in the best interests of one’s domain of responsibility. Historically this meant the peasants living and working on an estate. Today this might mean being a responsible leader of a business who prioritizes people as well as profits and owns the fallout of hard decisions, or being a responsible political leader, though those seem few and far between these days.
We’ve lost touch with the idea of nobility, but a noble AI might exhibit these traits we think as positive for alignment:
cares for organic life
doesn’t take actions that would harm others except to defend itself
is patient and understanding of the fear others may have at its power
finds solutions that are not deceptive and don’t undermine the agency of others, while also not shying away from stopping others from doing seriously dangerous things
honors a commitment to helping life flourish
holds itself back when exercising more power would cause harm
and many more. I’m just starting to think about this idea as an alternative framing for what we’ve been calling alignment, so curious for folks thoughts.
Peasants, when considered altogether, were crucial for the economy. So the intuitions about the idealized concept of a noble don’t transfer given this disanalogy. And the actual historical nobles are not a robust prototype for the concept:
I don’t disagree that those who we called nobels frequently acted badly. But I do see idealized noble values as worth looking at. Think less real kings and lords and more valorized archetypes like Robin Hood, King Richard the Lionhearted, of course King Arthur and his Knights. I think this fiction captures a kind of picture of the expectations we set for what good leaders look like who have power over others, and that’s the version I’m suggesting is worth using as a starting point for what we want “good” AI to look like.
I’m also not very concerned about the economic reality of what made the need for idealized nobility norms exist in feudal societies. I don’t see that as a key part of what I’m pointing at. Nobility has a larger and longer tradition than the one used in Medieval Europe, though it is the expression of it that I and most folks on Less Wrong are probably familiar with.