We do this primarily via exogenous methods, using a combination of punishments and rewards, that molds their behavior by encouraging good behaviors and discouraging bad ones.
I’m not sure punishments and rewards are really the big story of moral alignment. Imitation and emulation are pretty central. Small children don’t just generate arbitrary behaviors and reinforce the ones that are rewarded and not punished; they overtly imitate the behavior of parents and others. And giving them tailored opportunities to imitate behaviors is a big part of teaching — “I do, we do, you do”.
And I think this includes moral behaviors, for good or ill.
A child who observes a parent giving alms, learns that giving alms is something we do around here. (They then can be invited to participate, and then rewarded for doing a good deed.)
A child who observes a parent making and keeping promises, learns that promises are something we do around here. (They can also notice the rewards of keeping promises.)
A child who observes a parent following rules, learns that following rules is something we do around here. Moral rules and other sorts of rules (table manners, safety rules, game rules) share a lot of cognitive structure.
A child who observes a parent breaking rules, cheating when they can get away with it, explicitly modeling whether they can get away with it, etc. learns that those are things we do around here. (I was maybe seven or eight when I learned why my dad’s car had a radar detector and my mom’s car didn’t. I ended up taking after my mom and stepdad; I don’t have a use for a radar detector.)
And a child who observes a parent being violent, learns that violence issomething we do around here — at least when we’re bigger than the other guy. (And the child learns that lesson even if the child is punished for violence; imitation can overrule conditioning.)
We mostly don’t try to align adults via exogenous means.
One area where we do: Employers use exogenous means to align employees to the employer’s values all the time. Sometimes these values align with morality; alas, not always. This is directly pertinent to current AI insofar as people seem to want to treat it like a sort of employee.
I’m not sure punishments and rewards are really the big story of moral alignment. Imitation and emulation are pretty central. Small children don’t just generate arbitrary behaviors and reinforce the ones that are rewarded and not punished; they overtly imitate the behavior of parents and others. And giving them tailored opportunities to imitate behaviors is a big part of teaching — “I do, we do, you do”.
And I think this includes moral behaviors, for good or ill.
A child who observes a parent giving alms, learns that giving alms is something we do around here. (They then can be invited to participate, and then rewarded for doing a good deed.)
A child who observes a parent making and keeping promises, learns that promises are something we do around here. (They can also notice the rewards of keeping promises.)
A child who observes a parent following rules, learns that following rules is something we do around here. Moral rules and other sorts of rules (table manners, safety rules, game rules) share a lot of cognitive structure.
A child who observes a parent breaking rules, cheating when they can get away with it, explicitly modeling whether they can get away with it, etc. learns that those are things we do around here. (I was maybe seven or eight when I learned why my dad’s car had a radar detector and my mom’s car didn’t. I ended up taking after my mom and stepdad; I don’t have a use for a radar detector.)
And a child who observes a parent being violent, learns that violence is something we do around here — at least when we’re bigger than the other guy. (And the child learns that lesson even if the child is punished for violence; imitation can overrule conditioning.)
One area where we do: Employers use exogenous means to align employees to the employer’s values all the time. Sometimes these values align with morality; alas, not always. This is directly pertinent to current AI insofar as people seem to want to treat it like a sort of employee.