“One consideration is the differences in cognitive abilities between AGIs and humans may make human preferences easily changeable for AGIs. As an intuition pump: consider a system composed of a five year old child and her parents. The child obviously has some preferences, but the parents can usually change these.”
Notably, though, parents of young children have their purchasing preferences and general interaction with the economy massively skewed by “the preferences of their children”. A fun example of this is how cinema attendance patterns for people aged 35-44 are identical to those of children. Children are a massive and lucrative marketing demographic specifically because their preferences are strong, developmentally less set than those of adults, and less malleable-by-parents than said parents hope. (#2 and #3 can exist simultaneously; children like what their peers are into, what’s endorsed by or advertised alongside media they enjoy, and other ‘nonshared environment’.)
“One consideration is the differences in cognitive abilities between AGIs and humans may make human preferences easily changeable for AGIs. As an intuition pump: consider a system composed of a five year old child and her parents. The child obviously has some preferences, but the parents can usually change these.”
Notably, though, parents of young children have their purchasing preferences and general interaction with the economy massively skewed by “the preferences of their children”. A fun example of this is how cinema attendance patterns for people aged 35-44 are identical to those of children. Children are a massive and lucrative marketing demographic specifically because their preferences are strong, developmentally less set than those of adults, and less malleable-by-parents than said parents hope. (#2 and #3 can exist simultaneously; children like what their peers are into, what’s endorsed by or advertised alongside media they enjoy, and other ‘nonshared environment’.)