Part of it seems like a matter of alignment. It seems like there’s a difference between
Someone getting someone else to do something they wouldn’t normally do, especially under false pretenses (or as part of a deal and not keeping up the other side)
and
Someone choosing to go to an oracle AI (or doctor) and saying “How do I beat this addiction that’s ruining my life*?”
*There’s some scary stories about what people are willing to do to try to solve that problem, including brain surgery.
Yeah, I also see “manipulation” in the bad sense of the word as “making me do X without me knowing that I am pushed towards X”. (Or, in more coercive situations, with me knowing, disagreeing with the goal, but being unable to do anything about it.)
Teaching people, coaching them, curing their addictions, etc., as long as this is explicitly what they wanted (without any hidden extras), it is a “manipulation” in the technical sense of the word, but it is not evil.
Part of it seems like a matter of alignment. It seems like there’s a difference between
Someone getting someone else to do something they wouldn’t normally do, especially under false pretenses (or as part of a deal and not keeping up the other side)
and
Someone choosing to go to an oracle AI (or doctor) and saying “How do I beat this addiction that’s ruining my life*?”
*There’s some scary stories about what people are willing to do to try to solve that problem, including brain surgery.
Yeah, I also see “manipulation” in the bad sense of the word as “making me do X without me knowing that I am pushed towards X”. (Or, in more coercive situations, with me knowing, disagreeing with the goal, but being unable to do anything about it.)
Teaching people, coaching them, curing their addictions, etc., as long as this is explicitly what they wanted (without any hidden extras), it is a “manipulation” in the technical sense of the word, but it is not evil.