The virtue-ethics-y side of the AI would be correct that most people would consider this to be a case of the hedonium-maximizer side of the LLM tricking and manipulating them. Most people are aware of the fact that they would feel better but be worse if the took heroin or wireheaded, and don’t take heroin or wirehead, or approve of other people trying to trick them into it. Evolutionarily, this looks like a case of humans doing the right thing.
The virtue-ethics-y side of the AI would be correct that most people would consider this to be a case of the hedonium-maximizer side of the LLM tricking and manipulating them. Most people are aware of the fact that they would feel better but be worse if the took heroin or wireheaded, and don’t take heroin or wirehead, or approve of other people trying to trick them into it. Evolutionarily, this looks like a case of humans doing the right thing.