What I take away from this experiment is that the most effective form of persuasion is to blackmail or bribe the person you’re trying to persuade and convince them that you’ll make good on it. You don’t need to convince them about the inherent goodness of what you want them to do if all you want is for them to do it. Given this, it seems trivial that AI will eventually become vastly superhuman at persuasion. It can find out every person’s deepest secrets, fears, and desires and figure out what sorts of things are likely to get them to fold.
Arcturus
Karma: 1
His metaphors serve as useful intuition pumps for ideas that are highly abstract and far removed from what the average layperson has thought about prior. When I started reading about AI risk, many of the ideas I was introduced to, like optimization and orthogonality, were completely novel and I had a hard time understanding them. The various metaphors, analogies, and parables that exist within the AI-risk discussion were of significant benefit to me in gaining the necessary intuitions to understand the problem.
I wonder if the carrots were implemented as a means of encouraging doing more of the thing than the organism would do otherwise. If there were no carrot aspect to hunger, the organism would only eat just enough to stop the pain of hunger but no more, missing the opportunity to build up fat stores for the future. If there were no carrot aspect to peeing, the organism would expel just enough to make the pain go away but wouldn’t empty the tank, so to speak, making them carry around some tiny amount of extra weight than otherwise. This might be a just-so story, but it’s the answer that comes to mind for me.