What if you build a super-intelligent AI and you are convinced that it is Friendly, and it tells you to do something like this? Go kill such-and-such a baby, and you will massively increase the future happiness of the human race. You argue and ask if there isn’t some other way to do it, and the FAI explains that every other alternative will involve much greater human suffering. Killing a baby is relatively humane, as newborn babies have only limited consciousness, and their experiences are not remembered anyway. You will kill the baby instantly and painlessly. And the circumstances are such that no one will know (the baby will have been abandoned but would be discovered and saved if you don’t intervene), so there is no hardship on anyone else, except you.
You ask, why me, and the FAI says (A) there’s not much time, and (B) you’re the kind of amoral person who can do this without carrying an enormous burden afterwards. I’ll just point out that from reading the various comments here, it sounds like many readers of this blog would fit this description!
What if you build a super-intelligent AI and you are convinced that it is Friendly, and it tells you to do something like this? Go kill such-and-such a baby, and you will massively increase the future happiness of the human race. You argue and ask if there isn’t some other way to do it, and the FAI explains that every other alternative will involve much greater human suffering. Killing a baby is relatively humane, as newborn babies have only limited consciousness, and their experiences are not remembered anyway. You will kill the baby instantly and painlessly. And the circumstances are such that no one will know (the baby will have been abandoned but would be discovered and saved if you don’t intervene), so there is no hardship on anyone else, except you.
You ask, why me, and the FAI says (A) there’s not much time, and (B) you’re the kind of amoral person who can do this without carrying an enormous burden afterwards. I’ll just point out that from reading the various comments here, it sounds like many readers of this blog would fit this description!
So, would you do it?