You won’t be horrified if you’re dead. More seriously though, if we got an Oracle AI that understood the intended meaning of our questions and did not lie or decieve us in any way, that would be an AI-alignment complete problem—in other words, just as hard as creating friendly AI in the first place.
“How do I create a safe AGI?”
Edit: Or, more likely, “this is my design for an AGI, (how) will running this AGI result in situations that I would be horrified by if they occure?”
You won’t be horrified if you’re dead. More seriously though, if we got an Oracle AI that understood the intended meaning of our questions and did not lie or decieve us in any way, that would be an AI-alignment complete problem—in other words, just as hard as creating friendly AI in the first place.