Bootstrap the FAI by first building a neutral obedient AI(OAI) that is constrained in such a way that it doesn’t act besides giving answers to questions.
If you’re smart enough, you could rule the world very easily just by giving answers to questions.
Very simple example:
Questioner: Is there a command economy that can produce much more wealth and large happiness for all?
AI: Yes.
Questioner: Could you design it?
AI: Yes.
Questioner: Would you need to micromanage it?
AI: Yes.
Questioner: Would it truly be fabulous?
AI: Yes.
Then people implement the command economy (remember that if the AI is a social genius, it can shade its previous answers to bring people to want such an economy).
Then a few years later, we have the AI in total control of the entire economy and absolutely vital to it—in which case it doesn’t matter that the constraint is only to answer questions, the AI has already ascended to absolute power. No need even for the dramatic finish:
Questioner: Would you shut down the entire economy, causing misery, war, and the probable extinction of humanity, if we don’t obey you?
AI: Yes.
Questioner: Is there any chance we could resist you?
AI: No.
And the term “obedient” doesn’t help your case. Unless you want the AI to obey, blindly, every single order given to it in every single case by every single human, then “obedient” is a massively subtle and complicated concept—nearly as hard as friendly, in my opinion (maybe harder, in fact—it may be that quasi-friendly is a prerequisite for obedient).
Bootstrap the FAI by first building a neutral obedient AI(OAI) that is constrained in such a way that it doesn’t act besides giving answers to questions.
If you’re smart enough, you could rule the world very easily just by giving answers to questions.
Very simple example: Questioner: Is there a command economy that can produce much more wealth and large happiness for all? AI: Yes. Questioner: Could you design it? AI: Yes. Questioner: Would you need to micromanage it? AI: Yes. Questioner: Would it truly be fabulous? AI: Yes.
Then people implement the command economy (remember that if the AI is a social genius, it can shade its previous answers to bring people to want such an economy).
Then a few years later, we have the AI in total control of the entire economy and absolutely vital to it—in which case it doesn’t matter that the constraint is only to answer questions, the AI has already ascended to absolute power. No need even for the dramatic finish:
Questioner: Would you shut down the entire economy, causing misery, war, and the probable extinction of humanity, if we don’t obey you? AI: Yes. Questioner: Is there any chance we could resist you? AI: No.
And the term “obedient” doesn’t help your case. Unless you want the AI to obey, blindly, every single order given to it in every single case by every single human, then “obedient” is a massively subtle and complicated concept—nearly as hard as friendly, in my opinion (maybe harder, in fact—it may be that quasi-friendly is a prerequisite for obedient).