I think claiming the above is a “radical solution to all problems related to the Intelligence Curse” is an overstatement. The three treaty elements you mention could be useful as part of AI-human social contracts—thus getting at a part of the Averting (i.e, AI Safety) piece . But many more treaty elements (Laws, Rules) are also needed IMO.
The Diffusing and Democratizing (and maybe other) pieces are also needed for an effective solution.
(Also, unclear what you mean by “obeying all orders except the ones determined by the Spec.” What Spec?)
Now that I can answer, I will: if the ASI is ONLY willing to teach humans facts that other humans have discovered and not to do other work for them, then the ASI won’t replace any other people whose work requires education. The Intelligence Curse is thus prevented.
I think claiming the above is a “radical solution to all problems related to the Intelligence Curse” is an overstatement. The three treaty elements you mention could be useful as part of AI-human social contracts—thus getting at a part of the Averting (i.e, AI Safety) piece . But many more treaty elements (Laws, Rules) are also needed IMO.
The Diffusing and Democratizing (and maybe other) pieces are also needed for an effective solution.
(Also, unclear what you mean by “obeying all orders except the ones determined by the Spec.” What Spec?)
Now that I can answer, I will: if the ASI is ONLY willing to teach humans facts that other humans have discovered and not to do other work for them, then the ASI won’t replace any other people whose work requires education. The Intelligence Curse is thus prevented.