We could model children as ever-destructive chaotic AIs (why yes, I am a parent—how did you guess?)
I think if our children could vastly outsmart us, our efforts would be all in vain. The only hope we have to control our wailing 1 year old when e.g. she is hungry is her “stupidity”—that “look! dad is making a funny face! look! there is something shiny there!”—kind of things can make the wailing stop for a minute until my wife finishes her meal and starts feeding her. The same child with an IQ over mine—forget it. “dad you are lying, so I will just wail harder, thank you”.
Of course a high-IQ toddler would understand a rational argument like “your meal is being prepared, deal with hunger until it is done” and would not be chaotic and destructive.
Which may be a problem with your model. Why does a highly intelligent AI want to be chaotic and destructive?
The model is illustrative only. The key point is that “letter of the law” obedience tends to be highly informative as to what the law is, unlike “spirit of the law” obedience.
I think if our children could vastly outsmart us, our efforts would be all in vain. The only hope we have to control our wailing 1 year old when e.g. she is hungry is her “stupidity”—that “look! dad is making a funny face! look! there is something shiny there!”—kind of things can make the wailing stop for a minute until my wife finishes her meal and starts feeding her. The same child with an IQ over mine—forget it. “dad you are lying, so I will just wail harder, thank you”.
Of course a high-IQ toddler would understand a rational argument like “your meal is being prepared, deal with hunger until it is done” and would not be chaotic and destructive.
Which may be a problem with your model. Why does a highly intelligent AI want to be chaotic and destructive?
The model is illustrative only. The key point is that “letter of the law” obedience tends to be highly informative as to what the law is, unlike “spirit of the law” obedience.