you don’t want someone asking it for the source code for some other type of AI, for example.
But that is as easy as not being reckless with it. One can still deliberately crash a car, but they’re pretty safe.
The relative sizes of space of safe/dangerous questions compares favorably to the sizes of space of FAI/UFAI designs.
I suspect that this part of the problem is FAI-complete.
If so, that’s not necessarily an argument against an oracle AI. It still may be ‘as hard as creating FAI’, but only because FAI can be made through an oracle AI (and all other paths being much harder).
But that is as easy as not being reckless with it. One can still deliberately crash a car, but they’re pretty safe.
The relative sizes of space of safe/dangerous questions compares favorably to the sizes of space of FAI/UFAI designs.
If so, that’s not necessarily an argument against an oracle AI. It still may be ‘as hard as creating FAI’, but only because FAI can be made through an oracle AI (and all other paths being much harder).