“Obey me” is actually a sane approach to creating FAI. It’s clear and simple. The obedient AI can then be used to create a FAI, assuming the author wishes to do so and is able to communicate the concept of friendliness (both prerequisites for creating a FAI on purpose). Since the FAI needs to obey a friendliness criteria, it needs to have an obey capability built in anyways. The author just needs to make sure not to say something stupid, which once again is a necessity anyways.
You seem to be expecting an obedient AI to understand “obey me” to mean “do only what I say”… e.g., you expect the AI not to interpret hand gestures, for example.
Is that right? If so, how confident are you of that expectation?
I’d expect the “obey me” aspect to be “read signed messages from this file or from your input and do what it says” then making sure that the AI can’t get the signing key and cut out the middleman. Definitely not something as simple to overwrite or fake as microphone or keyboard inputs. Also that way I don’t say things by accident, although any command could still have unintended consequences.
Unfortunately, that would be impossible, unless you can make an AI that can understand natural language before it is ever run. And that would require having a proper theory of mind right from the start.
“Obey me” is actually a sane approach to creating FAI. It’s clear and simple. The obedient AI can then be used to create a FAI, assuming the author wishes to do so and is able to communicate the concept of friendliness (both prerequisites for creating a FAI on purpose). Since the FAI needs to obey a friendliness criteria, it needs to have an obey capability built in anyways. The author just needs to make sure not to say something stupid, which once again is a necessity anyways.
You seem to be expecting an obedient AI to understand “obey me” to mean “do only what I say”… e.g., you expect the AI not to interpret hand gestures, for example.
Is that right?
If so, how confident are you of that expectation?
I’d expect the “obey me” aspect to be “read signed messages from this file or from your input and do what it says” then making sure that the AI can’t get the signing key and cut out the middleman. Definitely not something as simple to overwrite or fake as microphone or keyboard inputs. Also that way I don’t say things by accident, although any command could still have unintended consequences.
OK, thanks for clarifying that.
Do you expect the signed messages to be expressed in a natural human language?
Unfortunately, that would be impossible, unless you can make an AI that can understand natural language before it is ever run. And that would require having a proper theory of mind right from the start.
OK. Thanks for clarifying your expectations.
Hello? Seed .AI?