it does not have to turn into a paperclip maximizer, but could turn into a billion other things that are equally bad and behaves unexpectedly. Again, to me unexpected means that it has behave in a way that no one had foreseen or predicted. Hence that any guardrails would work would be just by chance or luck. Either we know how the system will behave and we can control that, or we don’t. Any unexpected or surprising behavior COULD spell disaster. If you that think that this kind of system behavior is to be expected you are in my world just saying that you expect disaster to happen.
it does not have to turn into a paperclip maximizer, but could turn into a billion other things that are equally bad and behaves unexpectedly. Again, to me unexpected means that it has behave in a way that no one had foreseen or predicted. Hence that any guardrails would work would be just by chance or luck. Either we know how the system will behave and we can control that, or we don’t. Any unexpected or surprising behavior COULD spell disaster. If you that think that this kind of system behavior is to be expected you are in my world just saying that you expect disaster to happen.
You want the AI to do unexpected things, otherwise you might as well just encode all your expectations as GOFAI.
can you rephrase what you mean by “you want”? I’m not sure I follow what level of implication that’s on or who wants it why.