On top of what Garrett said, reflection also pushes against this pretty hard. An AI that has gone through a few situations where it has acted against its own goals because of “context-specific heuristics” will be motivated to remove those heuristics, if that is an available option.
On top of what Garrett said, reflection also pushes against this pretty hard. An AI that has gone through a few situations where it has acted against its own goals because of “context-specific heuristics” will be motivated to remove those heuristics, if that is an available option.