You can’t get the coffee if you’re dead

WikiLast edit: 16 Jan 2017 19:33 UTC by Eric Rogstad

“You can’t get the coffee if you’re dead” is Stuart Russell’s capsule description of why almost any agent goal implies an instrumental strategy of surviving /​ not being shut down—by default and barring other measures to prevent this from happening; and assuming the agent has sufficient big-picture awareness to understand the relation between the agent’s non-operation and the coffee not being brought. See Shutdown problem and Instrumental convergence.

No comments.