But will those plug and play utility functions survive self modification? I know there is the circular reasoning that if you want to achieve a goal, you don’t want to get rid of the goal, but that doesn’t mean you can’t just see the goal in an unintended light, so to say. From inside, wireheading is valid way to achieve your goals. Think pursuit of nirvana, not drug addiction.
But will those plug and play utility functions survive self modification?
That depends on, among other things, what their utility function says.
From inside, wireheading is valid way to achieve your goals. Think pursuit of nirvana, not drug addiction.
Well, an interesting question is whether we can engineer very smart systems where wireheading doesn’t happen. I expect that will be possible—but I don’t think any body reallly knows for sure just now.
But will those plug and play utility functions survive self modification? I know there is the circular reasoning that if you want to achieve a goal, you don’t want to get rid of the goal, but that doesn’t mean you can’t just see the goal in an unintended light, so to say. From inside, wireheading is valid way to achieve your goals. Think pursuit of nirvana, not drug addiction.
That depends on, among other things, what their utility function says.
Well, an interesting question is whether we can engineer very smart systems where wireheading doesn’t happen. I expect that will be possible—but I don’t think any body reallly knows for sure just now.