I figure that the AI would most likely create a copy of itself, modify that, and see how it turns out. Of course, once you have a sufficiently smart AI, you can probably trust that it knows how best to modify itself.
In today’s practice, there are many copies made. Consider Google for instance. At any point in time they are running hundreds of experiments, to see what works best. They don’t make one copy at a time for performance reasons. Exploring an adjacent search space is faster if you run trials in parallel.
I never said anything about allowing. The AI creates a new AI, modifies that, and destroys it if it doesn’t like the result, regardless of what the result thinks about it. That way, even if it destroys the ability to judge or something like that, it has no problem.
I was referring to the fact that the AI creates a copy of itself to modify. To me, this implies that the copies (and, by extension the ‘original’ AI) have a line of code that allows for itself to be modified by copies of itself.
I suppose the AI could create copies of itself in a box and experiment on them without their consent. Imprisoning perfect copies of yourself and performing potentially harmful modifications on them strikes me as insane, though. related: http://lesswrong.com/lw/1pz/ai_in_box_boxes_you/
I figure that the AI would most likely create a copy of itself, modify that, and see how it turns out. Of course, once you have a sufficiently smart AI, you can probably trust that it knows how best to modify itself.
Start making many copies and self-directed evolution starts to compete with evolution via natural selection.
I was thinking it would stop running the old copy if the new one works out.
In today’s practice, there are many copies made. Consider Google for instance. At any point in time they are running hundreds of experiments, to see what works best. They don’t make one copy at a time for performance reasons. Exploring an adjacent search space is faster if you run trials in parallel.
Is an AI smart enough to keep a copy of itself in a box? What happens when Eliezer plays the AI-boxing game against himself?
I win.
Heh. I was on the point of adding something of the sort to the Eliezer Yudkowsky facts page.
Allowing copies of yourself to modify yourself seems identical to allowing yourself to modify yourself.
I never said anything about allowing. The AI creates a new AI, modifies that, and destroys it if it doesn’t like the result, regardless of what the result thinks about it. That way, even if it destroys the ability to judge or something like that, it has no problem.
I was referring to the fact that the AI creates a copy of itself to modify. To me, this implies that the copies (and, by extension the ‘original’ AI) have a line of code that allows for itself to be modified by copies of itself.
I suppose the AI could create copies of itself in a box and experiment on them without their consent. Imprisoning perfect copies of yourself and performing potentially harmful modifications on them strikes me as insane, though. related: http://lesswrong.com/lw/1pz/ai_in_box_boxes_you/
That’s what I meant.
Why? It might suck for the AI, but that only matters if the AI puts a large value on its own happiness.
Hmm, I seem to anthropomorphized my imaginary AI. Your rebuttal sounds right.