This post doesn’t seem to be making a logical argument. That could be why it’s getting downvotes.
It seems like you’re arguing that since paperclips don’t last forever (without protection), you can’t make as many paperclips as possible. This doesn’t seem to logically follow. It seems like you just need to include protecting those paperclips as part of your efforts to make as many as possible.
Your concluding paragraph says that, since making paperclips isn’t completely thought out, there’s no reason to think there’s a problem with an agent that pursues a goal singlemindedly being dangerous. That doesn’t follow either.
It seems like you could use the same logic and say that Napoleon trying to conquer Europe makes no sense, since Europe isn’t completely defined, and it won’t stay conquered forever. But if you used that logic to choose your actions, you’d wind up dead when Napoleon and his armies marched through (taking your food as a subgoal of their perhaps poorly-defined goal) despite their objectives being incompletely defined.
You can’t use sophistry to avoid physical death. And poorly defining an agent’s goals won’t make the outcome any better for us, just worse for those goals getting accomplished.
This post doesn’t seem to be making a logical argument. That could be why it’s getting downvotes.
It seems like you’re arguing that since paperclips don’t last forever (without protection), you can’t make as many paperclips as possible. This doesn’t seem to logically follow. It seems like you just need to include protecting those paperclips as part of your efforts to make as many as possible.
Your concluding paragraph says that, since making paperclips isn’t completely thought out, there’s no reason to think there’s a problem with an agent that pursues a goal singlemindedly being dangerous. That doesn’t follow either.
It seems like you could use the same logic and say that Napoleon trying to conquer Europe makes no sense, since Europe isn’t completely defined, and it won’t stay conquered forever. But if you used that logic to choose your actions, you’d wind up dead when Napoleon and his armies marched through (taking your food as a subgoal of their perhaps poorly-defined goal) despite their objectives being incompletely defined.
You can’t use sophistry to avoid physical death. And poorly defining an agent’s goals won’t make the outcome any better for us, just worse for those goals getting accomplished.