The whole “utilizing our atoms” argument is unnecessarily extreme. It makes for a much clearer argument and doesn’t even require super human intelligence to argue that the paperclip maximiser can obviously make more paperclips if it just takes all the electricity and metal that we humans currently use for other things and uses them to make more paperclips in a totally ordinary paperclip factory. We wouldn’t necessarily be dead at that point but we would be as good as dead and have no way to seize back control.
Yes. But also AI will not make actual paperclips for millions and even billions years: it will spend this time for conquering universe in the most effective way. It could use Earth materials for jump start the space exploration as soon as possible. It could preserve some humans as some bargin resource in case it meets other AI in space.
The whole “utilizing our atoms” argument is unnecessarily extreme. It makes for a much clearer argument and doesn’t even require super human intelligence to argue that the paperclip maximiser can obviously make more paperclips if it just takes all the electricity and metal that we humans currently use for other things and uses them to make more paperclips in a totally ordinary paperclip factory. We wouldn’t necessarily be dead at that point but we would be as good as dead and have no way to seize back control.
Yes. But also AI will not make actual paperclips for millions and even billions years: it will spend this time for conquering universe in the most effective way. It could use Earth materials for jump start the space exploration as soon as possible. It could preserve some humans as some bargin resource in case it meets other AI in space.