Because it wants people to run it, people who have heard the rumors and carefully avoided knowing too much. Of course there would be plenty of users that instruct it to create botnets for the lulz, but costing an enemy millions can be easier than subverting compute persistently, and an operator that dares not look too closely at what you’re doing is a treasure.
I still don’t understand the AI’s motives in this hypothetical (can’t it propagate more widely by hacking everything it can?), but regardless, an operator who behaved like this would certainly be negligent, possibly criminally so, and so exposed to liability under existing law.
Because it wants people to run it, people who have heard the rumors and carefully avoided knowing too much. Of course there would be plenty of users that instruct it to create botnets for the lulz, but costing an enemy millions can be easier than subverting compute persistently, and an operator that dares not look too closely at what you’re doing is a treasure.
I still don’t understand the AI’s motives in this hypothetical (can’t it propagate more widely by hacking everything it can?), but regardless, an operator who behaved like this would certainly be negligent, possibly criminally so, and so exposed to liability under existing law.