If military AI is dangerous, it’s not because it’s military. If a military robot can wield a gun, a civilian robot can certainly acquire one as well.
The military may create AI systems that are designed to be amoral, but it will not want systems that overinterpret orders or violate the chain of command. Here as everywhere, if intentional misuse is even possible at all, alignment is critical and unintentional takeoff remains the dominant risk.
In seminal AI safety work Terminator, the Skynet system successfully triggers a world war because it is a military AI in command of the US nuclear arsenal, and thus has the authority to launch ICBMs. This, ironically to how it is usually ridiculed, gets AI risks quite right but grievously misjudges the state of computer security. If Skynet was running on Amazon AWS instead of a military server cluster, it would only be marginally delayed from the same outcome.
The prompting is not the hard part of operating an AI. If you can talk an AI ship into going rogue, a civilian AI can talk it into going rogue. This situation is inherently brimming with doom- it is latently doomed in multiple ways- the military training and direct access to guns merely removes small roadbumps. All the risk materialized at once, when you created an AI that had the cognitive capability to conceive of and implement plans that used a military vessel for its own goals. Whether the AI was specifically trained on this task is, in this case, really not the primary source of danger.
“My AI ship has gone rogue and is shelling the US coastline.”
“I hope you learnt a lesson here.”
“Yes. I will not put the AI on the ship next time.”
“You may be missing the problem here—”
I think your 100 billion people holding thousands of hands each are definitely conscious. I also think the United States and in fact nearly every nationstate are probably conscious as well. Also, my Linux system may be conscious.
I believe consciousness is, at its core, a very simple system: something closer to the differentiation operator than to a person. We merely think that it is a complicated big thing because we confuse the mechanism with the contents—a lot of complicated systems in the brain exchange data using consciousness in various formats, including our ego. However, I believe just consciousness—the minimal procedure in itself—is simply not actually very mysterious or complicated as far as algorithms go.
A meme, in text form:
“Textbooks from the Future”
A time traveller handing a professor a book, simply labeled “Consciousness”. The man is clearly elated, as the book is very thin. Behind the time traveller’s back is a trolley carrying a very thick book labelled “Cognitive algorithms that use consciousness, vol. 1.”