We seem to be in the midst of a trend away from direct confrontation and toward sowing discord. We want to be able to spread chaos in the ranks of our opponents, and have that intervention be perceived as a failure in our opponents’ ability to enforce discipline or coordinate effectively. As a concrete example, Russia may interfere with American social media websites in order to promote violent mysogyny or racism, provoking attacks by Americans on Americans. Americans may then blame the perpetrators, our own citizens, and our own culture for these outcomes, even if Russian interference was a necessary condition for many of these attacks.
I could imagine that AI will enhance the ability of militaries and intelligence organizations to execute such attacks in other ways. For example, we could imagine a whole field of “bug design,” in which AI systems are created and sold with deliberately built-in bugs that nonetheless appear accidental, and cause the AI to act up at some pre-determined point in the future. AI might not just help programmers write code—it might help them write buggy code that nevertheless appears bug-free. The outcome can be framed as an accident, the developers of the weaponized bug may be impossible to identify, and it becomes very difficult to know what a proportional response looks like.
Another example would be a country imposing healthy eating incentives on its own population, while subsidizing the export of tasty and unhealthy foods to its adversaries. Then the population in the other country blames itself, or its own economic structure, for the spread of obesity.
I feel OK about posting this here, because I think that on net, the risk of spreading new conceptual ideas to an attacker is lower than the value of a population considering how to defend itself against such attacks on its sense of responsibility. The attack can be done by a small number of specialists. Defending against such an attack will require a heightened level of awareness at the population level, and that requires free and open discussion.
We seem to be in the midst of a trend away from direct confrontation and toward sowing discord. We want to be able to spread chaos in the ranks of our opponents, and have that intervention be perceived as a failure in our opponents’ ability to enforce discipline or coordinate effectively. As a concrete example, Russia may interfere with American social media websites in order to promote violent mysogyny or racism, provoking attacks by Americans on Americans. Americans may then blame the perpetrators, our own citizens, and our own culture for these outcomes, even if Russian interference was a necessary condition for many of these attacks.
I could imagine that AI will enhance the ability of militaries and intelligence organizations to execute such attacks in other ways. For example, we could imagine a whole field of “bug design,” in which AI systems are created and sold with deliberately built-in bugs that nonetheless appear accidental, and cause the AI to act up at some pre-determined point in the future. AI might not just help programmers write code—it might help them write buggy code that nevertheless appears bug-free. The outcome can be framed as an accident, the developers of the weaponized bug may be impossible to identify, and it becomes very difficult to know what a proportional response looks like.
Another example would be a country imposing healthy eating incentives on its own population, while subsidizing the export of tasty and unhealthy foods to its adversaries. Then the population in the other country blames itself, or its own economic structure, for the spread of obesity.
I feel OK about posting this here, because I think that on net, the risk of spreading new conceptual ideas to an attacker is lower than the value of a population considering how to defend itself against such attacks on its sense of responsibility. The attack can be done by a small number of specialists. Defending against such an attack will require a heightened level of awareness at the population level, and that requires free and open discussion.