The Pentagon is working to solve vulnerabilities in its AI systems that could be exploited by attackers using visual trickery or manipulated signals. Their research initiative, Guaranteeing AI Robustness Against Deception (GARD), has been looking at these “adversarial attacks” since 2022.
Researchers have demonstrated how seemingly innocent patterns might trick AI into misidentifying objects, potentially leading to terrible outcomes on the battlefield. An AI, for example, may mistake a bus with passengers for a tank if it is labeled with the appropriate “visual noise.”
These concerns come amid widespread public worry about the Pentagon’s development of autonomous weapons. To address this, the Department of Defense recently changed its AI development guidelines, emphasizing “responsible behaviour” and requiring permission for all deployed systems.
The GARD program has made strides in developing defenses against such threats. They’ve even given some tools to the newly established Defense Department’s Chief Digital and AI Office (CDAO).
However, some advocacy groups are concerned. They are concerned that AI-powered weapons may misinterpret conditions and attack without cause, even if signals are not purposely manipulated. They warn that such weapons could cause unintended escalation, particularly in volatile locations.
The Pentagon is actively modernizing its arsenal with autonomous weapons
The Pentagon is actively modernizing its arsenal with autonomous weapons, emphasizing the importance of resolving these weaknesses and guaranteeing responsible development of this technology.
According to a statement from the Defense Advanced Research Projects Agency, GARD researchers from Two Six Technologies, IBM, MITRE, the University of Chicago, and Google Research created the following virtual testbed, toolbox, benchmarking dataset, and training materials, which are now available to the larger research community:
- The Armory virtual platform, available on GitHub, acts as a “testbed” for researchers seeking repeatable, scalable, and rigorous evaluations of adversarial defenses.
- Adversarial Robustness Toolbox (ART) helps developers and researchers protect and evaluate their machine learning models and applications against a variety of adversarial challenges.
- The Adversarial Patches Rearranged In COnText (APRICOT) dataset allows for repeatable research on the real-world effectiveness of physical adversarial patch attacks on object detection systems.
- The Google Research Self-Study repository includes “test dummies” that illustrate a common notion or method for building defenses.