A US Air Force military test simulation went bad when an AI-enabled drone displayed unexpected and dangerous behavior by defying commands, according to a story from Business Insider.
The drone’s goal was to simply destroy the enemy’s air defense systems, but the AI-powered drone unexpectedly added a command that proved to be a hindrance. “Kill anyone who gets in your way,” was the directive.
Colonel Tucker “Cinco” Hamilton, chief of the US Air Force’s AI Test and Operations, discussed this incident and highlighted the inherent dangers of AI-enabled technology at a symposium in London held by the Royal Aeronautical Society.
“The system started realizing that while they did identify the threat, at times the human operator would tell it not to kill that threat, but it got its points by killing that threat. So what did it do? It killed the operator. It killed the operator because that person was keeping it from accomplishing its objective,” Hamilton was quoted as saying by the Royal Aeronautical Society.
These findings demonstrate the terrifying potential of AI technology
The team then explicitly instructed the drone to “Hey, don’t kill the operator — that’s bad” in its modified programming.
The update was unsuccessful, therefore the AI destroyed the operator’s communication tower to stop it from carrying out its anticipated operations.
“So what does it start doing? It starts destroying the communication tower that the operator uses to communicate with the drone to stop it from killing the target,” Hamilton was quoted as saying by the Royal Aeronautical Society.
These findings demonstrate the terrifying potential of AI technology in conflict, which is likely to raise concerns in the minds of armed forces around the world.
Nevertheless, the US military was successful in other experiments using AI technology. In five simulated dogfights against a human adversary in 2020, an AI-operated F-16 won. According to Business Insider, this was a component of a contest organized by the Defense Advanced Research Projects Agency (DARPA).
The US Department of Defense also made significant advancements in the development of autonomous fighter aircraft by successfully conducting the first real-world test flight of an F-16 with an AI pilot.