SUBSCRIBER+EXCLUSIVE REPORTING — The marvels of artificial intelligence caught the imagination again this month when Frank Kendall, the secretary of the Air Force, took a spin in an AI-controlled F-16 fighter that flew a mock dogfight against a human-piloted opponent.
The AI-driven F-16 – known as the X-62 Vista — maneuvered to within 1,000 feet of a piloted F-16, as each fighter tried to expose a vulnerability. In other words, a human being and an AI-piloted war machine were feeling one another out, and angling for an edge in the skies.
“When you’re in an engagement like that, what the two pilots are trying to do to gain an advantage is fly the most optimal possible trajectory for their aircraft,” Kendall said, describing the experience. “We (the AI-piloted plane) were up against a pilot who had two or three thousand hours of experience. He was very good. It was roughly an even fight. But against a less experienced pilot the AI … [and] the automation would have performed better.”
The flight represented the latest step in the Pentagon’s drive to develop “autonomous” systems by exploiting the staggering computational capabilities of AI. The Air Force has set a goal for a fleet of more than 1,000 unmanned combat aircraft in the coming years. The Navy, Army, and Space Force have parallel autonomy projects in their respective domains.
Kendall and other senior military officials say the goals are driven in part by the knowledge that America’s rivals and potential adversaries are already embarked on similar paths.
“This is the path of the future,” he said. “That’s where we’re going.”
To date, no one has deployed a fully autonomous weapon – which Paul Sharre, Executive Vice President of the Center for a New American Security, defines as a device so advanced, it completes “the engagement cycle — searching for, deciding to engage, and engaging targets — on its own.”
“In my opinion there are no truly autonomous weapons,” Gen. Philip Breedlove (Ret.), a former Supreme Allied Commander Europe, told The Cipher Brief. “There are human-configured, human-trained, human-launched weapons that then go out into human-designated areas and situations and kill targets that they autonomously recognize as the target.”
Either way, it’s a whole new kind of warfare.
A China-U.S. race for autonomous weapons
Even if the “killer robots” of Hollywood and popular imagination are not at hand, experts agree that in several states hostile to the U.S., ambitious research and development is underway in the autonomous military space. China has signaled its intent to make autonomous and AI-enhanced systems a centerpiece of its future force structure and strategy for warfare.
According to the latest DoD report on Chinese military and security developments, “intelligentized technology” — Pentagon-speak for unmanned systems — is a critical area for development in China’s People’s Liberation Army (PLA). This includes “pursuing greater autonomy for unmanned aerial, surface, and underwater vehicles to enable manned and unmanned teaming, swarm attacks, optimized logistic support, and distributed ISR (Intelligence, Surveillance, and Reconnaissance), among other capabilities.”
Translation: China is looking to develop a wide range of AI-driven capabilities across all the branches of its military.
That said, detailed insights into the scope and specifics of Chinese systems are hard to come by. Martijn Rasser, a former CIA analyst and now managing director of the Datenna China intelligence portal, told Reuters recently that “the biggest challenge is we don’t really know how good the Chinese are, particularly when it comes to the military applications of AI.” Rasser credits China with world-class technology R&D but added that “what the PLA and PLA-affiliated research institutions are doing specifically is much more difficult to discern.”
What we do know is that China – joined by like-minded regimes in Russia, Iran, and North Korea – has put a premium on out-dueling the U.S. in the race for the smartest AI weapons systems.
In an interview with The Cipher Brief, Dr. Stacie Pettyjohn, director of the Defense Program at the Center for a New American Security (CNAS), said that China has “a lot of drones that look strikingly similar to American systems like the Reaper and the Predator and the Global Hawk.” But she also said China has expanded autonomous applications beyond those fielded by the U.S., with platforms “on their naval vessels performing maritime domain awareness and warfare operations.”
Just how invested China is in maritime autonomous capabilities was demonstrated by the recent journey of a crewed research ship, Zhu Hai Yun, which is laden with autonomous maritime drones.
While China said the ship’s mission involves solely civilian scientific research, a recent analysis by the Center for Strategic and International Studies (CSIS) concluded that “the lines separating the Zhu Hai Yun and the People’s Liberation Army (PLA) are exceedingly blurred.”
Late last year, CSIS monitored the vessel’s odd navigation routes around Taiwan, a unique and carefully calculated route that suggests it went beyond a standard research mission.
According to CSIS, the Zhu Hai Yun “stands out even among the most advanced ships in China’s research fleet,” serving as a “mother ship” for an array of AI-enabled unmanned air, surface, and undersea vehicles and as a platform to live-test drone-swarm operations. The true purpose of the Zhu Hai Yun’s Taiwan operation remains unknown, but at a minimum the ship’s advanced capabilities point to dual-use applications of interest to the PLA.
Pettyjohn stressed that these and other new dual-use drones would have potentially dramatic impact in the event of any conflict in the Taiwan Strait. The PLA Rocket Force, she said, “has a bunch of military drones that…are more than adequate to help with targeting and closing kill chains from the Chinese mainland out to Taiwan.”
The gathering “swarm”
Experts say that in the event of a Taiwan conflict, there is a real danger of Taiwanese forces - and possibly U.S. forces as well, should they come to Taiwan’s aid - being outnumbered and overwhelmed by unmanned craft in the air, on the water, and even beneath the sea.
These scenarios are reportedly among the motivations behind the Pentagon’s Replicator initiative, which was announced last August and recently revealed its first projects. These include bulk buys of Switchblade-600 UAV loitering munitions, and development of uncrewed surface vehicles and counter-drone systems. The U.S. has provided hundreds of Switchblade “kamikaze” drones to Ukraine, where they have proved their value as lightweight, collapsible weapons allowing soldiers to carry and deploy them quickly.
Beyond a buying spree of AI-enhanced gear, experts say the latest technology will demand a whole new approach for the Pentagon.
Gen. Breedlove believes the U.S. needs “to completely rethink the way we defend against drone armies - land, air, sea, and undersea.” While U.S. long-range bomber fleets and aircraft carrier battle groups are more advanced than anything currently in a potential enemy’s arsenal, Breedlove says “these large capital assets are more vulnerable than we realize to drones and new swarming and drone attack profiles.” Consequently, he says, “we are going to have to really shift our emphasis to those types of threat vectors.”
The war in Ukraine has become a real-time testbed for a variety of lethal autonomous weapons. Pettyjohn said that while she agrees with Breedlove that there are no truly “autonomous” systems being used in the war - a more “narrowly bounded autonomy” appears to be in play, as both sides deploy drones capable of automated takeoff-and-landing, collision avoidance, object recognition, and navigation.
And these capabilities are constantly being refined and expanded. Serhii Kuprienko, founder and CEO of the Ukrainian drone company Swarmer, said recently that AI-enabled drones “can do in seconds what it would take a human several hours.”
For its part, Russia has made extensive use of Orlan and Zala drones for surveillance, reconnaissance and targeting, and Lancet loitering munitions for precision strikes. The combined battlefield value of these platforms, Pettyjohn notes, has revealed itself in effective Russian attacks on “high-value targets like air defenses and for counter-battery fire.”
Warfare analyst Zak Kallenborn adds that Russian Lancet attack drones have been equipped with Nvidia AI chips to enable some autonomous capabilities, but to uneven effect. “It's not clear that the capability is widespread,” he told The Cipher Brief, “nor that it works particularly well. In fact, there's been some evidence the autonomy is generally not great.” But Kallenborn added that “Russia, China, Iran, and others are definitely pursuing (drone-) swarming capabilities.”
And in a comment echoing Breedlove’s concern, Kallenborn told The Cipher Brief that these nations “recognize that cheap mass could be useful in defeating expensive, exquisite American platforms. Or at least holding them back.”
The proliferation fears
With China, Iran and North Korea in mind, another critical dimension of the weaponized AI threat involves proliferation.
Gregory Allen, director of the Wadhwani Center for AI and Advanced Technologies at the Center for Strategic and International Studies (CSIS), told a congressional hearing last spring that Ziyan, a Chinese military drone manufacturer, has marketed an unmanned helicopter drone dubbed Blowfish in the Middle East and South Asia.
The Blowfish, Allen noted, “autonomously performs more complex combat missions, including fixed-point timing detection, fixed-range reconnaissance, and targeted precision strikes.”
Meanwhile, Algeria, Egypt, Iraq, Jordan, Saudi Arabia, the United Arab Emirates, and Pakistan are among countries that have purchased AI-enabled Chinese drones, including the advanced Wing Loong II and CH-5 UAV’s.
China also has showcased a jet-powered drone designed to accompany manned combat aircraft formations, a clear parallel to the ongoing U.S. project to deploy its Collaborative Combat Aircraft (CCA) drone. Later this year, China’s International Aviation and Aerospace Exhibition will boast an "exhibition zone" for unmanned equipment inclduing UAVs and unmanned ships.
Pettyjohn singled out Iran as a prime proliferator of autonomous weapons, citing its collaboration with Russia in the export and manufacturing of Shahed attack drones. Tehran has “a whole fleet of different types of drones…cropping up in Africa and other places,” she said, adding that Iran “is definitely the biggest supplier of non-state groups, whether it's the Houthis, Hezbollah, Hamas, or its proxies in Iraq.”
Gen. Breedlove pointed to a related concern – the lack of constraints on both source nations and proxies when using AI–enabled weapons. The U.S. and its allies, he said, will apply safeguards on both the capabilities and sale of lethal systems, unlike “our opponents, who will be eminently less concerned with collateral damage in the way these weapons will be used.”
These threats are not lost on the world’s diplomats. Last month an international conference — Humanity at the Crossroads: Autonomous Weapons Systems and the Challenge of Regulation – took up the issue in Vienna. Austrian Foreign Minister Alexander Schallenberg warned of the likelihood that AI-driven weapons "very quickly will end up in the hands of non-governmental actors or terrorist groups."
"Now is the time to agree on international rules and norms to ensure human control," Schallenberg said. It is the world community’s responsibility, he added, to ensure that "who lives and who dies remains in the hands of humans and not of machines."
Read more expert-driven national security insights, perspective and analysis in The Cipher Brief because National Security is Everyone’s Business.
