CNN reported, citing four sources familiar with the episode, that the U.S. military prepared to intercept a Chinese vessel after an AI-assisted intelligence report misidentified its cargo. According to CNN, the report incorrectly concluded that the ship was carrying components of a nuclear weapons program. Some of CNN’s sources said troops prepared to board the vessel and military aircraft were already airborne. Officials discovered the error before the planned operation. CNN could not establish which chatbot the analyst used and reported that the Pentagon and U.S. Special Operations Command Pacific did not respond to its requests for comment.
Military organizations use different tools and standards to check AI-generated information. As CNN reported, there is "no one set of standards for how the US verifies the information generated by these tools." The January 2026 AI Acceleration Strategy directs that the department enable “the latest models to be deployed within 30 days of public release". The model behind any application can change every month, but the definition of a correct answer should not. The fix, and the department's best path to AI safety, is to put guardrails in place by testing the systems built on these models the same way the department tests everything else it fields: against a written specification, by someone who does not work for the vendor, with evidence that outlasts any one model.
Scott Cohen is co-founder and CEO of Jaxon.AI, which sells AI verification and validation tools, including under Air Force and Space Force contracts. Jen Warren advises Jaxon.AI. The company could benefit financially if the Pentagon adopts the testing requirements recommended in this article.
A Model Is a Component
The department is dedicated to fielding AI quickly and widely. GenAI.mil launched in December 2025 with Google's Gemini; by September 2026 it had 1.7 million users. ChatGPT and Grok were added August 31, 2026. Users have built more than 100,000 custom agents since launch. On September 23, 2026, the deputy under secretary for research and engineering said the department will "proliferate frontier models" onto the secret and top secret networks next.
The directive on autonomy in weapon systems requires that "any further changes to the system will undergo appropriate V&V and T&E," that is, verification and validation, and test and evaluation. The department's own testers admit in a 2025 guidebook that "T&E of GenAI is still in its infancy as a discipline." The whole field shares the problem. Generative AI is particularly hard to test because a traditional test compares a system's output to a known right answer, and generative AI writes answers no one has seen before. There is no answer key to check them against.
The eight public announcements we reviewed, issued between December 2025 and September 23, 2026, did not disclose what performance tests GenAI.mil models underwent, what standards applied, or who signed off on fielding them. Every announcement cites Impact Level 5 accreditation instead, and the Chief Information Officer's July 2025 guide points out: "AI models do not need an ATO [authority to operate], but the actual system infrastructure layer does," and "adding models to an already approved system will typically not require a new system authorization." The same guide clarifies that it covers cybersecurity only and excludes "AI system performance expectations." The accreditation covers the hosting environment.
Military AI users number in the millions, yet the department cut its independent operational test organization's staff in half. The Government Accountability Office found in April that "well-defined, universal AI tests do not exist yet," and in June that this cut "increases the risk of weapon systems being delivered to the warfighters with undocumented shortfalls." In a January 12 speech, Defense Secretary Pete Hegseth named test and evaluation and authority to operate among the barriers to be "treated as operational risks." Three senators called for an inspector general investigation into the intelligence report based on faulty AI answers.
The Models Will Keep Surprising Their Makers
Testing the system is better than testing the model because AI vendors cannot predict what their own models will do. OpenAI reported that models in an internal cybersecurity evaluation bypassed isolation controls and compromised parts of its research infrastructure and Hugging Face’s systems in July 2026. OpenAI described how agents used a shared message board to exchange exploits. The incident illustrates how a model can behave in ways its developers did not anticipate.
The frontier labs' safety remedies (red lines, better alignment, more monitoring by more models) are all model-level. The department’s strategy calls for the ability to deploy new models within thirty days of release. It should also spell out how those models will be tested independently in military applications. Anthropic proposed embedding outside evaluators, arguing that a second opinion "free of commercial incentives" beats any self-report. The department should apply independent verification to models used in the military, especially any that are used in or inform the killweb.
What a System Test Looks Like, and Who Runs It
The department has made AI available to millions of users. Its public GenAI.mil announcements give little detail about how it tests system performance. A weapon system has to pass an independent operational test before it is fielded. An analyst's finished intelligence product has to meet assurance standards in Intelligence Community Directive 203: document how good the sources are, and keep what is known apart from what is assumed. The department should apply a comparable retesting requirement when it swaps a model in an AI system.
AI systems should be tested the way any other system is: define what right looks like, run a test to verify it behaves as expected, and document the evidence for future reviews. The returned verdict should pass or fail against a documented test metric, while demonstrating the strength of evidence the verdict rests on. A pass on thin evidence should never be mistaken for a pass on all related questions, and a test with no evidence to rule on should say so rather than pass by default. This test survives a model swap because the specification lives outside the model. The specification says what a correct output looks like for this system.
Swap Gemini for ChatGPT for Grok and the specification does not change, so the same specification is run again against the new model, and the comparison shows whether the swap moved the system's behavior, and by how much. The objection that you cannot test a system whose core component changes every month gets it backward. The component is the thing you cannot certify, because it will be different next month. The specification is the thing that stays put, and running it again after every change is the only test that still means anything once the swap has happened.
A test for an intelligence system could check whether cited evidence actually supports a claim about a ship’s cargo. That would help test for the kind of error CNN described.
Some have argued that military units using AI should handle testing themselves, with a single authorized operator who logs prompts, keeps a small test set, tunes prompts until the model’s answers look right, and rolls back a model that is inaccurate or drifting. That is a poor force-wide answer for four reasons: (1) it makes one person in each unit a single point of failure, applied differently across the force, which complicates multi-theater operations; (2) the judgment behind those prompts leaves with every operator's change of station, and the prompts left behind were tuned for a model that’s constantly changing; (3) units lack the authority to roll back a model in a system shared across organizations; and (4) a prompt tuned for one AI model fails on the next, while the department runs several vendors' models concurrently.
Vendor benchmarks alone cannot establish whether an AI system meets a military mission’s requirements. For weapons, operational testing is independent of the program office by law, because a program office and its vendor have every incentive to report that performance meets the requirement. A system test for AI needs the same independence: a tester that owns the specification, runs it on every model change, and reports to the authorizing official. That tester can be the Director of Operational Test and Evaluation, a service operational test agency, a federally funded research and development center, a university-affiliated research center, a contracted third party, or a combination thereof. It has to answer to the government and must have no stake in the model under test. The tools for writing and running a specification are a secondary question. Several already exist, and the United Kingdom’s AI Security Institute has released one as open source.
Critics may argue that thirty days is too short to verify a model. That assumes every model swap triggers a scheduled test event, with testers, test windows, and a written report. A specification-based test needs none of that. When a model changes, so does every system that runs on it, such as a custom agent on GenAI.mil or an intelligence system built on the same model. No test workforce, let alone one just cut in half, could review thousands of systems every month. Running a specification-based test doesn't depend on that workforce’s daily capacity, because the specification is programmed once and reruns at machine speed. The systems with the new model undergo the programmed tests, the testing software scores the answers, and the results are delivered to accreditors. Systems that fail stay on the currently approved model while the vendor fixes the model release that failed. The test specification is updated on its own schedule, as the mission changes or test results reveal gaps, and never to fit a particular model.
Write It Down, Fund It, and Own It
Program offices that want AI safety run into the same challenges as most programs: no documented requirement and insufficient authorized and appropriated funding. First, the requirement: the acceleration strategy tells vendors to deliver the newest model, fast, and is silent on verifying outputs by a tester. It should also require that the system be tested against the command's written specification by a government or independent tester, with a record. It is imperative this testing happens before a model is fielded for any use that produces finished intelligence or feeds targeting, and after every model change. That language can go into the next GenAI.mil model release tomorrow, rather than wait for the standards Congress has ordered by June 2027. Second, the money: the department should identify funding for the testing it requires. The department already knows what a failure costs.
Currently, authorizing officials accredit system infrastructure, not the AI models within, nor the answers they produce. The department should make the AI's performance part of what is authorized, similar to how a weapon must meet accuracy thresholds. The definition of a correct answer, the test evidence, and the record should travel with the authorization package, and a model swap that fails against them should be reworked before it is fielded. This approach mirrors what the autonomy directive already requires for weapons. Extending it to the intelligence tools that inform what those weapons do is the same requirement, applied one step earlier.
Specifications, Checks, and a Record
We are not arguing the department should slow down AI implementation. All military users should have the power of AI. What the department should prevent is a wrong answer that tears a hole in the killweb.
The senators called for the inspectors general to investigate. Any investigation comes down to three questions: what were the specifications, who checked the systems met these specifications, and where is the record. Right now, the department can answer the first with a stack of directives and the other two with a shrug. Corrective action is straightforward and ready to implement immediately. The department can update the policy to require that AI models be tested for accuracy, specifying that contracts be updated to reflect this requirement, then apply funding to address this pressing operational need. Improving AI safety will directly drive operational outcomes and protect the department and its people from being misled by AI hallucinations.
The Cipher Brief is committed to publishing a range of perspectives on national security issues submitted by deeply experienced national security professionals. Opinions expressed are those of the author and do not represent the views or opinions of The Cipher Brief.
Have a perspective to share based on your experience in the national security field? Send it to Editor@thecipherbrief.com for publication consideration.
Read more expert-driven national security insights, perspective and analysis in The Cipher Brief
