Picture an intelligent analyst's screen. Data about troop movements, a port closure, a terrorist attack, or a weapons transfer sits at the center of a clean interface, with a confidence indicator glowing green. Two hundred outlets are carrying it, and the platform has counted every one. It looks like a corroboration. In many cases, it is a single Telegram post, copied, translated, summarized, and republished until its origin disappeared behind its own echo.
Nobody must be negligent for this to happen. It is what you get when a product or platform is built to measure volume and the user is tempted to read volume as truth.
I say this as someone who works with open-source material and believes in it. Commercial imagery, ship-tracking data, procurement records, leaked databases, and social media have changed what is knowable about conflicts and illicit networks; but the market that has grown up around these sources is selling something different from the discipline that makes them valuable, and governments and companies are buying it without noticing the difference.
A Crowded Market and a Thin Product
Open-source intelligence platforms are multiplying quickly. Walk through the company stands at any security conference and you will find dozens of them, each promising to fuse, monitor, and surface threats across the open internet, now almost always with an AI layer that summarizes, ranks, and alerts.
Strip away the interface and most of them offer little that a competent analyst couldn't already do with a search engine, a handful of specialist tools, and some patience. Reverse image searching, scraping social platforms, pulling corporate registry data, and geolocating an image are all learnable skills, and many of the best practitioners picked them up in public, often as volunteers. What the platforms add is convenience and scale, which is genuinely useful, but convenience is not analysis. The real product, in most cases, is the dashboard.
Dashboards are persuasive by design. They squeeze a messy information environment into something that looks like a picture of the world, with counts, trend lines, heat maps, and sentiment scores. Each of those findings implies that someone has already done the hard work of deciding what matters and what is true. Usually, no one has. The system has counted things, and counting is the easy part.
Counting Sources Is Not Testing Them
The core flaw is in what these systems reward. They tend to measure the number of sources behind a claim, not the independence of those sources. The two are very different things.
Independence is the foundation of corroboration. Two witnesses who saw an event separately make a claim stronger. Two witnesses who heard about it from the same third person do not. The same logic applies to reporting. A claim that traces back to a single channel but appears in two hundred outlets has one source and two hundred echoes.
Intelligence tradecraft has a name for this: circular reporting. It is among the oldest and best-documented ways analysts go wrong. The failures that most damaged the credibility of Western intelligence in the past generation involved, among other things, uncorroborated claims that were repeated, cross-referenced, and eventually treated as established because they seemed to come from many places. The lesson drawn at the time was that provenance matters more than prevalence. An aggregator that treats prevalence as confirmation is rebuilding the vulnerability that analysts spent years learning to guard against.
AI makes this problem fourfold. First, language models summarize and synthesize, and in doing so they take away the provenance trail that an analyst needs. A fluent paragraph that blends fifteen sources does not tell you that twelve of them are derivative. Second, the cost of generating plausible content has collapsed. Fabricated articles, cloned news sites, and coordinated amplification networks can now create the appearance of breadth cheaply, which is exactly the signal these platforms are built to count. Third, as more web content is produced or recycled by automated systems, the pool that analytics tools draw from becomes more self-referential. The echo chamber is increasingly populated by machines quoting each other. Lastly, AI can make uncertainty harder to see. Language models are designed to produce coherent answers, not to preserve the ambiguity of the evidence on which those answers are based. Contradictory sources can be reconciled into a smooth narrative, gaps can disappear beneath plausible inference, and tentative conclusions can acquire the tone of established fact. The danger is therefore not simply that an analyst receives false information, but that the uncertainty surrounding otherwise legitimate information becomes less visible.
None of this means the technology is useless. Entity extraction, translation, deduplication, and pattern detection across large data sets are real contributions. But the same tools that can help trace a claim to its origin are too often deployed to certify it instead.
Information Is Not Intelligence
The deeper problem is what happens after the dashboard produces its output. Companies market it as intelligence, and buyers, including government agencies, intelligence firms, financial institutions, and corporate security teams, purchase it as intelligence, then act on it.
However, open-source material is raw information. It becomes intelligence only after a process that no interface can fully automate. Someone has to establish where the material came from, grade how reliable the source has been, assess how credible the specific claim is, and test it against other collection. Established standards exist for this. Grading systems that rate source reliability separately from information credibility have been used for decades, and analytic standards require analysts to describe the quality and credibility of their sources, to distinguish between underlying evidence and judgment, and to account for alternative explanations. These are not bureaucratic formalities. They are the mechanism by which an organization knows how much weight a conclusion can bear.
When that step is skipped, the result is aggregated internet content with a confident interface. The output reads like a finished product, but the epistemic work behind it is missing, and the person reading it often can't see that. A decisionmaker shown a green indicator does not know whether it reflects three independent confirmations or one rumor with good distribution.
The consequences are practical. A government that bases sanctions designations, force posture, or diplomatic messaging on an unverified claim risks embarrassment at best and escalation at worst. A company that reroutes shipments or exits a market because a risk platform flagged a rumor can lose real money; and an agency that comes to rely on commercial feeds may quietly stop investing in the capabilities that would let it check them.
What Should Change
I would offer the following two principles.
One, to treat provenance and independence as essential to intelligence. Open-source material should be treated as a cue and a corroborator, not a conclusion. Open sources are at their best when they point collection in a useful direction or when they test and strengthen what other sources have suggested. A satellite image that shows construction at a site tells analysts where to look and what to ask. A procurement database entry might confirm a pattern already suspected. Neither is a finding on its own. The workflow should run from open-source lead to targeted collection to assessment, not from dashboard alert to briefing slide. As AI makes it easier to summarize, reproduce and generate information, analysts need to know not only what a source says, but where the information originated and whether apparently separate sources are actually independent. A fluent paragraph that blends fifteen sources does not tell an analyst that twelve of them are derivative. Likewise, a cluster of articles, websites or database entries may create the appearance of broad corroboration while ultimately tracing back to a single underlying claim. Platforms should therefore preserve provenance wherever possible and make the distinction between original reporting, derivative content and AI-generated material visible to the user. The workflow should run from open-source lead to targeted collection to assessment, not from dashboard alert to briefing slide.
Two, protect human assessment and the collection that supports it. AI can make uncertainty harder to see. Language models are designed to produce coherent answers, not to preserve the ambiguity of the evidence on which those answers are based. Contradictory sources can be reconciled into a smooth narrative, gaps can disappear beneath plausible inference, and tentative conclusions can acquire the tone of established fact. Human analysts therefore need both the time and the authority to question what automated systems produce rather than simply validate it. That also means protecting the collection capabilities that AI cannot replace. As open-source material becomes more abundant, the value of what cannot be found online rises. Human sources, financial intelligence, law enforcement access and legal process take years to cultivate and moments to lose. A budget cycle that trims them in favor of subscriptions is trading durable capability for a license that expires. The objective should not be to remove automation from intelligence work, but to ensure that automation remains subordinate to collection, verification and human judgment.
What Open Sources Cannot Close
I learned the limits of open-source work firsthand while researching foreign components in Russian strike drones. Open sources got me a long way. Recovered wreckage data, teardown imagery, part markings, and manufacturer documentation could establish what was inside these systems and who made it. That is valuable, and it was achievable by skilled analysts working in public.
But the questions that mattered most were different. Who actually bought a given part? Through which intermediaries did it move? Who paid for it, and through which accounts? Those gaps could not be closed by any amount of scraping. They required the tools that sit behind a legal threshold or inside a trusted relationship: law enforcement access to shipping and customs records, financial intelligence on transactions, and human sources with knowledge of how the networks operate. A dashboard could tell us a component existed. Only that other collection could tell us how it got there and who was responsible, which is the information policymakers need if they want to disrupt the supply chain rather than simply describe it.
That experience shaped how I think about the current issue. Open sources and closed collection are not competitors. The first made the second more efficient, because it let investigators ask better questions. The second made the first more meaningful, because it turned an interesting observation into an attributable finding. Treating either as a substitute for the other would have left us with a weaker product.
Defending the Analytical Craft
None of this is an argument against open-source intelligence. It is an argument for it, properly understood.
The practitioners who built the field's reputation, whether in newsrooms, nonprofits, academia, or government, are rigorous precisely because they distrust easy answers. They document their methods, show their chains of evidence, and say so when something can't be confirmed. That culture is the real asset. It is also what the commercial market is most at risk of undermining, because rigor is slow and expensive to sell, while a dashboard can be demoed in ten minutes.
The test I would offer any buyer, policymaker, or analyst is simple. When you are shown a claim, ask where it began, who else has independent knowledge of it, and what other collection could confirm or contradict it. If the system in front of you cannot answer those questions, it hasn't given you intelligence. It has given you a well-organized rumor.
The volume of open information will keep growing, and the tools to process it will keep improving. The scarce resource will not be data. It will be judgment, along with the access and tradecraft that make judgment possible. Agencies and firms that remember that, and invest accordingly, will see more clearly than those who confuse a full screen with a full picture.
The Cipher Brief is committed to publishing a range of perspectives on national security issues submitted by deeply experienced national security professionals. Opinions expressed are those of the author and do not represent the views or opinions of The Cipher Brief.
Have a perspective to share based on your experience in the national security field? Send it to Editor@thecipherbrief.com for publication consideration.
Read more expert-driven national security insights, perspective and analysis in The Cipher Brief
