OPINION — The intelligence community (IC) has long struggled with the idea of Open Source intelligence, or OSINT. For the last 80 years, it has attempted in fits and starts to incorporate public information in intelligence analysis, never finding the right fit for the collection or the tradecraft. But today, effectively averting strategic surprise means uncovering secrets hidden in plain sight and using IC talent to sort and process them. To get there, we must engage in a radical rework of both the concept of Open Source intelligence and the tools the IC uses to exploit it.
Intelligence agencies were created to discover what adversaries attempt to hide. Generations of leaders thought it best to preserve the IC’s exquisite capabilities for high-risk clandestine and covert operations. They assumed that what policymakers needed from open source, they would get from the Washington Post or cable news.
That approach made sense in the past, but today it means standing, frozen, on a proverbial burning platform. China has shifted decisively toward collection of data, training data scientists, and developing Artificial Intelligence/Machine Learning (AI/ML) to exploit it. Meanwhile, the U.S. government acknowledges that AI is the future but is merely talking about the problem as the platform burns around us.
Remaining stuck in the old model ignores three critical developments in data and analysis. First, the type and quantity of information available in the open domain has expanded dramatically. Analysts can find the VKontakte feed of a Russian operative and the photos a tourist posted on Twitter of that operative visiting, for example, a Cathedral, though decidedly not on vacation. Companies such as Planet, Maxar, and HawkEye 360 have stretched capably into orbit—territory that once belonged only to wealthy nation-states—and provide comprehensive satellite coverage of the planet. All this data is created for commercial or personal use, but it has vast, largely untapped intelligence potential.
Second, unclassified, scalable cloud can house and provide the platform to process all this data in a way the IC has never been able to replicate on internal systems. In the past, the IC has attempted to drag unclassified data onto classified networks, which is slow, expensive, and limited. Continuous upgrades to cloud security and near-infinite scalability mean the IC can ingest data and process it with ease.
Listen to The Cipher Brief’s Open Source Report Podcast – a weekday open source collection of the stories impacting national security with your hosts Brad Christian and Suzanne Kelly. Subscribe wherever you listen to podcasts.
The third critical development is the crux of the revolution: AI/ML tools to process the data. Industry is already gleaning astonishing insights from a combination of publicly available information and AI/ML tools. Today, machines can read, see, write, and think in limited ways. Personal digital assistants such as Siri, Alexa, and Cortana can read and respond to emails, suggest grammar and substantive changes, and sort important emails from spam. Image-recognition algorithms can be trained to identify cats on YouTube or Chinese naval vessels “seen” by unmanned aerial vehicles over the South China Sea. The intelligence community could adapt these tools to comb through images, articles, and reports; build a structured database; and classify the data into categories created by analysts or by the machine itself. Further, unclassified cloud can host commercially-available tools, rather than force existing AI/ML applications to run the molasses-soaked obstacle course of security regulations for approval to work on the high side.
These three critical developments together build a future right out of science-fiction. In the next few years, every intelligence professional can have Iron Man’s JARVIS on their screen, saving hundreds of hours of work a year for nearly every IC employee. We should conceptualize this IC version of JARVIS as “OSCAR”: Open Source, Cloud-based, AI/ML-enabled Reporting.
Our report, “Move over, JARVIS, meet OSCAR,” includes actionable recommendations on how to get from here to a future where OSCAR is real. For example, the IC should embed its security professionals with cloud providers for a short tour to learn about unclassified cloud’s security features. The IC, in conjunction with Congress, should conduct a zero-based review for the software acquisition process. Part of that review should be shifting from statements of work to statements of objectives, which would focus on the needed outcome and allow more vendor innovation. A new policy should force explicit accounting for decisions to build in-house rather than buy software; agencies should have to prove their build is obviously better than commercial offerings based on performance, cost, efficiency, and security. Furthermore, agencies should involve analysts, operators, and support staff in acquisition decisions to ensure a close link between need and purchases. Finally, the IC should retrain acquisition officers on the blistering speed of buying cycles for emerging tech, emphasizing the need to find flexibility in software acquisition contracts.
The IC’s core mission is stealing secrets—that will never change. But now is the time for a decisive push to make OSINT an essential intelligence discipline. Open source data living on an unclassified cloud is how we will identify the next pandemic, predict the next outbreak of hostilities, or stop the next disinformation campaign. OSCAR is coming to help.
Listen to The Cipher Brief’s Open Source Report Podcast – a weekday open source collection of the stories impacting national security with your hosts Brad Christian and Suzanne Kelly. Subscribe wherever you listen to podcasts.
Read more expert-driven national security insights, perspective and analysis in The Cipher Brief
