British Air Vice-Marshal Sean Corbett CB MBE MA RAF retired from the Royal Air Force in September 2018 after a 30-year career as a professional intelligence officer. His last appointment in the military was spent in Washington DC as the first non-US Deputy Director of a major US Intelligence Agency. His primary role was to optimise intelligence sharing with US allies by developing and implementing a transformational change programme throughout the US intelligence community.
OPINION — The contemporary security environment is arguably the most complex and diverse ever faced by the free world. The ‘return’ of the nation state adversary in the form of the Russian Federation, with a renewed appetite for global power projection and seemingly a budget to match has required a recharacterization of the cold war dynamic with a hybrid warfare twist. The perennial rogue states of Iran and the Democratic People’s Republic of Korea that remain immune to international norms, diplomatic and financial pressure are inching towards true nuclear power status. Violent extremism has not gone away and will continue to re-emerge and evolve in ungoverned spaces wherever the conditions allow. China has finally shown its hand and seeks to become a global hegemon through its belt and road initiative, combining an overtly aggressive economic posture with a parallel and equally ambitious military-industrial programme. The list goes on.
The blurring of state and non-state actors and the consequent difficulties in proving accountability, and non-traditional sources of insecurity such as climate change, water security and global pandemics present additional challenges. All of this needs to be considered through the lens of ‘warfare in the information age’ in which information and disinformation are frontline tools, accessible to all, and in which information dominance, or even advantage, is by no means a given.
Scrutinising the dynamics and establishing ground-truth against each separate scenario is difficult enough, but cumulatively they are simply too big, complex, interdependent and fast-changing for any government to fully understand through traditional intelligence frameworks in which exquisite data is collected, analysed and disseminated at highly classified levels. The burden on the intelligence community is palpable and requires an innovative approach.
Intelligence analysis is an inherently labour-intensive discipline and collection capabilities are eye-wateringly expensive. The tyranny of the ‘now’ is particularly keenly felt within the community, as a huge proportion of the available capacity is necessarily focused on current issues and short-term threats. The time and resource available to devote to horizon-scanning, indicators and warnings and maintaining the foundational intelligence that provides the critical underpinning layer from which confident intelligence assessments are derived, are therefore at a premium.
The quality, depth and veracity of any intelligence assessment is dependent on the efficacy of the source material and the rigour applied to it. It requires all applicable data to be considered, in whatever form it takes, regardless of whether it is structured or labelled or how it is collected. This includes publicly available information (PAI).
The intelligence community has traditionally been reluctant to incorporate PAI into its assessments, preferring to rely on the highly classified information gathered from dedicated collection capabilities. There are multiple reasons for this, including the difficulties in integrating data at different classification levels, a lack of trust in the data, and, for agencies, the need to remain relevant and protect budgets. A primary issue is culture. There are many within the community who do not recognise PAI as valuable or a valid source of intelligence. Open source intelligence therefore remains under-utilised as a legitimate tool for the analyst.
While sensitive and focused source material will often provide a unique perspective, it is often at its most valuable when supported with other, less sensitive information. A pervading assumption that the more highly classified an intelligence product, the better it is, and more avidly read, remains prevalent. There should, however, be two primary considerations when classifying an intelligence product. The first is to protect the source (whether that is an individual or a discrete technical capability) and the second to protect the nature of the intelligence question or gap.
Open source intelligence, by definition, is derived from PAI, and there are now many readily available (and legal) applications and tools available to the open source analyst to collect, process and analyse the huge amount of PAI, particularly that accessible through the internet. The proliferation of high-resolution Commercial Satellite Imagery in numerous formats, communications analytics, and academic forums can all provide data of high enough quality to service many intelligence questions. But just as classified intelligence does not necessarily guarantee ground-truth, PAI also needs to be validated. Disinformation (deliberate or otherwise) is a fact of life and ever more prevalent in the open source domain.
Rigorous tradecraft and a validation process are therefore critical in ensuring that the data is trusted in the same way as applied to classified data. The vast quantities of accessible data in multiple formats are both a blessing and a curse. The more information available, the more likely a full picture can be pieced together and validated, but the challenges of wrangling, curation, management and manipulation of this data is compounded by its many different formats (both structured and unstructured). This is as true of the classified world as that of the public domain, but not to the same extent. Data analytics have a significant role to play here, with the ability to determine correlations, anomalies and trends. It will not necessarily identify the good information from the bad, but it can highlight the unexpected.
There is rarely a discussion about data management manipulation without a mention of Artificial Intelligence (AI). Applied properly, there is no doubt that AI can significantly ease the burden of the analyst, and while the intelligence community has begun to embrace this technology, it could be argued that the commercial sector is better positioned to embrace and apply these techniques at scale and in an innovative way. The application of AI naturally leads to question of trust in the outputs from the process, particularly without a clear understanding of the underlying processes. The intelligence community is obligated to provide confidence levels in their assessments, which requires a defined and rigorous methodology.
Ironically, it is often the low confidence assessments that prove to be most valuable to the decision-maker, since they typically focus on the hardest intelligence questions with the least evidence and which therefore require more judgement. Rather than becoming fixated on the methodology in the creation of ‘safe’ analyses, perhaps there is a need to become more comfortable with some ambiguity. Just as an analyst relies on track-record and credibility, the same is true of the algorithms and trust will be built over time.
The security classification conundrum must also be addressed. There is a popular belief within the community that the aggregation of open source information makes the output classified. While it is relatively straightforward to appropriately classify single source reporting, the same cannot be said for amalgamated information, particularly at the unclassified level. Two tests can be applied to address this. Would the adversary change their activity or behaviour if they knew they were the subject of an intelligence question, and do they have enough control of the data source to manipulate or change it and therefore conduct deception operations?
If the answer to both is no, then then the reporting can probably remain unclassified Protecting the nature of the intelligence gap is more challenging. For very good reason, intelligence agencies do not want a potential adversary to know their intelligence gaps, either from a technical capability or a reporting perspective. But the real value of open source intelligence is in providing uncontentious, foundational, baseline intelligence that is so important in underpinning more focused problem sets, or in determining timely ground truth for a high-profile specific event which may be the subject of media reporting. And of course, PAI can be used within a classified environment to provide additional value to classified reporting.
There are broader benefits of producing open-source intelligence. An obvious, but rarely articulated advantage is that it can be widely distributed among decision-makers who either do not have the necessary clearances or the means by which they can access classified information. PAI can also be used to facilitate intelligence sharing with partners and allies, which is an increasingly important requirement in today’s diverse and connected global security environment in which coalitions are the norm.
At the operational and tactical level there is an imperative to ensure that partner forces have the intelligence they need to conduct effective operations and adopt an effective force protective posture. At the strategic level, ensuring that all allies have a common understanding of the threats and security challenges collectively faced facilitates effective, consensus-based decision-making. Open-source intelligence that has previously been correlated with that to sensitive to share can mitigate an uneven level of understanding and reassure partners.
It would be naive to suggest that open source intelligence will replace exquisite, classified reporting within the intelligence community. But there is little doubt that, used correctly, and with the necessary checks and balances, it could be utilised to a much greater extent. As the analyst becomes more comfortable with PAI as a legitimate source of information, the weight of effort could shift from PAI augmenting classified information on an ad hoc basis, to it becoming an underpinning layer to which classified intelligence can be applied to validate, enhance and provides extra assurance. It would also free resource to enable more focused and specialised intelligence collection assets to address the harder target-sets and, particularly when combined with commercially available and powerful analytics tools fill the gaps.
There are organisations in which the value of OSINT has been recognized an applied to great effect, but there remains a long way to go.
Read more expert-driven national security insights, perspective and analysis in The Cipher Brief
