Role of Automated Open Source Systems in the Intelligence Analysis of a High-Level Domain Expert

Abstract

This exploratory case study examines the interactions between a high-level domain expert engaged in political analysis and the output of a semi-automated commercial open-source analytic system. The study used an in-context think-aloud methodology to extract rich information on the analytical strategies used by the expert as he addressed a typical socio-political analytical task. The task involved the political stability of the uranium-mining region of Northern Niger and included access to relevant processed open source information. The focus of the research was on the cognitive processes of the analyst, including analytic strategies and information gathering behaviors. Three main strategic patterns were observed: (1) the population of a pre-existing cognitive model with relevant information; (2) the identification of relationships between model elements; and (3) the application of analytic judgment to these relationships. The findings in this study reveal misalignments between expert analytic processes and the current design of open source systems, but they also strongly suggest that, designed and used appropriately, automated open-source analytic systems have a valuable role to play in supporting the cognitive tasks of high-level analysts and in helping ensure that experts consider key lines of inquiry that directly impact analytic conclusions. The findings of this study have application for shaping the co-evolution of intelligence analysis and computerized open-source extraction systems.