Investigating people: a qualitative analysis of the search behaviours of open-source intelligence analysts

The Internet and the World Wide Web have become integral parts of the lives of many modern individuals, enabling almost instantaneous communication, sharing and broadcasting of thoughts, feelings and opinions. Much of this information is publicly facing, and as such, it can be utilised in a multitude of online investigations, ranging from employee vetting and credit checking to counter-terrorism and fraud prevention/detection. However, the search needs and behaviours of these investigators are not well documented in the literature. In order to address this gap, an in-depth qualitative study was carried out in cooperation with a leading investigation company. The research contribution is an initial identification of Open-Source Intelligence investigator search behaviours, the procedures and practices that they undertake, along with an overview of the difficulties and challenges that they encounter as part of their domain. This lays the foundation for future research in to the varied domain of Open-Source Intelligence gathering

Surveillance and falsification implications for open source intelligence investigations

© 2015 ACM. Legitimacy of surveillance is crucial to safeguarding validity of OSINT data as a tool for law-enforcement agencies

OSINT from a UK perspective: considerations from the law enforcement and military domains

Both law enforcement and the military have incorporated the use of open source intelligence (OSINT) into their daily operations. Whilst there are observable similarities in how these organisations employ OSINT there are also differences between military and policing approaches towards the understanding of open source information and the goals for the intelligence gathered from it. In particular, we focus on evaluating potential similarities and differences between understandings and approaches of operational OSINT between British law enforcement agencies and UK based MoD researchers and investigators. These observations are gathered towards the aim of increasing interoperability as well as creating opportunities for specific strengths and competencies of particular organisational approaches to be shared and utilised by both the military and law enforcement

Oportunidades, riesgos y aplicaciones de la inteligencia de fuentes abiertas en la ciberseguridad y la ciberdefensa

The intelligence gathering has transformed significantly in the digital age. A qualitative leap within this domain is the sophistication of Open Source Intelligence (OSINT), a paradigm that exploits publicly available information for planned and strategic objectives. The main purpose of this PhD thesis is to motivate, justify and demonstrate OSINT as a reference paradigm that should complement the present and future of both civilian cybersecurity solutions and cyberdefence national and international strategies. The first objective concerns the critical examination and evaluation of the state of OSINT under the current digital revolution and the growth of Big Data and Artificial Intelligence (AI). The second objective is geared toward categorizing security and privacy risks associated with OSINT. The third objective focuses on leveraging the OSINT advantages in practical use cases by designing and implementing OSINT techniques to counter online threats, particularly those from social networks. The fourth objective embarks on exploring the Dark web through the lens of OSINT, identifying and evaluating existing techniques for discovering Tor onion addresses, those that enable the access to Dark sites hosted in the Tor network, which could facilitate the monitoring of underground sites. To achieve these objectives, we follow a methodology with clearly ordered steps. Firstly, a rigorous review of the existing literature addresses the first objective, focusing on the state of OSINT, its applications, and its challenges. This serves to identify existing research gaps and establish a solid foundation for an updated view of OSINT. Consequently, a critical part of the methodology involves assessing the potential security and privacy risks that could emerge from the misuse of OSINT by cybercriminals, including using AI to enhance cyberattacks, fulfilling the second objective. Thirdly, to provide practical evidence regarding the power of OSINT, we work in a Twitter use case in the context of the 2019 Spanish general election, designing and implementing OSINT methods to understand the behaviour and impact of automated accounts. Through AI and social media analysis, this process aims to detect social bots in the wild for further behaviour characterization and impact assessment, thus covering the third objective. The last effort is dedicated to the Dark web, reviewing different works in the literature related to the Tor network to identify and characterize the techniques for gathering onion addresses essential for accessing anonymous websites, completing the fourth objective. This comprehensive methodology led to the publication of five remarkable scientific papers in peer-reviewed journals, collectively forming the basis of this PhD thesis. As main conclusions, this PhD thesis underlines the immense potential of OSINT as a strategic tool for problem-solving across many sectors. In the age of Big Data and AI, OSINT aids in deriving insights from vast, complex information sources such as social networks, online documents, web pages and even the corners of the Deep and Dark web. The practical use cases developed in this PhD thesis prove that incorporating OSINT into cybersecurity and cyberdefence is increasingly valuable. Social Media Intelligence (SOCMINT) helps to characterize social bots in disinformation contexts, which, in conjunction with AI, returns sophisticated results, such as the sentiment of organic content generated in social media or the political alignment of automated accounts. On the other hand, the Dark Web Intelligence (DARKINT) enables gathering the links of anonymous Dark web sites. However, we also expose in this PhD thesis that the development of OSINT carries its share of risks. Open data can be exploited for social engineering, spear-phishing, profiling, deception, blackmail, spreading disinformation or launching personalized attacks. Hence, the adoption of legal and ethical practices is also important.La recolección de inteligencia ha sufrido una transformación significativa durante la era digital. En particular, podemos destacar el auge y sofisticicación de la Inteligencia de Fuentes Abiertas (OSINT, por sus siglas en inglés de Open Source Intelligence), paradigma que recolecta y analiza la información públicamente disponible para objetivos estratégicos y planificados. El cometido principal de esta tesis doctoral es motivar, justificar y demostrar que OSINT es un paradigma de referencia para complementar el presente y futuro de las soluciones de ciberseguridad civiles y las estrategias de ciberdefensa nacionales e internacionales. El primer objetivo es examinar y evaluar el estado de OSINT en el contexto actual de revolución digital y crecimiento del Big Data y la Inteligencia Artificial (IA). El segundo objetivo está orientado a categorizar los riesgos de seguridad y privacidad asociados con OSINT. El tercer objetivo se centra en aprovechar las ventajas de OSINT en casos de uso prácticos, diseñando e implementando técnicas de OSINT para contrarrestar amenazas online, particularmente aquellas provenientes de las redes sociales. El cuarto objetivo es explorar la Dark web, buscando identificar y evaluar técnicas existentes para descubrir las direcciones aleatorias de las páginas alojadas en la red Tor. Para alcanzar estos objetivos seguimos una metodología con pasos ordenados. Primero, para abordar el primer objetivo, realizamos una revisión rigurosa de la literatura existente, centrándonos en el estado de OSINT, sus aplicaciones y sus desafíos. A continuación, en relación con el segundo objetivo, evaluamos los posibles riesgos de seguridad y privacidad que podrían surgir del mal uso de OSINT por parte de ciberdelincuentes, incluido el uso de IA para mejorar los ciberataques. En tercer lugar, para proporcionar evidencia práctica sobre el poder de OSINT, trabajamos en un caso de uso de Twitter en el contexto de las elecciones generales españolas de 2019, diseñando e implementando métodos de OSINT para entender el comportamiento y el impacto de las cuentas automatizadas. A través de la IA y el análisis de redes sociales, buscamos detectar bots sociales en Twitter para una posterior caracterización del comportamiento y evaluación del impacto, cubriendo así el tercer objetivo. Luego, dedicamos otra parte de la tesis al cuarto objetivo relacionado con la Dark web, revisando diferentes trabajos en la literatura de la red Tor para identificar y caracterizar las técnicas para recopilar direcciones onion, esenciales para acceder a sitios web anónimos de la red Tor. Esta metodología llevó a la publicación de cinco destacados artículos científicos en revistas revisadas por pares, formando colectivamente la base de esta tesis doctoral. Como principales conclusiones, esta tesis doctoral subraya el inmenso potencial de OSINT como herramienta estratégica para resolver problemas en muchos sectores. En la era de Big Data e IA, OSINT extrae conocimiento a partir de grandes y complejas fuentes de información en abierto como redes sociales, documentos online, páginas web, e incluso en la Deep y Dark web. Por otro lado, los casos prácticos desarrollados evidencian que la incorporación de OSINT en ciberseguridad y ciberdefensa es cada vez más valiosa. La Inteligencia de Redes Sociales (SOCMINT, por sus siglas en inglés Social Media Intelligence) ayuda a caracterizar bots sociales en contextos de desinformación. Por su parte, la Inteligencia de la Web Oscura (DARKINT, por sus siglas en inglés Dark Web Intelligence) permite recopilar enlaces de sitios anónimos de la Dark web. Sin embargo, esta tesis expone como el desarrollo de OSINT lleva consigo una serie de riesgos. Los datos abiertos pueden ser explotados para ingeniería social, spear-phishing, perfilado, engaño, chantaje, difusión de desinformación o lanzamiento de ataques personalizados. Por lo tanto, la adopción de prácticas legales y éticas es también imprescindible

Mapping Tools for Open Source Intelligence with Cyber Kill Chain for Adversarial Aware Security

publishedVersio

Ethical Hacking for a Good Cause: Finding Missing People using Crowdsourcing and Open-Source Intelligence (OSINT) Tools

Over 600,000 people go missing every year in the US alone. Despite the extensive resources allocated to investigating these cases, the high volume of missing person cases constitutes one of the biggest challenges for law enforcement agencies. One approach to tackle this challenge is using crowdsourcing. That is, volunteers use freely available tools and techniques to aid the existing efforts to investigate missing person cases. Open-Source Intelligence (OSINT) refers to gathering information from publicly available sources and analyzing it through a comprehensive set of open-source tools to produce meaningful and actionable intelligence. OSINT has been applied to address various societal challenges and crimes, including environmental abuse, human rights violations, child exploitation, domestic violence, disasters, and locating missing people. Building on this premise, this case examines a crowdsourced initiative called Trace Labs that aims to assist law enforcement agencies in solving missing person cases using OSINT tools. The case emphasizes socio-technical aspects of cybersecurity, highlighting both the bright and dark sides of technology. It demonstrates the potential of information systems to serve the public good by examining topics such as open-source software, crowdsourcing, and intelligence gathering, while acknowledging that the very same underlying technology can be used for malicious purposes

Social media intelligence: The national security–privacy nexus

Globally, changes in technology have always shaped the intelligence collection environment. South Africa is no exception. The emergence of satellite imagery had a significant influence on geographic intelligence (GEOINT) capabilities and, similarly, the emergence of the telegram and later the telephone had an equally significant effect on the signals intelligence (SIGINT) environment. With communications being revolutionised by mobile technology, such as recording, geo-positioning and photography, collection and distribution are ubiquitous. Smart mobile communication technology is also the driver of social media everywhere – at all ages, for state and non-state purposes, non-stop. More recently, social media intelligence (SOCMINT) became a key content domain for exploitation by the intelligence community. Examples of the successful exploitation of SOCMINT can be found internationally. It would be surprising if South Africa is not yet a statistic in terms of this phenomenon. Initially, many organisations viewed (and some still do) SOCMINT as an open-source intelligence (OSINT) tool. However, when considering the South African (SA) intelligence landscape, the concepts ‘democracy’, ‘transparency’ and ‘intelligence oversight’ are calibrating factors to bear in mind. It is also important to consider the influence of the national legislative framework governing the use of SOCMINT in South Africa. It then becomes clear that issues – such as the right to privacy – mean that SOCMINT is probably no longer covered by the scope of the OSINT definition and that intelligence organisations collecting social media content and producing SOCMINT should adhere to the legislative framework governing the collection and use of social media content and the production of SOCMINT. This article argues that SOCMINT and OSINT should be separate collection domains to protect the imperative of the right to privacy and national security requirements in a balanced manner by means of unambiguous national regulation in the interest of all citizens