Delayed and time-variant patrolling strategies against attackers with local observation capabilities

Surveillance of graph-represented environments is an application of autonomous patrolling robots that received remarkable attention during the last years. In this problem setting, computing a patrolling strategy is a central task to guarantee an effective protection level. Literature provides a vast set of methods where the patrolling strategies explicitly consider the presence of a rational adversary and fully informed attacker, which is characterized by worst-case (for the patroller) observation capabilities. In this work, we consider an attacker that does not have any prior knowledge on the environment and the patrolling strategy. Instead, we assume that the attacker can only access local observations on the vertex potentially under attack. We study the definition of patrolling strategies under the assumption that the attacker, when planning an attack on a particular location, tries to forecast the arrivals of the patroller on that particular location. We model our patrolling strategies with Markov chains where we seek the generation of arrivals that are difficult to forecast. To this end we introduce time-variance in the transition matrix used to determine the patrollers movements on the graph-represented environment

Aprendizagem de coordenação em sistemas multi-agente

The ability for an agent to coordinate with others within a system is a valuable property in multi-agent systems. Agents either cooperate as a team to accomplish a common goal, or adapt to opponents to complete different goals without being exploited. Research has shown that learning multi-agent coordination is significantly more complex than learning policies in singleagent environments, and requires a variety of techniques to deal with the properties of a system where agents learn concurrently. This thesis aims to determine how can machine learning be used to achieve coordination within a multi-agent system. It asks what techniques can be used to tackle the increased complexity of such systems and their credit assignment challenges, how to achieve coordination, and how to use communication to improve the behavior of a team. Many algorithms for competitive environments are tabular-based, preventing their use with high-dimension or continuous state-spaces, and may be biased against specific equilibrium strategies. This thesis proposes multiple deep learning extensions for competitive environments, allowing algorithms to reach equilibrium strategies in complex and partially-observable environments, relying only on local information. A tabular algorithm is also extended with a new update rule that eliminates its bias against deterministic strategies. Current state-of-the-art approaches for cooperative environments rely on deep learning to handle the environment’s complexity and benefit from a centralized learning phase. Solutions that incorporate communication between agents often prevent agents from being executed in a distributed manner. This thesis proposes a multi-agent algorithm where agents learn communication protocols to compensate for local partial-observability, and remain independently executed. A centralized learning phase can incorporate additional environment information to increase the robustness and speed with which a team converges to successful policies. The algorithm outperforms current state-of-the-art approaches in a wide variety of multi-agent environments. A permutation invariant network architecture is also proposed to increase the scalability of the algorithm to large team sizes. Further research is needed to identify how can the techniques proposed in this thesis, for cooperative and competitive environments, be used in unison for mixed environments, and whether they are adequate for general artificial intelligence.A capacidade de um agente se coordenar com outros num sistema é uma propriedade valiosa em sistemas multi-agente. Agentes cooperam como uma equipa para cumprir um objetivo comum, ou adaptam-se aos oponentes de forma a completar objetivos egoístas sem serem explorados. Investigação demonstra que aprender coordenação multi-agente é significativamente mais complexo que aprender estratégias em ambientes com um único agente, e requer uma variedade de técnicas para lidar com um ambiente onde agentes aprendem simultaneamente. Esta tese procura determinar como aprendizagem automática pode ser usada para encontrar coordenação em sistemas multi-agente. O documento questiona que técnicas podem ser usadas para enfrentar a superior complexidade destes sistemas e o seu desafio de atribuição de crédito, como aprender coordenação, e como usar comunicação para melhorar o comportamento duma equipa. Múltiplos algoritmos para ambientes competitivos são tabulares, o que impede o seu uso com espaços de estado de alta-dimensão ou contínuos, e podem ter tendências contra estratégias de equilíbrio específicas. Esta tese propõe múltiplas extensões de aprendizagem profunda para ambientes competitivos, permitindo a algoritmos atingir estratégias de equilíbrio em ambientes complexos e parcialmente-observáveis, com base em apenas informação local. Um algoritmo tabular é também extendido com um novo critério de atualização que elimina a sua tendência contra estratégias determinísticas. Atuais soluções de estado-da-arte para ambientes cooperativos têm base em aprendizagem profunda para lidar com a complexidade do ambiente, e beneficiam duma fase de aprendizagem centralizada. Soluções que incorporam comunicação entre agentes frequentemente impedem os próprios de ser executados de forma distribuída. Esta tese propõe um algoritmo multi-agente onde os agentes aprendem protocolos de comunicação para compensarem por observabilidade parcial local, e continuam a ser executados de forma distribuída. Uma fase de aprendizagem centralizada pode incorporar informação adicional sobre ambiente para aumentar a robustez e velocidade com que uma equipa converge para estratégias bem-sucedidas. O algoritmo ultrapassa abordagens estado-da-arte atuais numa grande variedade de ambientes multi-agente. Uma arquitetura de rede invariante a permutações é também proposta para aumentar a escalabilidade do algoritmo para grandes equipas. Mais pesquisa é necessária para identificar como as técnicas propostas nesta tese, para ambientes cooperativos e competitivos, podem ser usadas em conjunto para ambientes mistos, e averiguar se são adequadas a inteligência artificial geral.Apoio financeiro da FCT e do FSE no âmbito do III Quadro Comunitário de ApoioPrograma Doutoral em Informátic

Model-Predictive Strategy Generation for Multi-Agent Pursuit-Evasion Games

Multi-agent pursuit-evasion games can be used to model a variety of different real world problems including surveillance, search-and-rescue, and defense-related scenarios. However, many pursuit-evasion problems are computationally difficult, which can be problematic for domains with complex geometry or large numbers of agents. To compound matters further, practical applications often require planning methods to operate under high levels of uncertainty or meet strict running-time requirements. These challenges strongly suggest that heuristic methods are needed to address pursuit-evasion problems in the real world. In this dissertation I present heuristic planning techniques for three related problem domains: visibility-based pursuit-evasion, target following with differential motion constraints, and distributed asset guarding with unmanned sea-surface vehicles. For these domains, I demonstrate that heuristic techniques based on problem relaxation and model-predictive simulation can be used to efficiently perform low-level control action selection, motion goal selection, and high-level task allocation. In particular, I introduce a polynomial-time algorithm for control action selection in visibility-based pursuit-evasion games, where a team of pursuers must minimize uncertainty about the location of an evader. The algorithm uses problem relaxation to estimate future states of the game. I also show how to incorporate a probabilistic opponent model learned from interaction traces of prior games into the algorithm. I verify experimentally that by performing Monte Carlo sampling over the learned model to estimate the location of the evader, the algorithm performs better than existing planning approaches based on worst-case analysis. Next, I introduce an algorithm for motion goal selection in pursuit-evasion scenarios with unmanned boats. I show how a probabilistic model accounting for differential motion constraints can be used to project the future positions of the target boat. Motion goals for the pursuer boat can then be selected based on those projections. I verify experimentally that motion goals selected with this technique are better optimized for travel time and proximity to the target boat when compared to motion goals selected based on the current position of the target boat. Finally, I introduce a task-allocation technique for a team of unmanned sea-surface vehicles (USVs) responsible for guarding a high-valued asset. The team of USVs must intercept and block a set of hostile intruder boats before they reach the asset. The algorithm uses model-predictive simulation to estimate the value of high-level task assignments, which are then realized by a set of learned low-level behaviors. I show experimentally that using model-predictive simulations based on Monte-Carlo sampling is more effective than hand-coded evaluation heuristics

Enabling NATO’s Collective Defense: Critical Infrastructure Security and Resiliency (NATO COE-DAT Handbook 1)

In 2014 NATO’s Center of Excellence-Defence Against Terrorism (COE-DAT) launched the inaugural course on “Critical Infrastructure Protection Against Terrorist Attacks.” As this course garnered increased attendance and interest, the core lecturer team felt the need to update the course in critical infrastructure (CI) taking into account the shift from an emphasis on “protection” of CI assets to “security and resiliency.” What was lacking in the fields of academe, emergency management, and the industry practitioner community was a handbook that leveraged the collective subject matter expertise of the core lecturer team, a handbook that could serve to educate government leaders, state and private-sector owners and operators of critical infrastructure, academicians, and policymakers in NATO and partner countries. Enabling NATO’s Collective Defense: Critical Infrastructure Security and Resiliency is the culmination of such an effort, the first major collaborative research project under a Memorandum of Understanding between the US Army War College Strategic Studies Institute (SSI), and NATO COE-DAT. The research project began in October 2020 with a series of four workshops hosted by SSI. The draft chapters for the book were completed in late January 2022. Little did the research team envision the Russian invasion of Ukraine in February this year. The Russian occupation of the Zaporizhzhya nuclear power plant, successive missile attacks against Ukraine’s electric generation and distribution facilities, rail transport, and cyberattacks against almost every sector of the country’s critical infrastructure have been on world display. Russian use of its gas supplies as a means of economic warfare against Europe—designed to undermine NATO unity and support for Ukraine—is another timely example of why adversaries, nation-states, and terrorists alike target critical infrastructure. Hence, the need for public-private sector partnerships to secure that infrastructure and build the resiliency to sustain it when attacked. Ukraine also highlights the need for NATO allies to understand where vulnerabilities exist in host nation infrastructure that will undermine collective defense and give more urgency to redressing and mitigating those fissures.https://press.armywarcollege.edu/monographs/1951/thumbnail.jp

The British Way of War in North West Europe 1944-45: A Study of Two Infantry Divisions

This thesis will examine the British way of war as experienced by two British Infantry Divisions - the 43rd ‘Wessex’ and 53rd ‘Welsh’ - during the Overlord campaign in North West Europe in 1944 and 1945. The main locus of research centres on the fighting components of those divisions; the infantry battalions and their supporting regiments. In order to understand the way the British fought this part of the war, the thesis will consider the British Army’s history since 1918: its level of expertise at the end of the First World War; the impact of inter-war changes, and the experience of the early part of the Second World War, as these factors were fundamental in shaping how the British Army operated during the period covered in this study. These themes will be considered in the first chapter. The following seven chapters will study each of the two infantry divisions in turn, to maintain a chronological order. This is so that the experiences of each division can be examined in a logical way, from their initial experiences of combat in late June 1944 through to March 1945. Naturally, their major battles will be considered but so will their minor engagements and day-to-day experiences, as this will give a good, detailed, overview of each division’s campaign. This layout of chapters is also convenient for allowing comparisons between the two divisions as the campaign progressed. This thesis contains several strands of enquiry which will consider how Montgomery’s prosecution of the war actually translated to the smaller units of the division (the battalions, 4 companies, platoons and sections). The historiography for this campaign tends to suggest that the British Army fought the war in a cautious way, and that this approach was characterised by the use of overwhelming material superiority and rehearsed set piece attacks; tactics that were designed not only to destroy the enemy, but also to avoid the heavy casualties of the major battles of the First World War; a factor that was perceived to be vital to the maintenance of fragile infantry morale. Although the basic premise of a ‘cautious’ British way of war is generally accepted (along with its attendant emphasis on consolidation of objectives rather than exploitation of opportunities, and a reliance on adherence to lengthy orders), this study will conclude that the way the war was fought at sub-divisional levels was frequently at a pace that did not allow for such caution. Instead, it was characterised by command pressure to achieve results quickly, hasty planning and a reliance on massive artillery and mortar contributions to compensate for deficiencies in anti-tank and armoured support. This thesis will further conclude that a conscious policy of casualty conservation appears not to have been a priority at divisional command level, but was instead a consideration for company, platoon and section commanders and the men that they led

Cyber-Physical Threat Intelligence for Critical Infrastructures Security

Modern critical infrastructures can be considered as large scale Cyber Physical Systems (CPS). Therefore, when designing, implementing, and operating systems for Critical Infrastructure Protection (CIP), the boundaries between physical security and cybersecurity are blurred. Emerging systems for Critical Infrastructures Security and Protection must therefore consider integrated approaches that emphasize the interplay between cybersecurity and physical security techniques. Hence, there is a need for a new type of integrated security intelligence i.e., Cyber-Physical Threat Intelligence (CPTI). This book presents novel solutions for integrated Cyber-Physical Threat Intelligence for infrastructures in various sectors, such as Industrial Sites and Plants, Air Transport, Gas, Healthcare, and Finance. The solutions rely on novel methods and technologies, such as integrated modelling for cyber-physical systems, novel reliance indicators, and data driven approaches including BigData analytics and Artificial Intelligence (AI). Some of the presented approaches are sector agnostic i.e., applicable to different sectors with a fair customization effort. Nevertheless, the book presents also peculiar challenges of specific sectors and how they can be addressed. The presented solutions consider the European policy context for Security, Cyber security, and Critical Infrastructure protection, as laid out by the European Commission (EC) to support its Member States to protect and ensure the resilience of their critical infrastructures. Most of the co-authors and contributors are from European Research and Technology Organizations, as well as from European Critical Infrastructure Operators. Hence, the presented solutions respect the European approach to CIP, as reflected in the pillars of the European policy framework. The latter includes for example the Directive on security of network and information systems (NIS Directive), the Directive on protecting European Critical Infrastructures, the General Data Protection Regulation (GDPR), and the Cybersecurity Act Regulation. The sector specific solutions that are described in the book have been developed and validated in the scope of several European Commission (EC) co-funded projects on Critical Infrastructure Protection (CIP), which focus on the listed sectors. Overall, the book illustrates a rich set of systems, technologies, and applications that critical infrastructure operators could consult to shape their future strategies. It also provides a catalogue of CPTI case studies in different sectors, which could be useful for security consultants and practitioners as well

Counter Unmanned Aircraft Systems Technologies and Operations

As the quarter-century mark in the 21st Century nears, new aviation-related equipment has come to the forefront, both to help us and to haunt us. (Coutu, 2020) This is particularly the case with unmanned aerial vehicles (UAVs). These vehicles have grown in popularity and accessible to everyone. Of different shapes and sizes, they are widely available for purchase at relatively low prices. They have moved from the backyard recreation status to important tools for the military, intelligence agencies, and corporate organizations. New practical applications such as military equipment and weaponry are announced on a regular basis – globally. (Coutu, 2020) Every country seems to be announcing steps forward in this bludgeoning field. In our successful 2nd edition of Unmanned Aircraft Systems in the Cyber Domain: Protecting USA’s Advanced Air Assets (Nichols, et al., 2019), the authors addressed three factors influencing UAS phenomena. First, unmanned aircraft technology has seen an economic explosion in production, sales, testing, specialized designs, and friendly / hostile usages of deployed UAS / UAVs / Drones. There is a huge global growing market and entrepreneurs know it. Second, hostile use of UAS is on the forefront of DoD defense and offensive planners. They are especially concerned with SWARM behavior. Movies like “Angel has Fallen,” where drones in a SWARM use facial recognition technology to kill USSS agents protecting POTUS, have built the lore of UAS and brought the problem forefront to DHS. Third, UAS technology was exploding. UAS and Counter- UAS developments in navigation, weapons, surveillance, data transfer, fuel cells, stealth, weight distribution, tactics, GPS / GNSS elements, SCADA protections, privacy invasions, terrorist uses, specialized software, and security protocols has exploded. (Nichols, et al., 2019) Our team has followed / tracked joint ventures between military and corporate entities and specialized labs to build UAS countermeasures. As authors, we felt compelled to address at least the edge of some of the new C-UAS developments. It was clear that we would be lucky if we could cover a few of – the more interesting and priority technology updates – all in the UNCLASSIFIED and OPEN sphere. Counter Unmanned Aircraft Systems: Technologies and Operations is the companion textbook to our 2nd edition. The civilian market is interesting and entrepreneurial, but the military and intelligence markets are of concern because the US does NOT lead the pack in C-UAS technologies. China does. China continues to execute its UAS proliferation along the New Silk Road Sea / Land routes (NSRL). It has maintained a 7% growth in military spending each year to support its buildup. (Nichols, et al., 2019) [Chapter 21]. They continue to innovate and have recently improved a solution for UAS flight endurance issues with the development of advanced hydrogen fuel cell. (Nichols, et al., 2019) Reed and Trubetskoy presented a terrifying map of countries in the Middle East with armed drones and their manufacturing origin. Guess who? China. (A.B. Tabriski & Justin, 2018, December) Our C-UAS textbook has as its primary mission to educate and train resources who will enter the UAS / C-UAS field and trust it will act as a call to arms for military and DHS planners.https://newprairiepress.org/ebooks/1031/thumbnail.jp