Automatic Pipeline Surveillance Air-Vehicle

This thesis presents the developments of a vision-based system for aerial pipeline Right-of-Way surveillance using optical/Infrared sensors mounted on Unmanned Aerial Vehicles (UAV). The aim of research is to develop a highly automated, on-board system for detecting and following the pipelines; while simultaneously detecting any third-party interference. The proposed approach of using a UAV platform could potentially reduce the cost of monitoring and surveying pipelines when compared to manned aircraft. The main contributions of this thesis are the development of the image-analysis algorithms, the overall system architecture and validation of in hardware based on scaled down Test environment. To evaluate the performance of the system, the algorithms were coded using Python programming language. A small-scale test-rig of the pipeline structure, as well as expected third-party interference, was setup to simulate the operational environment and capture/record data for the algorithm testing and validation. The pipeline endpoints are identified by transforming the 16-bits depth data of the explored environment into 3D point clouds world coordinates. Then, using the Random Sample Consensus (RANSAC) approach, the foreground and background are separated based on the transformed 3D point cloud to extract the plane that corresponds to the ground. Simultaneously, the boundaries of the explored environment are detected based on the 16-bit depth data using a canny detector. Following that, these boundaries were filtered out, after being transformed into a 3D point cloud, based on the real height of the pipeline for fast and accurate measurements using a Euclidean distance of each boundary point, relative to the plane of the ground extracted previously. The filtered boundaries were used to detect the straight lines of the object boundary (Hough lines), once transformed into 16-bit depth data, using a Hough transform method. The pipeline is verified by estimating a centre line segment, using a 3D point cloud of each pair of the Hough line segments, (transformed into 3D). Then, the corresponding linearity of the pipeline points cloud is filtered within the width of the pipeline using Euclidean distance in the foreground point cloud. Then, the segment length of the detected centre line is enhanced to match the exact pipeline segment by extending it along the filtered point cloud of the pipeline. The third-party interference is detected based on four parameters, namely: foreground depth data; pipeline depth data; pipeline endpoints location in the 3D point cloud; and Right-of-Way distance. The techniques include detection, classification, and localization algorithms. Finally, a waypoints-based navigation system was implemented for the air- vehicle to fly over the course waypoints that were generated online by a heading angle demand to follow the pipeline structure in real-time based on the online identification of the pipeline endpoints relative to a camera frame

Survey of the Status of Small Armed and Unarmed Uninhabited Aircraft

The project ‘Preventive Arms Control for Small and Very Small Armed Aircraft and Missiles’ investigates the properties of ever smaller aircraft and missiles. This project report no. 1 covers the status of aircraft worldwide, including relevant unarmed vehicles but excluding hobby aircraft. Small and very small aircraft are defined by size: below 2 m and below 0.2 m, respectively. After an elementary introduction into aerodynamics a technical overview is given, looking at airframe configurations, materials and manufacturing, power and propulsion, guidance, launch and recovery, and payloads. Future possibilities and trends are illustrated by presenting military research and development of the technological leader, the USA. Short chapters deal with swarms and with countermeasures. The worldwide survey has resulted in a database that contains 129 types from 27 countries. The publicly available properties are given in 26 categories. Statistical evaluations cover several key parameters

Technological Perspectives of Countering UAV Swarms

Conventional AD systems have been found less effective for countering UAVs and loitering munitions. Thishas necessitated the development of counter-UAV systems with different functionalities. A cluster of armed UAVsas swarm formations has further rendered the conventional AD systems far from effective, emphasizing the need to consider countering swarms as the most crucial element in new-generation aerial threat mitigation strategies. In this paper, the capabilities of UAV swarms and vital military assets exposed to such attacks are identified. To protect the vital assets from aerial swarm threats, ideal system characteristics of a counter-UAV (C-UAV) swarm system to overcome the challenges are discussed. Currently available acquisition & engagement technology is analyzed and the application of these systems to counter swarm applications is brought out. New requirements are discussed and a conceptual design of a layered system is derived which can handle a large spectrum of aerial threats including a swarm of UAVs. This system is expected to have a higher rate of engagement and can be designed with low-cost network-integrated systems

ANALYSIS AND ASSESSMENT OF LETHALITY AND SURVIVABILITY FOR THE MARINE LITTORAL REGIMENT

As the Marine Corps activates the Marine Littoral Regiment (MLR) to serve as the joint force’s reconnaissance and counter-reconnaissance effort, questions abound regarding the MLR’s ability to provide a persistent and lethal presence well inside the reach of our adversaries’ advanced long-range precision fires. In this study, the author uses agent-based combat simulations to inform future force design decisions, live-force experimentation, and tactics. The simulated scenario imagines a future MLR conducting sea control operations in the littorals of the Western Pacific against a peer naval threat. This research investigates the effect that a guard force of autonomous and/or semi-autonomous surface vessels, operating as the guard force of the MLR’s defense in depth, has on the survivability and lethality of the MLR’s land-based anti-ship missile platforms. Summary statistics generated by the simulation indicate that the future battlefield will see high losses on both sides. However, based on the results of 27,200 simulated engagements, this study finds that an MLR using a guard force of armed and unarmed “scouts” as described above can inflict a prohibitively high and unsustainable cost on an enemy naval force.Outstanding ThesisMajor, United States Marine CorpsApproved for public release. Distribution is unlimited

Loitering Munitions and Unpredictability: Autonomy in Weapon Systems and Challenges to Human Control

This report, published by the Center for War Studies, University of Southern Denmark and the Royal Holloway Centre for International Security, highlights the immediate need to regulate autonomous weapon systems, or ‘killer robots’ as they are colloquially called. Written by Dr. Ingvild Bode and Dr. Tom F.A. Watts, authors of an earlier study of air defence systems published with Drone Wars UK, the “Loitering Munitions and Unpredictability” report examines whether the use of automated, autonomous, and AI technologies as part of the global development, testing, and fielding of loitering munitions since the 1980s has impacted emerging practices and social norms of human control over the use of force. It is commonly assumed that the challenges generated by the weaponization of autonomy will materialise in the near to medium term future. The report’s central argument is that whilst most existing loitering munitions are operated by a human who authorizes strikes against system-designated targets, the integration of automated and autonomous technologies into these weapons has created worrying precedents deserving of greater public scrutiny. Loitering munitions – or ‘killer drones’ as they are often popularly known – are expendable uncrewed aircraft which can integrate sensor-based analysis to hover over, detect and explode into targets. These weapons are very important technologies within the international regulatory debates on autonomous weapon systems – a set of technologies defined by Article 36 as weapons “where force is applied automatically on the basis of a sensor-based targeting system”. The earliest loitering munitions such as the Israel Aerospace Industries Harpy are widely considered as being examples of weapons capable of automatically applying force via sensor-based targeting without human intervention. A May 2021 report authored by a UN Panel of Experts on Libya suggests that Kargu-2 loitering munitions manufactured by the Turkish defence company STM may have been “programmed to attack targets without requiring data connectivity between the operator and the munition”. According to research published by Daniel Gettinger, the number of states producing these weapons more than doubled from fewer than 10 in 2017 to almost 24 by mid-2022. The sizeable role which loitering munitions have played in the ongoing fighting between Russia and the Ukraine further underscores the timeliness of this report, having raised debates on whether so called “killer robots are the future of war?” Most manufacturers of these weapons characterize loitering munitions as “human in the loop” systems. The operators of these systems are required to authorize strikes against system-designated targets. The findings of this report, however, suggest that the global trend toward increasing autonomy in targeting has already affected the quality and form of control over the use of force that humans can exercise over specific targeting decisions. Loitering munitions can use automated, autonomous, and to a limited extent, AI technologies to identify, track, and select targets. Some manufacturers also allude to the potential capacity of the systems to attack targets without human intervention. This suggests that human operators of loitering munitions may not always retain an ability to visually verify targets before attack. This report highlights three principal areas of concern: Greater uncertainties regarding how human agents exert control over specific targeting decisions. The use of loitering munitions as anti-personnel weapons and in populated areas. Potential indiscriminate and wide area effects associated with the fielding of loitering munitions. This report’s analysis is drawn from two sources of data: first, a new qualitative data catalogue which compiles the available open-source information about the technical details, development history, and use of autonomy and automation in a global sample of 24 loitering munitions; and second, an in-depth study of how such systems have been used in three recent conflicts – the Libyan Civil War (2014-2020), the 2020 Nagorno-Karabakh War, and the War in Ukraine (2022-). Based on its findings, the authors urge the various stakeholder groups participating in the debates at the United Nations Convention on Certain Conventional Weapons Group of Governmental Experts and elsewhere to develop and adopt legally binding international rules on autonomy in weapon systems, including loitering munitions as a category therein. It is recommended that states: Affirm, retain, and strengthen the current standard of real-time, direct human assessment of, and control over, specific targeting decisions when using loitering munitions and other weapons integrating automated, autonomous, and AI technologies as a firewall for ensuring compliance with legal and ethical norms. Establish controls over the duration and geographical area within which weapons like loitering munitions that can use automated, autonomous, and AI technologies to identify, select, track, and apply force can operate. Prohibit the integration of machine learning and other forms of unpredictable AI algorithms into the targeting functions of loitering munitions because of how this may fundamentally alter the predictability, explainability, and accountability of specific targeting decisions and their outcomes. Establish controls over the types of environments in which sensor-based weapons like loitering munitions that can use automated, autonomous, and AI technologies to identify, select, track, and apply force to targets can operate. Loitering munitions functioning as AWS should not be used in populated areas. Prohibit the use of certain target profiles for sensor-based weapons which use automated, autonomous, and AI technologies in targeting functions. This should include prohibiting the design, testing, and use of autonomy in weapon systems, including loitering munitions, to “target human beings” as well as limiting the use of such weapons “to objects that are military objectives by nature” (ICRC, 2021: 2.). Be more forthcoming in releasing technical details relating to the quality of human control exercised in operating loitering munitions in specific targeting decisions. This should include the sharing, where appropriate, of details regarding the level and character of the training that human operators of loitering munitions receive.  Funding: Research for the report was supported by funding from the European Union’s Horizon 2020 research and innovation programme (under grant agreement No. 852123, AutoNorms project) and from the Joseph Rowntree Charitable Trust. Tom Watts’ revisions to this report were supported by the funding provided by his Leverhulme Trust Early Career Research Fellowship (ECF-2022-135). We also collaborated with Article 36 in writing the report. About the authors: Dr Ingvild Bode is Associate Professor at the Center for War Studies, University of Southern Denmark and a Senior Research Fellow at the Conflict Analysis Research Centre, University of Kent. She is the Principal Investigator of the European Research Council-funded “AutoNorms” project, examining how autonomous weapons systems may change international use of force norms. Her research focuses on understanding processes of normative change, especially through studying practices in relation to the use of force, military Artificial Intelligence, and associated governance demands. More information about Ingvild’s her research is available here. Dr Tom F.A. Watts is a Leverhulme Trust Early Career Researcher based at the Department of Politics, International Relations, and Philosophy at Royal Holloway, University of London. His current project titled “Great Power Competition and Remote Warfare: Change or Continuity in Practice?” (ECF-2022-135) examines the relationship between the use of the strategic practices associated with the concept of remote warfare, the dynamics of change and continuity in contemporary American foreign policy, and autonomy in weapons systems. More information about Tom’s research is available here

Modeling Expeditionary Advanced Base Operations in the Combined Arms Analysis Tool for the 21st Century (COMBATXXI)

The United States Marine Corps (USMC) is undergoing organizational and operational changes to adapt to new warfighting requirements in today’s world. The USMC Force Design 2030 describes new concepts, such as Expeditionary Advanced Base Operations (EABO), with a focus on reconnaissance/counter-reconnaissance and maritime interdiction. To examine and evaluate new concepts of operation, force structures, weapon systems, tactics, techniques, and procedures, as well as other adaptations for such operations, the USMC requires models and simulations that can represent the full range of variations related to these expected changes. The Combined Arms Analysis Tool for the 21st Century (COMBATXXI) is a combat simulation jointly developed by the USMC and the US Army to support modeling and analysis. Developed over the past 20 years, COMBATXXI possesses many of the fundamental capabilities needed to study these new concepts but currently lacks realistic representation in some key areas, such as maritime surface combatants needed for examining critical aspects of the new role of maritime interdiction. Such representation requires platform identification, targeting, and assessment of damage that can lead to determination of their continued ability to perform operational missions. The purpose of this study is to examine new warfighting concepts related to EABO and to identify relevant modeling approaches using the COMBATXXI simulation. The study describes a modeling approach, initial implementation of that approach in COMBATXXI, and preliminary evaluation of the utility of the model for supporting scenarios and studies relevant to the new USMC concepts of operation. The study concludes with recommendations for follow-on work to further improve or employ the developed capability.Marine Corps Combat Development Command Operations Analysis Directorate Capabilities Development and Integration.Approved for public release; distribution is unlimited

Convergence of Intelligent Data Acquisition and Advanced Computing Systems

This book is a collection of published articles from the Sensors Special Issue on "Convergence of Intelligent Data Acquisition and Advanced Computing Systems". It includes extended versions of the conference contributions from the 10th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS’2019), Metz, France, as well as external contributions

Mitigation of Human Supervisory Control Wait Times through Automation Strategies

The application of network centric operations principles to human supervisory control (HSC) domains means that humans are increasingly being asked to manage multiple simultaneous HSC processes. However, increases in the number of available information sources, volume of information and operational tempo, all which place higher cognitive demands on operators, could become constraints limiting the success of network centric processes. In time-pressured scenarios typical of networked command and control scenarios, efficiently allocating attention between a set of dynamic tasks is crucial for mission success. Inefficient attention allocation leads to system wait times, which could eventually lead to critical events such as missed times on targets and degraded overall mission success. One potential solution to mitigating wait times is the introduction of automated decision support in order to relieve operator workload. However, it is not obvious what automated decision support is appropriate, as higher levels of automation may result in a situation awareness decrement and other problems typically associated with excessive automation such as automation bias. To assess the impact of increasing levels of automation on human and system performance in a time-critical HSC multiple task management context, an experiment was run in which an operator simultaneously managed four highly autonomous unmanned aerial vehicles (UAVs) executing an air tasking order, with the overall goal of destroying a pre-determined set of targets within a limited time period. Four increasing levels automated decision support were investigated as well as high and low operational replanning tempos. The highest level of automation, management-byexception, had the best performance across several metrics but had a greater number of catastrophic events during which a UAV erroneously destroyed a friendly target. Contrary to expectations, the collaborative level of decision support, which provided predictions for possible periods of task overload as well as possible courses of action to relieve the high workload, produced the worst performance. This is attributable to an unintended consequence of the automation where the graphical visualization of the computer’s predictions caused users to try to globally optimize the schedules for all UAVs instead of locally optimizing schedules in the immediate future, resulting in them being overwhelmed. Total system wait time across both experimental factors was dominated by wait time caused by lack of situation awareness, which is difficult to eliminate, implying that there will be a clear upper limit on the number of vehicles that any one person can supervise because of the need to stay cognitively aware of unfolding events.Prepared for Boeing, Phantom Work

Utilizing Ground-Based LIDAR Measurements to Aid Autonomous Airdrop Systems

Uncertainty in atmospheric winds represents one of the primary sources of landing error in airdrop systems. In this work, a ground-based LIDAR system samples the wind field at discrete points above the target and transmits real-time data to approaching autonomous airdrop systems. In simulation and experimentation, the inclusion of a light detection and ranging (LIDAR) system showed a maximum of 40% improvement over unaided autonomous airdrop systems. Wind information nearest ground level has the largest impact on improving accuracy