The Effect of Task Load, Automation Reliability, and Environment Complexity on UAV Supervisory Control Performance

Over the last decade, military unmanned aerial vehicles (UAVs) have experienced exponential growth and now comprise over 40% of military aircraft. However, since most military UAVs require multiple operators (usually an air vehicle operator, payload operator, and mission commander), the proliferation of UAVs has created a manpower burden within the U.S. military. Fortunately, simultaneous advances in UAV automation have enabled a switch from direct control to supervisory control; future UAV operators will no longer directly control a single UAV subsystem but, rather, will control multiple advanced, highly autonomous UAVs. However, research is needed to better understand operator performance in a complex UAV supervisory control environment. The Naval Research Lab (NRL) developed SCOUT™ (Supervisory Control Operations User Testbed) to realistically simulate the supervisory control tasks that a future UAV operator will likely perform in a dynamic, uncertain setting under highly variable time constraints. The study reported herein used SCOUT to assess the effects of task load, environment complexity, and automation reliability on UAV operator performance and automation dependence. The effects of automation reliability on participants’ subjective trust ratings and the possible dissociation between task load and subjective workload ratings were also explored. Eighty-one Navy student pilots completed a 34:15 minute pre-scripted SCOUT scenario, during which they managed three helicopter UAVs. To meet mission goals, they decided how to best allocate the UAVs to locate targets while they maintained communications, updated UAV parameters, and monitored their sensor feeds and airspace. After completing training on SCOUT, participants were randomly sorted into low and high automation reliability groups. Within each group, task load (the number of messages and vehicle status updates that had to be made and the number of new targets that appeared) and environment complexity (the complexity of the payload monitoring task) were varied between low and high levels over the course of the scenario. Participants’ throughput, accuracy, and expected value in response to mission events were used to assess their performance. In addition, participants rated their subjective workload and fatigue using the Crew Status Survey. Finally, a four-item survey modeled after Lee and Moray’s validated (1994) scale was used to assess participants’ trust in the payload task automation and their self-confidence that they could have manually performed the payload task. This study contributed to the growing body of knowledge on operator performance within a UAV supervisory control setting. More specifically, it provided experimental evidence of the relationship between operator task load, task complexity, and automation reliability and their effects on operator performance, automation dependence, and operators’ subjective experiences of workload and fatigue. It also explored the relationship between automation reliability and operators’ subjective trust in said automation. The immediate goal of this research effort is to contribute to the development of a suite of domain-specific performance metrics to enable the development and/or testing and evaluation of future UAV ground control stations (GCS), particularly new work support tools and data visualizations. Long-term goals also include the potential augmentation of the current Aviation Selection Test Battery (ASTB) to better select future UAV operators and operational use of the metrics to determine mission-specific manpower requirements. In the far future, UAV-specific performance metrics could also contribute to the development of a dynamic task allocation algorithm for distributing control of UAVs amongst a group of operators

MISSION ENGINEERING FOR HYBRID FORCE 2025

This report focuses on the mission engineering process for a hybrid force in 2025. Updated tasking from OPNAV N9I emphasized the necessity of focusing on the benefits of using cost-conservative unmanned systems. Specifically, the focus was placed on the near-peer competitor China and the problems that could be expected in an anti-access/area denial (A2/AD) situation in the South China Sea. The Naval Surface Warfare Center mission engineering approach was used to identify specific vignettes for proposed alternative fleet architectures and then analyzed using combat simulation and optimization models. Research on performance characteristics and cost were compiled on current unmanned systems, specifically those in development at a high technology readiness level. Proposed unmanned systems architectures were developed as solutions to the A2/AD problem and proposed vignettes. The unmanned systems architectures were then run through an optimization model to maximize system performance while minimizing cost. The results of the architecture optimization were then input into modeling and simulation. The overall effectiveness of each architecture in each vignette were then compared to find the most effective solution. An analysis of the results was performed to show the expected mission effectiveness and proposed cost of utilizing the proposed solution unmanned architectures. The most effective architectures included search, counter swarm, delivery, and attack systems.Lieutenant, United States NavyLieutenant, United States NavyLieutenant, United States NavyMajor, Republic of Singapore NavyMajor, Singapore ArmyLieutenant, United States NavyLieutenant, United States NavyLieutenant, United States NavyCommander, United States NavyApproved for public release. Distribution is unlimited

Military Innovation in the Third Age of U.S. Unmanned Aviation, 1991–2015

Military innovation studies have largely relied on monocausal accounts—rationalism, institutionalism, or culture—to explain technologically innovative and adaptive outcomes in defense organizations. None of these perspectives alone provided a compelling explanation for the adoption outcomes of unmanned aerial vehicles (UAVs) in the U.S. military from 1991 to 2015. Two questions motivated this research: Why, despite abundant material resources, mature technology, and operational need, are the most-capable UAVs not in the inventory across the services? What accounts for variations and patterns in UAV innovation adoption? The study selected ten UAV program episodes from the Air Force and Navy, categorized as high-, medium-, and low-end cases, for within-case and cross-case analysis. Primary and secondary sources, plus interviews, enabled process tracing across episodes. The results showed a pattern of adoption or rejection based on a logic-of-utility effectiveness and consistent resource availability: a military problem to solve, and a capability gap in threats or tasks and consistent monetary capacity; furthermore, ideational factors strengthened or weakened adoption. In conclusion, the study undermines single-perspective arguments as sole determinants of innovation, reveals that military culture is not monolithic in determining outcomes, and demonstrates that civil-military relationships no longer operate where civilian leaders hold inordinate sway over military institutions.Lieutenant Colonel, United States Air ForceApproved for public release; distribution is unlimited

Autonomy of Military Robots: Assessing the Technical and Legal (“Jus In Bello”) Thresholds, 32 J. Marshall J. Info. Tech. & Privacy L. 57 (2016)

While robots are still absent from our homes, they have started to spread over battlefields. However, the military robots of today are mostly remotely controlled platforms, with no real autonomy. This paper will disclose the obstacles in implementing autonomy for such systems by answering a technical question: What level of autonomy is needed in military robots and how and when might it be achieved, followed by a techno-legal one: How to implement the rules of humanitarian law within autonomous fighting robots, in order to allow their legal deployment? The first chapter scrutinizes the significance of autonomy in robots and the metrics used to quantify it, which were developed by the US Department of Defense. The second chapter focuses on the autonomy of state-of-the-art” robots (e.g.; Google’s self-driving car, DARPA’s projects, etc.) for navigation, ISR or lethal missions. Based on public information, we will get a hint of the architectures, the functioning, the thresholds and technical limitations of such systems. The bottleneck to a higher autonomy of robots seems to be their poor “perceptive intelligence.” The last chapter looks to the requirements of humanitarian law (rules of “jus in bello”/rules of engagement) to the legal deployment of autonomous lethal robots on the battlefields. The legal and moral reasoning of human soldiers, complying with humanitarian law, is a complex cognitive process which must be emulated by autonomous robots that could make lethal decisions. However, autonomous completion of such “moral” tasks by artificial agents is much more challenging than the autonomous implementation of other tasks, such as navigation, ISR or kinetic attacks. Given the limits of current Artificial Intelligence, it is highly unlikely that robots will acquire such moral capabilities anytime soon. Therefore, for the time being, the autonomous weapon systems might be legally deployed, but only in very particular circumstances, where the requirements of humanitarian law happen to be irrelevant

Crossbow Volume 1

Student Integrated ProjectIncludes supplementary materialDistributing naval combat power into many small ships and unmanned air vehicles that capitalize on emerging technology offers a transformational way to think about naval combat in the littorals in the 2020 time frame. Project CROSSBOW is an engineered systems of systems that proposes to use such distributed forces to provide forward presence to gain and maiantain access, to provide sea control, and to project combat power in the littoral regions of the world. Project CROSSBOW is the result of a yearlong, campus-wide, integrated research systems engineering effort involving 40 student researchers and 15 supervising faculty members. This report (Volume I) summarizes the CROSSBOW project. It catalogs the major features of each of the components, and includes by reference a separate volume for each of the major systems (ships, aircraft, and logistics). It also prresents the results of the mission and campaign analysis that informed the trade-offs between these components. It describes certain functions of CROSSBOW in detail through specialized supporting studies. The student work presented here is technologically feasible, integrated and imaginative. The student project cannot by itself provide definitive designs or analyses covering such a broad topic. It does strongly suggest that the underlying concepts have merit and deserve further serious study by the Navy as it transforms itself

Surveillance Planning against Smart Insurgents in Complex Terrain

This study is concerned with finding a way to solve a surveillance system allocation problem based on the need to consider intelligent insurgency that takes place in a complex geographical environment. Although this effort can be generalized to other situations, it is particularly geared towards protecting military outposts in foreign lands. The technological assets that are assumed available include stare-devices, such as tower-cameras and aerostats, as well as manned and unmanned aerial systems. Since acquiring these assets depends on the ability to control and monitor them on the target terrain, their operations on the geo-location of interest ought to be evaluated. Such an assessment has to also consider the risks associated with the environmental advantages that are accessible to a smart adversary. Failure to consider these aspects might render the forces vulnerable to surprise attacks. The problem of this study is formulated as follows: given a complex terrain and a smart adversary, what types of surveillance systems, and how many entities of each kind, does a military outpost need to adequately monitor its surrounding environment? To answer this question, an analytical framework is developed and structured as a series of problems that are solved in a comprehensive and realistic fashion. This includes digitizing the terrain into a grid of cell objects, identifying high-risk spots, generating flight tours, and assigning the appropriate surveillance system to the right route or area. Optimization tools are employed to empower the framework in enforcing constraints--such as fuel/battery endurance, flying assets at adequate altitudes, and respecting the climbing/diving rate limits of the aerial vehicles--and optimizing certain mission objectives--e.g. revisiting critical regions in a timely manner, minimizing manning requirements, and maximizing sensor-captured image quality. The framework is embedded in a software application that supports a friendly user interface, which includes the visualization of maps, tours, and related statistics. The final product is expected to support designing surveillance plans for remote military outposts and making critical decisions in a more reliable manner

Design flexibility in complex engineering systems under multiple uncertainties

Master'sMASTER OF ENGINEERIN

Autonomous Weapon Systems: A Brief Survey of Developmental, Operational, Legal, and Ethical Issues

What does the Department of Defense hope to gain from the use of autonomous weapon systems (AWS)? This Letort Paper explores a diverse set of complex issues related to the developmental, operational, legal, and ethical aspects of AWS. It explores the recent history of the development and integration of autonomous and semi-autonomous systems into traditional military operations. It examines anticipated expansion of these roles in the near future as well as outlines international efforts to provide a context for the use of the systems by the United States. As these topics are well-documented in many sources, this Paper serves as a primer for current and future AWS operations to provide senior policymakers, decisionmakers, military leaders, and their respective staffs an overall appreciation of existing capabilities and the challenges, opportunities, and risks associated with the use of AWS across the range of military operations. Emphasis is added to missions and systems that include the use of deadly force.https://press.armywarcollege.edu/monographs/1303/thumbnail.jp

Asynchronous, distributed optimization for the coordinated planning of air and space assets

Thesis (S.M.)--Massachusetts Institute of Technology, Sloan School of Management, Operations Research Center, 2010.Cataloged from PDF version of thesis.Includes bibliographical references (p. 189-194).Recent decades have seen the development of more advanced sensor and communication systems, with the future certainly holding more innovation in these areas. However, current operations involve "stovepipe" systems in which inefficiencies are inherent. In this thesis, we examine how to increase the value of Earth observations made by coordinating across multiple collection systems. We consider both air and space assets in an asynchronous and distributed environment. We consider requests with time windows and priority levels, some of which require simultaneous observations by different sensors. We consider how these improvements could impact Earth observing sensors in two use areas; climate studies and intelligence collection operations. The primary contributions of this thesis include our approach to the asynchronous and distributed nature of the problem and the development of a value function to facilitate the coordination of the observations with multiple surveillance assets. We embed a carefully constructed value function in a simple optimization problem that we prove can be solved as a Linear Programming (LP) problem. We solve the optimization problem repeatedly over time to intelligently allocate requests to single-mission planners, or "sub-planners." We then show that the value function performs as we intend through empirical and statistical analysis. To test our methodologies, we integrate the coordination planner with two types of sub-planners, an Unmanned Aerial Vehicle (UAV) sub-planner, and a satellite sub-planner. We use the coordinator to generate observation plans for two notional operational Earth Science scenarios. Specifically, we show that coordination offers improvements in the priority of the requests serviced, the quality of those observations, and the ability to take dual collections. We conclude that a coordinated planning framework provides clear benefits.by Thomas Michael Herold.S.M

Organic over-the-horizon targeting for the 2025 surface fleet

Please note that this activity was not conducted in accordance with Federal, DOD, and Navy Human Research Protection RegulationsAdversarial advances in the proliferation of anti-access/area-denial (A2/AD) techniques requires an innovative approach to the design of a maritime system of systems capable of detecting, classifying, and engaging targets in support of organic over-the-horizon (OTH) tactical offensive operations in the 2025–2030 timeframe. Using a systems engineering approach, this study considers manned and unmanned systems in an effort to develop an organic OTH targeting capability for U.S. Navy surface force structures of the future. Key attributes of this study include overall system requirements, limitations, operating area considerations, and issues of interoperability and compatibility. Multiple alternative system architectures are considered and analyzed for feasibility. The candidate architectures include such systems as unmanned aerial vehicles (UAVs), as well as prepositioned undersea and low-observable surface sensor and communication networks. These unmanned systems are expected to operate with high levels of autonomy and should be designed to provide or enhance surface warfare OTH targeting capabilities using emerging extended-range surface-to-surface weapons. This report presents the progress and results of the SEA-21A capstone project with the recommendation that the U.S. Navy explore the use of modestly-sized, network-centric UAVs to enhance the U.S. Navy’s ability to conduct surface-based OTH tactical offensive operations by 2025.http://archive.org/details/organicovertheho1094545933Approved for public release; distribution is unlimited