Operator Scheduling Strategies in Supervisory Control of Multiple UAVs

The application of network centric operations to time-constrained command and control environments will mean that human operators will be increasingly responsible for multiple simultaneous supervisory control tasks. One such futuristic application will be the control of multiple unmanned aerial vehicles (UAVs) by a single operator. To achieve such performance in complex, time critical, and high risk settings, automated systems will be required both to guarantee rapid system response as well as manageable workload for operators. Through the development of a simulation test bed for human supervisory control of multiple independent UAVs by a single operator, this paper presents recent efforts to investigate workload mitigation strategies as a function of increasing automation. A humanin- the-loop experiment revealed that under low workload conditions, operators’ cognitive strategies were relatively robust across increasing levels of automated decision support. However, when provided with explicit automated recommendations and with the ability to negotiate with external agencies for delays in arrival times for targets, operators inappropriately fixated on the need to globally optimize their schedules. In addition, without explicit visual representation of uncertainty, operators tended to treated all probabilities uniformly. This study also revealed that operators that reached cognitive saturation adapted two very distinct management strategies, which led to varying degrees of success. Lastly, operators with management-by-exception decision support exhibited evidence of automation bias.This research was sponsored by Boeing Phantom Works

The Impact of Intelligent Aiding for Multiple Unmanned Aerial Vehicle Schedule Management

There is increasing interest in designing systems such that the current many-to-one ratio of operators to unmanned vehicles (UVs) can be inverted. Instead of lower-level tasks performed by today’s UV teams, the sole operator would focus on high-level supervisory control tasks. A key challenge in the design of such single-operator systems will be the need to minimize periods of excessive workload that arise when critical tasks for several UVs occur simultaneously. Thus some kind of decision support is needed that facilitates an operator’s ability to evaluate different action alternatives for managing a multiple UV mission schedule in real-time. This paper describes two decision support experiments that attempted to provide UAV operators with multivariate scheduling assistance, with mixed results. Those automated decision support tools that provided more local, as opposed to global, visual recommendations produced superior performance, suggesting that meta-information displays could saturate operators and reduce performance.This research was sponsored by Boeing Phantom Work and Mitre, Inc

Teams organization and performance analysis in autonomous human-robot teams

This paper proposes a theory of human control of robot teams based on considering how people coordinate across different task allocations. Our current work focuses on domains such as foraging in which robots perform largely independent tasks. The present study addresses the interaction between automation and organization of human teams in controlling large robot teams performing an Urban Search and Rescue (USAR) task. We identify three subtasks: perceptual search-visual search for victims, assistance-teleoperation to assist robot, and navigation-path planning and coordination. For the studies reported here, navigation was selected for automation because it involves weak dependencies among robots making it more complex and because it was shown in an earlier experiment to be the most difficult. This paper reports an extended analysis of the two conditions from a larger four condition study. In these two "shared pool" conditions Twenty four simulated robots were controlled by teams of 2 participants. Sixty paid participants (30 teams) were recruited to perform the shared pool tasks in which participants shared control of the 24 UGVs and viewed the same screens. Groups in the manual control condition issued waypoints to navigate their robots. In the autonomy condition robots generated their own waypoints using distributed path planning. We identify three self-organizing team strategies in the shared pool condition: joint control operators share full authority over robots, mixed control in which one operator takes primary control while the other acts as an assistant, and split control in which operators divide the robots with each controlling a sub-team. Automating path planning improved system performance. Effects of team organization favored operator teams who shared authority for the pool of robots. © 2010 ACM

The Underpinnings of Workload in Unmanned Vehicle Systems

This paper identifies and characterizes factors that contribute to operator workload in unmanned vehicle systems. Our objective is to provide a basis for developing models of workload for use in design and operation of complex human-machine systems. In 1986, Hart developed a foundational conceptual model of workload, which formed the basis for arguably the most widely used workload measurement techniquethe NASA Task Load Index. Since that time, however, there have been many advances in models and factor identification as well as workload control measures. Additionally, there is a need to further inventory and describe factors that contribute to human workload in light of technological advances, including automation and autonomy. Thus, we propose a conceptual framework for the workload construct and present a taxonomy of factors that can contribute to operator workload. These factors, referred to as workload drivers, are associated with a variety of system elements including the environment, task, equipment and operator. In addition, we discuss how workload moderators, such as automation and interface design, can be manipulated in order to influence operator workload. We contend that workload drivers, workload moderators, and the interactions among drivers and moderators all need to be accounted for when building complex, human-machine systems

,The Impact of Human-Automation Collaboration in Decentralized Multiple Unmanned Vehicle Control

For future systems that require one or a small team of operators to supervise a network of automated agents, automated planners are critical since they are faster than humans for path planning and resource allocation in multivariate, dynamic, time-pressured environments. However, such planners can be brittle and unable to respond to emergent events. Human operators can aid such systems by bringing their knowledge-based reasoning and experience to bear. Given a decentralized task planner and a goal-based operator interface for a network of unmanned vehicles in a search, track, and neutralize mission, we demonstrate with a human-on-the-loop experiment that humans guiding these decentralized planners improved system performance by up to 50%. However, those tasks that required precise and rapid calculations were not significantly improved with human aid. Thus, there is a shared space in such complex missions for human–automation collaboration

Supporting UAVs with Edge Computing: A Review of Opportunities and Challenges

Over the last years, Unmanned Aerial Vehicles (UAVs) have seen significant advancements in sensor capabilities and computational abilities, allowing for efficient autonomous navigation and visual tracking applications. However, the demand for computationally complex tasks has increased faster than advances in battery technology. This opens up possibilities for improvements using edge computing. In edge computing, edge servers can achieve lower latency responses compared to traditional cloud servers through strategic geographic deployments. Furthermore, these servers can maintain superior computational performance compared to UAVs, as they are not limited by battery constraints. Combining these technologies by aiding UAVs with edge servers, research finds measurable improvements in task completion speed, energy efficiency, and reliability across multiple applications and industries. This systematic literature review aims to analyze the current state of research and collect, select, and extract the key areas where UAV activities can be supported and improved through edge computing

Attention Allocation for Human Multi-Robot Control: Cognitive Analysis based on Behavior Data and Hidden States

Human multi-robot interaction exploits both the human operator’s high-level decision-making skills and the robotic agents’ vigorous computing and motion abilities. While controlling multi-robot teams, an operator’s attention must constantly shift between individual robots to maintain sufficient situation awareness. To conserve an operator’s attentional resources, a robot with self reflect capability on its abnormal status can help an operator focus her attention on emergent tasks rather than unneeded routine checks. With the proposing self-reflect aids, the human-robot interaction becomes a queuing framework, where the robots act as the clients to request for interaction and an operator acts as the server to respond these job requests. This paper examined two types of queuing schemes, the self-paced Open-queue identifying all robots’ normal/abnormal conditions, whereas the forced-paced shortest-job-first (SJF) queue showing a single robot’s request at one time by following the SJF approach. As a robot may miscarry its experienced failures in various situations, the effects of imperfect automation were also investigated in this paper. The results suggest that the SJF attentional scheduling approach can provide stable performance in both primary (locate potential targets) and secondary (resolve robots’ failures) tasks, regardless of the system’s reliability levels. However, the conventional results (e.g., number of targets marked) only present little information about users’ underlying cognitive strategies and may fail to reflect the user’s true intent. As understanding users’ intentions is critical to providing appropriate cognitive aids to enhance task performance, a Hidden Markov Model (HMM) is used to examine operators’ underlying cognitive intent and identify the unobservable cognitive states. The HMM results demonstrate fundamental differences among the queuing mechanisms and reliability conditions. The findings suggest that HMM can be helpful in investigating the use of human cognitive resources under multitasking environments

Unmanned Aerial Vehicle Domain: Areas of Research

Unmanned aerial vehicles (UAVs) domain has seen rapid developments in recent years. As the number of UAVs increases and as the missions involving UAVs vary, new research issues surface. An overview of the existing research areas in the UAV domain has been presented including the nature of the work categorised under different groups. These research areas are divided into two main streams: Technological and operational research areas. The research areas in technology are divided into onboard and ground technologies. The research areas in operations are divided into organization level, brigade level, user level, standards and certifications, regulations and legal, moral, and ethical issues. This overview is intended to serve as a starting point for fellow researchers new to the domain, to help researchers in positioning their research, identifying related research areas, and focusing on the right issues.Defence Science Journal, Vol. 65, No. 4, July 2015, pp. 319-329, DOI: http://dx.doi.org/10.14429/dsj.65.863

Modified Cooper Harper scales for assessing unmanned vehicle displays

Unmanned vehicle (UV) displays are often the only information link between operators and vehicles, so their design is critical to mission success. However, there is currently no standardized methodology for operators to subjectively assess a display's support of mission tasks. This paper proposes a subjective UV display evaluation tool: the Modified Cooper-Harper for Unmanned Vehicle Displays (MCH-UVD). The MCH-UVD is adapted from the Cooper-Harper aircraft handling scale by shifting focus to support of operator information processing. An experiment was conducted to evaluate and refine the MCH-UVD, as well as assess the need for mission-specific versus general versions. Participants (86%) thought that MCH-UVD helped them identify display deficiencies, and 32% said that they could not have identified the deficiencies without the tool. No major additional benefits were observed with mission-specific versions over the general scale.U.S. Army Aberdeen Test Cente

Operator Objective Function Guidance for a Real-time Unmanned Vehicle Scheduling Algorithm

Advances in autonomy have made it possible to invert the typical operator-to-unmanned-vehicle ratio so that asingle operator can now control multiple heterogeneous unmanned vehicles. Algorithms used in unmanned-vehicle path planning and task allocation typically have an objective function that only takes into account variables initially identified by designers with set weightings. This can make the algorithm seemingly opaque to an operator and brittle under changing mission priorities. To address these issues, it is proposed that allowing operators to dynamically modify objective function weightings of an automated planner during a mission can have performance benefits. A multiple-unmanned-vehicle simulation test bed was modified so that operators could either choose one variable or choose any combination of equally weighted variables for the automated planner to use in evaluating mission plans. Results from a human-participant experiment showed that operators rated their performance and confidence highest when using the dynamic objective function with multiple objectives. Allowing operators to adjust multiple objectives resulted in enhanced situational awareness, increased spare mental capacity, fewer interventions to modify the objective function, and no significant differences in mission performance. Adding this form of flexibility and transparency to automation in future unmanned vehicle systems could improve performance, engender operator trust, and reduce errors.Aurora Flight Sciences, U.S. Office of Naval Researc