The Impact of Intelligent Aiding for Multiple Unmanned Aerial Vehicle Schedule Management

There is increasing interest in designing systems such that the current many-to-one ratio of operators to unmanned vehicles (UVs) can be inverted. Instead of lower-level tasks performed by today’s UV teams, the sole operator would focus on high-level supervisory control tasks. A key challenge in the design of such single-operator systems will be the need to minimize periods of excessive workload that arise when critical tasks for several UVs occur simultaneously. Thus some kind of decision support is needed that facilitates an operator’s ability to evaluate different action alternatives for managing a multiple UV mission schedule in real-time. This paper describes two decision support experiments that attempted to provide UAV operators with multivariate scheduling assistance, with mixed results. Those automated decision support tools that provided more local, as opposed to global, visual recommendations produced superior performance, suggesting that meta-information displays could saturate operators and reduce performance.This research was sponsored by Boeing Phantom Work and Mitre, Inc

Path planning and collision risk management strategy for multi-UAV systems in 3D environments

This article belongs to the Special Issue Smooth Motion Planning for Autonomous VehiclesMulti-UAV systems are attracting, especially in the last decade, the attention of researchers and companies of very different fields due to the great interest in developing systems capable of operating in a coordinated manner in complex scenarios and to cover and speed up applications that can be dangerous or tedious for people: search and rescue tasks, inspection of facilities, delivery of goods, surveillance, etc. Inspired by these needs, this work aims to design, implement and analyze a trajectory planning and collision avoidance strategy for multi-UAV systems in 3D environments. For this purpose, a study of the existing techniques for both problems is carried out and an innovative strategy based on Fast Marching Square¿for the planning phase¿and a simple priority-based speed control¿as the method for conflict resolution¿is proposed, together with prevention measures designed to try to limit and reduce the greatest number of conflicting situations that may occur between vehicles while they carry out their missions in a simulated 3D urban environment. The performance of the algorithm is evaluated successfully on the basis of certain conveniently chosen statistical measures that are collected throughout the simulation runs.This research was funded by the EUROPEAN COMMISSION: Innovation and Networks Executive Agency (INEA), through the European H2020 LABYRINTH project. Grant agreement H2020-MG-2019-TwoStages-861696

The Role of Human-Automation Consensus in Multiple Unmanned Vehicle Scheduling

Objective: This study examined the impact of increasing automation replanning rates on operator performance and workload when supervising a decentralized network of heterogeneous unmanned vehicles. Background: Futuristic unmanned vehicles systems will invert the operator-to-vehicle ratio so that one operator can control multiple dissimilar vehicles connected through a decentralized network. Significant human-automation collaboration will be needed because of automation brittleness, but such collaboration could cause high workload. Method: Three increasing levels of replanning were tested on an existing multiple unmanned vehicle simulation environment that leverages decentralized algorithms for vehicle routing and task allocation in conjunction with human supervision. Results: Rapid replanning can cause high operator workload, ultimately resulting in poorer overall system performance. Poor performance was associated with a lack of operator consensus for when to accept the automation’s suggested prompts for new plan consideration as well as negative attitudes toward unmanned aerial vehicles in general. Participants with video game experience tended to collaborate more with the automation, which resulted in better performance. Conclusion: In decentralized unmanned vehicle networks, operators who ignore the automation’s requests for new plan consideration and impose rapid replans both increase their own workload and reduce the ability of the vehicle network to operate at its maximum capacity. Application: These findings have implications for personnel selection and training for futuristic systems involving human collaboration with decentralized algorithms embedded in networks of autonomous systems.Aurora Flight Sciences Corp.United States. Office of Naval Researc

Decision Support Design for Workload Mitigation in Human Supervisory Control of Multiple Unmanned Aerial Vehicles

As UAVs become increasingly autonomous, the multiple personnel currently required to operate a single UAV may eventually be superseded by a single operator concurrently managing multiple UAVs. Instead of lower-level tasks performed by today’s UAV teams, the sole operator would focus on high-level supervisory control tasks such as monitoring mission timelines and reacting to emergent mission events. A key challenge in the design of such single-operator systems will be the need to minimize periods of excessive workload that could arise when critical tasks for several UAVs occur simultaneously. To a certain degree, it is possible to predict and mitigate such periods in advance. However, actions that mitigate a particular period of high workload in the short term may create long term episodes of high workload that were previously non-existent. Thus some kind of decision support is needed that facilitates an operator’s ability to evaluate different options for managing a mission schedule in real-time. This paper describes two decision support visualizations designed for supervisory control of four UAVs performing a time-critical targeting mission. A configural display common to both visualizations, named the StarVis, was designed to highlight potential periods of high workload corresponding to the current mission timeline, as well as “what if” projections of possible high workload periods based upon different operator options. The first visualization design allows an operator to compare different high workload mitigation options for individual UAVs. This is termed the local visualization. The second visualization is indicates the combined effects of multiple high workload mitigation decisions on the timeline. This is termed the global visualization. The main advantage of the local visualization is that options can be compared directly; however, the possible effects of these options on the mission timeline are only indicated for the individual UAV primarily affected by the decision. For the global visualization, different decisions can be combined to show possible effects on the system propagated across all UAVs, but the different alternatives of a single decision option alternative cannot be directly compared. An experiment was conducted testing these visualizations against a control with no visualization. Results showed that subject using the local visualization had better performance, higher situational awareness, and no significant increase in workload over the other two experimental conditions. This occurred despite the fact that the local and global StarVis displays were very similar. Not only did the Global StarVis produce degraded results as compared to the local StarVis, but those participants with no visualization performed as well as those with the global StarVis. This disparity in performance despite strong visual similarities in the StarVis designs is attributed to operators’ inability to process all the information presented in the global StarVis as well as the fact that participants with the local StarVis were able to rapidly develop effective cognitive problem strategies. This research effort highlights a very important design consideration, in that a single decision support design can produce very different performance results when applied at different levels of abstraction.Prepared for Kevin Burns, The MITRE Corporatio

Research on the collision avoidance algorithm for fixed-wing UAVs based on maneuver coordination and planned trajectories prediction

El mail de contacte de l'autor correspon a la Universitat Autònoma de BarcelonaThis paper presents a novel collision avoidance (CA) algorithm for a cooperative fixed-wing unmanned aerial vehicle (UAV). The method is based on maneuver coordination and planned trajectory prediction. Each aircraft in a conflict generates three available maneuvers and predicts the corresponding planned trajectories. The algorithm coordinates planned trajectories between participants in a conflict, determines which combination of planned trajectories provides the best separation, eventually makes an agreement on the maneuver for collision avoidance and activates the preferred maneuvers when a collision is imminent. The emphasis is placed on providing protection for UAVs, while activating maneuvers late enough to reduce interference, which is necessary for collision avoidance in the formation and clustering of UAVs. The CA has been validated with various simulations to show the advantage of collision avoidance for continuous conflicts in multiple, high-dynamic, high-density and three-dimensional (3D) environments. It eliminates the disadvantage of traditional CA, which has high uncertainty, and takes the performance parameters of different aircraft into consideration and makes full use of the maneuverability of fixed-wing aircraft

Exploration vs. Data Refinement via Multiple Mobile Sensors

We examine the deployment of multiple mobile sensors to explore an unknown region to map regions containing concentration of a physical quantity such as heat, electron density, and so on. The exploration trades off between two desiderata: to continue taking data in a region known to contain the quantity of interest with the intent of refining the measurements vs. taking data in unobserved areas to attempt to discover new regions where the quantity may exist. Making reasonable and practical decisions to simultaneously fulfill both goals of exploration and data refinement seem to be hard and contradictory. For this purpose, we propose a general framework that makes value-laden decisions for the trajectory of mobile sensors. The framework employs a Gaussian process regression model to predict the distribution of the physical quantity of interest at unseen locations. Then, the decision-making on the trajectories of sensors is performed using an epistemic utility controller. An example is provided to illustrate the merit and applicability of the proposed framework

A Cooperative Architecting Procedure for Systems of Systems Based on Self-Adaptive Multi-Agent Systems

Depuis la seconde guerre mondiale, l’ingénierie des systèmes a permis le développement de méthodologies pour contrôler le développement de systèmes et de projets de plus en plus complexes. En 1990, la chute de l’URSS a provoqué un changement de doctrine militaire aux Etats-Unis en passant d’une confrontation bipolaire à une mondialisation des conflits comportant une grande variété de menaces. Sa nouvelle doctrine était de faire collaborer ses systèmes de défense existants pour produire un système de défense de haut niveau, décentralisé, adaptable et composé de systèmes indépendants. C’est l’apparition du concept de Système de Systèmes (SdS).Cette thèse de doctorat propose un nouveau modèle de SdS appelé SApHESIA (SoS ArchitectingHEuriStIc based on Agent), ainsi qu’une nouvelle méthodologie d’architecture. Cette nouvelle méthodologie est basée sur une coopération complète entre tous les composants du SdS, lui permettant d’évoluer de lui-même afin de faire face à des événements inattendus de son environnement tels que des menaces. Enfin, ce travail est testé à travers 4exemples issus de différents domaines (militaire, logistique et exploratoire).Since the World War II, researchers have tended to develop methodologies and tools tobuild and control the development of more and more complex systems and projects. Thisinter-disciplinary research area has been called Systems Engineering (SE) and continues tobe developed nowadays. In 1990, the fall of USSR led the US Department of Defense (DoD)to re-think its defense doctrine and to switch from a one opponent confrontation to a globalizationof conflicts with a huge variety of scenarios. Its idea was to re-use and join itsdefense systems by producing a huge, decentralized and adaptive defense system that iscomposed of existing and independents (complex) systems. This is the apparition of theSystem of Systems (SoS) concept. After 2000’s, this concept spreads in civil domains suchas crisis management or logistic systems. More precisely, a SoS is a complex system characterizedby the particular nature of its components: these latter, which are systems, tend tobe managerially and operationally independent as well as geographically distributed. Thisspecific characterization led to re-think research areas of classic SE such as definition, taxonomy,modeling, architecting and so on. SoS architecting focuses on the way independentcomponents of a SoS can be dynamically structured and can change autonomously theirinteractions in an efficient manner to fulfill the goal of the SoS and to cope with the highdynamics of the environment. This PhD thesis mainly focuses on two SoS research areas: 1)SoS modeling and 2) SoS architecting. To achieve the first point, we propose a new modelcalled SApHESIA (SoS Architecting HEuriStIc based on Agent). We have used set theoryand ABM (Agent-Based Model) paradigm to define this model that takes into account thecharacteristics of SoS. Secondly, we propose a new SoS architecting procedure based on theAdaptive Multi-Agent System (AMAS) approach that advocates full cooperation betweenall the components of the SoS through the concept of criticality. This criticality is a metricthat represents the distance between the current state of a component and its goals. In thisprocedure, the SoS architecture evolves over time to self-adapt to the dynamics of the environmentin which it is plunged, while taking into account the respective local goals of itscomponents. Finally we instantiate this model and this procedure through 4 examples fromdifferent domains (military, logistics and exploratory missions) and validate the feasibility,the efficiency, the effectiveness and the robustness of the SoS architecting procedure we havedeveloped and proposed

A Cooperative Architecting Procedure for Systems of Systems Based on Self-Adaptive Multi-Agent Systems

Depuis la seconde guerre mondiale, l’ingénierie des systèmes a permis le développement de méthodologies pour contrôler le développement de systèmes et de projets de plus en plus complexes. En 1990, la chute de l’URSS a provoqué un changement de doctrine militaire aux Etats-Unis en passant d’une confrontation bipolaire à une mondialisation des conflits comportant une grande variété de menaces. Sa nouvelle doctrine était de faire collaborer ses systèmes de défense existants pour produire un système de défense de haut niveau, décentralisé, adaptable et composé de systèmes indépendants. C’est l’apparition du concept de Système de Systèmes (SdS).Cette thèse de doctorat propose un nouveau modèle de SdS appelé SApHESIA (SoS ArchitectingHEuriStIc based on Agent), ainsi qu’une nouvelle méthodologie d’architecture. Cette nouvelle méthodologie est basée sur une coopération complète entre tous les composants du SdS, lui permettant d’évoluer de lui-même afin de faire face à des événements inattendus de son environnement tels que des menaces. Enfin, ce travail est testé à travers 4exemples issus de différents domaines (militaire, logistique et exploratoire).Since the World War II, researchers have tended to develop methodologies and tools tobuild and control the development of more and more complex systems and projects. Thisinter-disciplinary research area has been called Systems Engineering (SE) and continues tobe developed nowadays. In 1990, the fall of USSR led the US Department of Defense (DoD)to re-think its defense doctrine and to switch from a one opponent confrontation to a globalizationof conflicts with a huge variety of scenarios. Its idea was to re-use and join itsdefense systems by producing a huge, decentralized and adaptive defense system that iscomposed of existing and independents (complex) systems. This is the apparition of theSystem of Systems (SoS) concept. After 2000’s, this concept spreads in civil domains suchas crisis management or logistic systems. More precisely, a SoS is a complex system characterizedby the particular nature of its components: these latter, which are systems, tend tobe managerially and operationally independent as well as geographically distributed. Thisspecific characterization led to re-think research areas of classic SE such as definition, taxonomy,modeling, architecting and so on. SoS architecting focuses on the way independentcomponents of a SoS can be dynamically structured and can change autonomously theirinteractions in an efficient manner to fulfill the goal of the SoS and to cope with the highdynamics of the environment. This PhD thesis mainly focuses on two SoS research areas: 1)SoS modeling and 2) SoS architecting. To achieve the first point, we propose a new modelcalled SApHESIA (SoS Architecting HEuriStIc based on Agent). We have used set theoryand ABM (Agent-Based Model) paradigm to define this model that takes into account thecharacteristics of SoS. Secondly, we propose a new SoS architecting procedure based on theAdaptive Multi-Agent System (AMAS) approach that advocates full cooperation betweenall the components of the SoS through the concept of criticality. This criticality is a metricthat represents the distance between the current state of a component and its goals. In thisprocedure, the SoS architecture evolves over time to self-adapt to the dynamics of the environmentin which it is plunged, while taking into account the respective local goals of itscomponents. Finally we instantiate this model and this procedure through 4 examples fromdifferent domains (military, logistics and exploratory missions) and validate the feasibility,the efficiency, the effectiveness and the robustness of the SoS architecting procedure we havedeveloped and proposed