Improving situation awareness of a single human operator interacting with multiple unmanned vehicles: first results

In the context of the supervision of one or several unmanned vehicles by a human operator, the design of an adapted user interface is a major challenge. Therefore, in the context of an existing experimental set up composed of a ground station and heterogeneous unmanned ground and air vehicles we aim at redesigning the human-robot interactions to improve the operator's situation awareness. We base our new design on a classical user centered approach

SUAVE: Integrating UAV video using a 3D model

Controlling an unmanned aerial vehicle (UAV) requires the operator to perform continuous surveillance and path planning. The operator's situation awareness degrades as an increasing number of surveillance videos must be viewed and integrated. The Picture-in-Picture display (PiP) provides a solution for integrating multiple UAV camera video by allowing the operator to view the video feed in the context of surrounding terrain. The experimental SUAVE (Simple Unmanned Aerial Vehicle Environment) display extends PiP methods by sampling imagery from the video stream to texture a 3D map of the terrain. The operator can then inspect this imagery using world in miniature (WIM) or fly-through methods. We investigate the properties and advantages of SUAVE in the context of a search mission with 3 UAVs

Architecting Human Operator Trust in Automation to Improve System Effectiveness in Multiple Unmanned Aerial Vehicles (UAV)

Current Unmanned Aerial System (UAS) designs require multiple operators for each vehicle, partly due to imperfect automation matched with the complex operational environment. This study examines the effectiveness of future UAS automation by explicitly addressing the human/machine trust relationship during system architecting. A pedigreed engineering model of trust between human and machine was developed and applied to a laboratory-developed micro-UAS for Special Operations. This unprecedented investigation answered three primary questions. Can previous research be used to create a useful trust model for systems engineering? How can trust be considered explicitly within the DoD Architecture Framework? Can the utility of architecting trust be demonstrated on a given UAS architecture? By addressing operator trust explicitly during architecture development, system designers can incorporate more effective automation. The results provide the Systems Engineering community a new modeling technique for early human systems integration

A Briefing on Metrics and Risks for Autonomous Decision-Making in Aerospace Applications

Significant technology advances will enable future aerospace systems to safely and reliably make decisions autonomously, or without human interaction. The decision-making may result in actions that enable an aircraft or spacecraft in an off-nominal state or with slightly degraded components to achieve mission performance and safety goals while reducing or avoiding damage to the aircraft or spacecraft. Some key technology enablers for autonomous decision-making include: a continuous state awareness through the maturation of the prognostics health management field, novel sensor development, and the considerable gains made in computation power and data processing bandwidth versus system size. Sophisticated algorithms and physics based models coupled with these technological advances allow reliable assessment of a system, subsystem, or components. Decisions that balance mission objectives and constraints with remaining useful life predictions can be made autonomously to maintain safety requirements, optimal performance, and ensure mission objectives. This autonomous approach to decision-making will come with new risks and benefits, some of which will be examined in this paper. To start, an account of previous work to categorize or quantify autonomy in aerospace systems will be presented. In addition, a survey of perceived risks in autonomous decision-making in the context of piloted aircraft and remotely piloted or completely autonomous unmanned autonomous systems (UAS) will be presented based on interviews that were conducted with individuals from industry, academia, and government

A Framework for Analyzing and Discussing Level of Human Control Abstraction

It is often useful to understand the impact of an artificial teammate upon human workload in human-machine teams. Levels of Autonomy (LoA) differentiate systems based on control authority. Unfortunately, human workload is not necessarily correlated with LoA. An alternate classification framework, designated the Level of Human Control Abstraction (LHCA), is proposed. LHCA differentiates system states based on the control and monitoring tasks performed and the level of decisions made by humans. The framework defines five levels, designed to differentiate between system states based upon anticipated levels of human attention. This presentation will summarize the framework and demonstrate its application

Designing Decision and Collaboration Support Technology for Operators in Multi-UAV Operations Teams

Effective team collaboration and timely decision-making significantly influence the outcome of time-sensitive military operations. The increasing complexity introduced by the recent move towards network centric operations (NCO) in U.S. military operations provides additional challenges for efficient decision-making. Future operations will include co-located and distributed teams composed of operators from difference services, often at different global locations. Military operations which require extremely quick decisions, such as operations dealing with time-sensitive targets (TST) like improvised explosive devices (IEDs), are particularly challenging in NCO teaming environments. Operators in TST environments not only have to manage overwhelming amounts of target-related information, but also have the overhead of communicating and coordinating with co-located and distributed team members. Given the increasing trend for modern hostile forces to employ unconventional weapons such as IEDs and suicide bombs, the success of TST operations are becoming critical to current and future military operations. Providing TST teams with effective tools for communicating and coordinating their efforts is key to enabling their success.Prepared For Boeing, Phantom Work

Mission programming for flying ensembles: combining planning with self-organization

The application of autonomous mobile robots can improve many situations of our daily lives. Robots can enhance working conditions, provide innovative techniques for different research disciplines, and support rescue forces in an emergency. In particular, flying robots have already shown their potential in many use-cases when cooperating in ensembles. Exploiting this potential requires sophisticated measures for the goal-oriented, application-specific programming of flying ensembles and the coordinated execution of so defined programs. Because different goals require different robots providing different capabilities, several software approaches emerged recently that focus on specifically designed robots. These approaches often incorporate autonomous planning, scheduling, optimization, and reasoning attributable to classic artificial intelligence. This allows for the goal-oriented instruction of ensembles, but also leads to inefficiencies if ensembles grow large or face uncertainty in the environment. By leaving the detailed planning of executions to individuals and foregoing optimality and goal-orientation, the selforganization paradigm can compensate for these drawbacks by scalability and robustness. In this thesis, we combine the advantageous properties of autonomous planning with that of self-organization in an approach to Mission Programming for Flying Ensembles. Furthermore, we overcome the current way of thinking about how mobile robots should be designed. Rather than assuming fixed-design robots, we assume that robots are modifiable in terms of their hardware at run-time. While using such robots enables their application in many different use cases, it also requires new software approaches for dealing with this flexible design. The contributions of this thesis thus are threefold. First, we provide a layered reference architecture for physically reconfigurable robot ensembles. Second, we provide a solution for programming missions for ensembles consisting of such robots in a goal-oriented fashion that provides measures for instructing individual robots or entire ensembles as desired in the specific use case. Third, we provide multiple self-organization mechanisms to deal with the system’s flexible design while executing such missions. Combining different self-organization mechanisms ensures that ensembles satisfy the static requirements of missions. We provide additional self-organization mechanisms for coordinating the execution in ensembles ensuring they meet the dynamic requirements of a mission. Furthermore, we provide a solution for integrating goal-oriented swarm behavior into missions using a general pattern we have identified for trajectory-modification-based swarm behavior. Using that pattern, we can modify, quantify, and further process the emergent effect of varying swarm behavior in a mission by changing only the parameters of its implementation. We evaluate results theoretically and practically in different case studies by deploying our techniques to simulated and real hardware.Der Einsatz von autonomen mobilen Robotern kann viele Abläufe unseres täglichen Lebens erleichtern. Ihr Einsatz kann Arbeitsbedingungen verbessern, als innovative Technik für verschiedene Forschungsdisziplinen dienen oder Rettungskräfte im Einsatz unterstützen. Insbesondere Flugroboter haben ihr Potenzial bereits in vielerlei Anwendungsfällen gezeigt, gerade wenn mehrere in Ensembles eingesetzt werden. Das Potenzial fliegender Ensembles zielgerichtet und anwendungsspezifisch auszuschöpfen erfordert ausgefeilte Programmiermethoden und Koordinierungsverfahren. Zu diesem Zweck sind zuletzt viele unterschiedliche und auf speziell entwickelte Roboter zugeschnittene Softwareansätze entstanden. Diese verwenden oft klassische Planungs-, Scheduling-, Optimierungs- und Reasoningverfahren. Während dies vor allem den zielgerichteten Einsatz von Ensembles ermöglicht, ist es jedoch auch oft ineffizient, wenn die Ensembles größer oder deren Einsatzumgebungen unsicher werden. Die genannten Nachteile können durch das Paradigma der Selbstorganisation kompensiert werden: Falls Anwendungen nicht zwangsläufig auf Optimalität und strikte Zielorientierung ausgelegt sind, kann so Skalierbarkeit und Robustheit im System erreicht werden. In dieser Arbeit werden die vorteilhaften Eigenschaften klassischer Planungstechniken mit denen der Selbstorganisation in einem Ansatz zur Missionsprogrammierung für fliegende Ensembles kombiniert. In der dafür entwickelten Lösung wird von der aktuell etablierten Ansicht einer unveränderlichen Roboterkonstruktion abgewichen. Stattdessen wird die Hardwarezusammenstellung der Roboter als zur Laufzeit modifizierbar angesehen. Der Einsatz solcher Roboter erfordert neue Softwareansätze um mit genannter Flexibilität umgehen zu können. Die hier vorgestellten Beiträge zu diesem Thema lassen sich in drei Punkten zusammenfassen: Erstens wird eine Schichtenarchitektur als Referenz für physikalisch konfigurierbare Roboterensembles vorgestellt. Zweitens wird eine Lösung zur zielorientierten Missions-Programmierung für derartige Ensembles präsentiert, mit der sowohl einzelne Roboter als auch ganze Ensembles instruiert werden können. Drittens werden mehrere Selbstorganisationsmechanismen vorgestellt, die die autonome Ausführung so erstellter Missionen ermöglichen. Durch die Kombination verschiedener Selbstorganisationsmechanismen wird sichergestellt, dass Ensembles die missionsspezifischen Anforderungen erfüllen. Zusätzliche Selbstorganisationsmechanismen ermöglichen die koordinierte Ausführung der Missionen durch die Ensembles. Darüber hinaus bietet diese Lösung die Möglichkeit der Integration zielorientierten Schwarmverhaltens. Durch ein allgemeines algorithmisches Verfahren für auf Trajektorien-Modifikation basierendes Schwarmverhalten können allein durch die Änderung des Parametersatzes unterschiedliche emergente Effekte in einer Mission erzielt, quantifiziert und weiterverarbeitet werden. Zur theoretischen und praktischen Evaluierung der Ergebnisse dieser Arbeit wurden die vorgestellten Techniken in verschiedenen Fallstudien auf simulierter sowie realer Hardware zum Einsatz gebracht