Cognitively Sensitive User Interface for Command and Control Applications

While there are broad guidelines for display or user interface design, creating effective human-computer interfaces for complex, dynamic systems control is challenging. Ad hoc approaches which consider the human as an afterthought are limiting. This research proposed a systematic approach to human / computer interface design that focuses on both the semantic and syntactic aspects of display design in the context of human-in-the-loop supervisory control of intelligent, autonomous multi-agent simulated unmanned aerial vehicles (UAVs). A systematic way to understand what needs to be displayed, how it should be displayed, and how the integrated system needs to be assessed is outlined through a combination of concepts from naturalistic decision making, semiotic analysis, and situational awareness literature. A new sprocket-based design was designed and evaluated in this research. For the practical designer, this research developed a systematic, iterative design process: design using cognitive sensitive principles, test the new interface in a laboratory situation; bring in subject matter experts to examine the interface in isolation; and finally, incorporate the resulting feedback into a full-size simulation. At each one of these steps, the operator, the engineer and the designer reexamined the results

Development of a Multi-UAV Simulator to Analyze the Behavior of Operators in Coastal Surveillance Missions

This Master Thesis 1 presents the design and development of a computer simulator created for executing and supervising missions carried out by multiple Unmanned Aerial Vehicles (UAVs). The aim of this simulator is to provide an open, simple and accessible environment to train and analyze the performance and evolution of low-experienced human operators supervising and controlling a team of UAVs. This work is divided into two parts. The rst one is focused on describing the simulator mechanisms and architecture. To accomplish the required accessibility of this tool for novice users, the simulator has been implemented following a web architecture, where only a web browser is needed to execute it. Also, in order to engage and challenge the operator, some gami cation elements have been added, bringing the simulation closer to a videogame experience. The second part of this work uses the developed simulator to carry out several experiments with novice users. A set of performance metrics is designed to de ne the pro le of a user, and based on those pro les, we run and validate some clustering algorithms to obtain groups of users with common performance pro les. These results are analyzed to extract behavioral patterns that distinguish and rank the di erent users in the experiment, allowing the identi cation and selection of potential expert operators.El presente Trabajo Fin de Máster 2 presenta el diseño y desarrollo de un simulador creado con el fin de ejecutar y supervisar misiones llevadas a cabo por múltiples Vehículos Aéreos no Tripulados (UAVs). El objetivo de este simulador es ofrecer un entorno simple y accesible donde entrenar y analizar el rendimiento y la evolución de operadores inexpertos mientras supervisan y controlan un equipo de UAVs. Este trabajo se divide en dos partes. La primera está enfocada en describir el funcionamiento del simulador y su arquitectura. Para lograr la accesibilidad que esta herramienta requiere de cara a usuarios inexpertos, el simulador ha sido implementado siguiendo una arquitectura web, donde solamente se requiere un navegador web para ejecutarlo. Además, para atraer y retar al operador, se han introducido algunos elementos de gamificación, que acercan este simulador a una experiencia propia del mundo de los videojuegos. La segunda parte del trabajo se basa en el simulador desarrollado para llevar a cabo varios experimentos con usuarios inexpertos. Se ha diseñado un conjunto de métricas de rendimiento con las cuales se de fine el perfil de un usuario. Usando estos perfiles, se ejecutan y validan algoritmos de clustering para obtener grupos de usuarios con perfiles de rendimiento comunes. Los resultados se analizan de cara a extraer patrones de comportamiento que distingan a los diferentes usuarios del experimento, permitiendo la identificación y selección de operadores expertos potenciales

Context-Enabled Visualization Strategies for Automation Enabled Human-in-the-loop Inspection Systems to Enhance the Situation Awareness of Windstorm Risk Engineers

Insurance loss prevention survey, specifically windstorm risk inspection survey is the process of investigating potential damages associated with a building or structure in the event of an extreme weather condition such as a hurricane or tornado. Traditionally, the risk inspection process is highly subjective and depends on the skills of the engineer performing it. This dissertation investigates the sensemaking process of risk engineers while performing risk inspection with special focus on various factors influencing it. This research then investigates how context-based visualizations strategies enhance the situation awareness and performance of windstorm risk engineers. An initial study investigated the sensemaking process and situation awareness requirements of the windstorm risk engineers. The data frame theory of sensemaking was used as the framework to carry out this study. Ten windstorm risk engineers were interviewed, and the data collected were analyzed following an inductive thematic approach. The themes emerged from the data explained the sensemaking process of risk engineers, the process of making sense of contradicting information, importance of their experience level, internal and external biases influencing the inspection process, difficulty developing mental models, and potential technology interventions. More recently human in the loop systems such as drones have been used to improve the efficiency of windstorm risk inspection. This study provides recommendations to guide the design of such systems to support the sensemaking process and situation awareness of windstorm visual risk inspection. The second study investigated the effect of context-based visualization strategies to enhance the situation awareness of the windstorm risk engineers. More specifically, the study investigated how different types of information contribute towards the three levels of situation awareness. Following a between subjects study design 65 civil/construction engineering students completed this study. A checklist based and predictive display based decision aids were tested and found to be effective in supporting the situation awareness requirements as well as performance of windstorm risk engineers. However, the predictive display only helped with certain tasks like understanding the interaction among different components on the rooftop. For remaining tasks, checklist alone was sufficient. Moreover, the decision aids did not place any additional cognitive demand on the participants. This study helped us understand the advantages and disadvantages of the decision aids tested. The final study evaluated the transfer of training effect of the checklist and predictive display based decision aids. After one week of the previous study, participants completed a follow-up study without any decision aids. The performance and situation awareness of participants in the checklist and predictive display group did not change significantly from first trial to second trial. However, the performance and situation awareness of participants in the control condition improved significantly in the second trial. They attributed this to their exposure to SAGAT questionnaire in the first study. They knew what issues to look for and what tasks need to be completed in the simulation. The confounding effect of SAGAT questionnaires needs to be studied in future research efforts

The Effects of Commercial Video Game Playing: A Comparison of Skills and Abilities for the Predator UAV

Currently, Predator unmanned aerial vehicles (UAV) are operated by pilots and navigators experienced with manned combat aircraft. With a projected increase in UAVs, more combat pilots will be needed to operate these aircraft. Yet, if the current operational tempo continues, the supply of combat pilots may not be able to meet the demand. Perhaps alternative pools of Air Force personnel could be considered for UAV duty to meet operational requirements. Because the Predator UAV is a software-driven aircraft, video game players (VGPs) already possess and use many skills that may be similar to those of Predator UAV pilots. A variety of games can add situational awareness skills that a player/airman can bring to a new situation. This research examines the applicability of video-games-based skills to the operation of the Predator UAV. Nine people were interviewed to determine the overlap between piloting skills, UAV-specific skills, and skills gained and developed from gaming. The results indicate that frequent VGPs have the confidence and the consistent ability to obtain and retain new skills, many of which are related to operating the Predator UAV in a 2-D environment while not relying on the visual and nonvisual cues of the manned aircraft pilot

The Next Generation of Human-Drone Partnerships: Co-Designing an Emergency Response System

The use of semi-autonomous Unmanned Aerial Vehicles (UAV) to support emergency response scenarios, such as fire surveillance and search and rescue, offers the potential for huge societal benefits. However, designing an effective solution in this complex domain represents a "wicked design" problem, requiring a careful balance between trade-offs associated with drone autonomy versus human control, mission functionality versus safety, and the diverse needs of different stakeholders. This paper focuses on designing for situational awareness (SA) using a scenario-driven, participatory design process. We developed SA cards describing six common design-problems, known as SA demons, and three new demons of importance to our domain. We then used these SA cards to equip domain experts with SA knowledge so that they could more fully engage in the design process. We designed a potentially reusable solution for achieving SA in multi-stakeholder, multi-UAV, emergency response applications.Comment: 10 Pages, 5 Figures, 2 Tables. This article is publishing in CHI202

Geospatial technologies for physical planning: Bridging the gap between earth science and planning

The application of geospatial information technologies has increased recently due to increase in data sources from the earth sciences. The systematic data collection, storage and processing together with data transformation require geospatial information technologies. Rapidly developing computer technology has become an effective tool in design and physical planning in international platforms. Especially, the availability of geospatial information technologies (remote sensing, GIS, spatial models and GPS) for diverse disciplines and the capability of these technologies in data conversion from two dimensions to the three dimensions provide great efficiency. Thus, this study explores how digital technologies are reshaping physical planning and design. While the potential of digital technologies is well documented within physical planning and visualization, its application within practice is far less understood. This paper highlights the role of the geospatial information technologies in encouraging a new planning and design logic that moves from the privileging of the visual to a focus on processes of formation, bridging the interface of the earth science and physical planning