Simulation-based Cognitive Workload Modeling And Evaluation Of Adaptive Automation Invoking And Revoking Strategies

In human-computer systems, such as supervisory control systems, large volumes of incoming and complex information can degrade overall system performance. Strategically integrating automation to offload tasks from the operator has been shown to increase not only human performance but also operator efficiency and safety. However, increased automation allows for increased task complexity, which can lead to high cognitive workload and degradation of situational awareness. Adaptive automation is one potential solution to resolve these issues, while maintaining the benefits of traditional automation. Adaptive automation occurs dynamically, with the quantity of automated tasks changing in real-time to meet performance or workload goals. While numerous studies evaluate the relative performance of manual and adaptive systems, little attention has focused on the implications of selecting particular invoking or revoking strategies for adaptive automation. Thus, evaluations of adaptive systems tend to focus on the relative performance among multiple systems rather than the relative performance within a system. This study takes an intra-system approach specifically evaluating the relationship between cognitive workload and situational awareness that occurs when selecting a particular invoking-revoking strategy for an adaptive system. The case scenario is a human supervisory control situation that involves a system operator who receives and interprets intelligence outputs from multiple unmanned assets, and then identifies and reports potential threats and changes in the environment. In order to investigate this relationship between workload and situational awareness, discrete event simulation (DES) is used. DES is a standard technique in the analysis iv of systems, and the advantage of using DES to explore this relationship is that it can represent a human-computer system as the state of the system evolves over time. Furthermore, and most importantly, a well-designed DES model can represent the human operators, the tasks to be performed, and the cognitive demands placed on the operators. In addition to evaluating the cognitive workload to situational awareness tradeoff, this research demonstrates that DES can quite effectively model and predict human cognitive workload, specifically for system evaluation. This research finds that the predicted workload of the DES models highly correlates with well-established subjective measures and is more predictive of cognitive workload than numerous physiological measures. This research then uses the validated DES models to explore and predict the cognitive workload impacts of adaptive automation through various invoking and revoking strategies. The study provides insights into the workload-situational awareness tradeoffs that occur when selecting particular invoking and revoking strategies. First, in order to establish an appropriate target workload range, it is necessary to account for both performance goals and the portion of the workload-performance curve for the task in question. Second, establishing an invoking threshold may require a tradeoff between workload and situational awareness, which is influenced by the task’s location on the workload-situational awareness continuum. Finally, this study finds that revoking strategies differ in their ability to achieve workload and situational awareness goals. For the case scenario examined, revoking strategies based on duration are best suited to improve workload, while revoking strategies based on revoking thresholds are better for maintaining situational awareness

The Underpinnings of Workload in Unmanned Vehicle Systems

This paper identifies and characterizes factors that contribute to operator workload in unmanned vehicle systems. Our objective is to provide a basis for developing models of workload for use in design and operation of complex human-machine systems. In 1986, Hart developed a foundational conceptual model of workload, which formed the basis for arguably the most widely used workload measurement techniquethe NASA Task Load Index. Since that time, however, there have been many advances in models and factor identification as well as workload control measures. Additionally, there is a need to further inventory and describe factors that contribute to human workload in light of technological advances, including automation and autonomy. Thus, we propose a conceptual framework for the workload construct and present a taxonomy of factors that can contribute to operator workload. These factors, referred to as workload drivers, are associated with a variety of system elements including the environment, task, equipment and operator. In addition, we discuss how workload moderators, such as automation and interface design, can be manipulated in order to influence operator workload. We contend that workload drivers, workload moderators, and the interactions among drivers and moderators all need to be accounted for when building complex, human-machine systems

Blink counts can differentiate between task type and load

Physiological measures have been increasing in popularity due to the growing availability of equipment that allows measurement in real time. Eye blinks are an easy measure to collect using video capture. Our findings indicate that blink counts effectively differentiate between taskloads and task types during a computer based task. Blink counts were significantly lower during the tasks involving high visual load when compared to non-visually demanding tasks. Lower numbers of blinks were observed under higher taskloads across all visual tasks. Paper originally presented international conference on Ergonomics & Human Factors, held 25 - 27 April 2017, Staverton Estate, Daventry, Northamptonshire

Comparing Types Of Adaptive Automation Within A Multi-tasking Environment

Throughout the many years of research examining the various effects of automation on operator performance, stress, workload, etc., the focus has traditionally been on the level of automation, and the invocation methods used to alter it. The goal of the current study is to instead examine the utilization of various types of automation with the goal of better meeting the operator’s cognitive needs, thus improving their performance, workload, and stress. The task, control of a simulated unmanned robotic system, is designed to specifically stress the operator’s visual perception capabilities to a greater degree. Two types of automation are implemented to support the operator’s performance of the task: an auditory beep aid intended to support visual perception resources, and a driving aid automating control of the vehicle’s navigation, offloading physical action execution resources. Therefore, a comparison can be made between types of automation intended to specifically support the mental dimension that is under the greatest demand (the auditory beep) against those that do not (the driving automation). An additional evaluation is made to determine the benefit of adaptively adjusting the level of each type of automation based on the current level of task demand, as well as the influence of individual differences in personality. Results indicate that the use of the auditory beep aid does improve performance, but also increases Temporal Demand and Effort. Use of driving automation appears to disengage the operator from the task, eliciting a vigilance response. Adaptively altering the level of automation to meet task demands has a mixed effect on performance and workload (reducing both) when the auditory beep automation is used. However, adaptive driving automation is clearly detrimental, iv causing an increase in workload while decreasing performance. Higher levels of Neuroticism are related to poorer threat detection performance, but personality differences show no indication of moderating the effects of either of the experimental manipulations. The results of this study show that the type of automation implemented within an environment has a considerable impact on the operator, in terms of performance as well as cognitive/emotional stat

MODEL-BASED ASSESSMENT OF ADAPTIVE AUTOMATION’S UNINTENDED CONSEQUENCES

Recent technological advances require development of human-centered principles for their inclusion into complex systems. While such programs incorporate revolutionary hardware and software advances, there is a necessary space for including human operator design considerations, such as cognitive workload. As technologies mature, it is essential to understand the impacts that these emerging systems will have on cognitive workload. Adaptive automation is a solution that seeks to manage cognitive workload at optimal levels. Human performance modeling shows potential for modeling the effects of adaptive automation on cognitive workload. However, the introduction of adaptive automation into a system can also present unintended negative consequences to an operator. This dissertation investigated potential negative unintended consequences of adaptive automation through the development of human performance models of a multi-tasking simulation. One hundred twenty participants were enrolled in three human-in-the-loop experimental studies (forty participants each) that collected objective and subjective surrogate measures of cognitive workload to validate the models. Results from this research indicate that there are residual increases in operator workload after transitions in system states between manual and automatic control of a task that need to be included in human performance models and in system design considerations.Approved for public release. Distribution is unlimited.Lieutenant Colonel, United States ArmyCommanding Officer, U.S. Army Combat Capabilities Development Command, Aviation and Missile Center Agency, Redstone Arsenal, Alabama 35898-500

A review of important cognitive concepts in aviation

Even considering the current low accident rate in aviation, the anticipated growth in the number of airplanes in the air in the next decades will lead to an inadmissible rise in the number of accidents. These have been mostly attributed to human error and a misunderstanding of automation by the crew, especially during periods of high workload and stress in the cockpit. Therefore, increased safety requires not only advances in technology, but improved cockpit design including better human-machine interface. These cannot be achieved however, without considering some of the cognitive constructs that affect the behaviour of pilots in the cockpit. In fact, given its characteristics and public visibility, the flight deck of commercial jets is one of the most common arenas for the study of complex and skilled human performance. Here I present a literature review on the selected topics of workload, situation awareness, stress and automation in the cockpit, with the goal of supporting the development of new technologies

Defining brain–machine interface applications by matching interface performance with device requirements

Interaction with machines is mediated by human-machine interfaces (HMIs). Brain-machine interfaces (BMIs) are a particular class of HMIs and have so far been studied as a communication means for people who have little or no voluntary control of muscle activity. In this context, low-performing interfaces can be considered as prosthetic applications. On the other hand, for able-bodied users, a BMI would only be practical if conceived as an augmenting interface. In this paper, a method is introduced for pointing out effective combinations of interfaces and devices for creating real-world applications. First, devices for domotics, rehabilitation and assistive robotics, and their requirements, in terms of throughput and latency, are described. Second, HMIs are classified and their performance described, still in terms of throughput and latency. Then device requirements are matched with performance of available interfaces. Simple rehabilitation and domotics devices can be easily controlled by means of BMI technology. Prosthetic hands and wheelchairs are suitable applications but do not attain optimal interactivity. Regarding humanoid robotics, the head and the trunk can be controlled by means of BMIs, while other parts require too much throughput. Robotic arms, which have been controlled by means of cortical invasive interfaces in animal studies, could be the next frontier for non-invasive BMIs. Combining smart controllers with BMIs could improve interactivity and boost BMI applications. © 2007 Elsevier B.V. All rights reserved

Supporting Quality of Service in Scientific Workflows

While workflow management systems have been utilized in enterprises to support businesses for almost two decades, the use of workflows in scientific environments was fairly uncommon until recently. Nowadays, scientists use workflow systems to conduct scientific experiments, simulations, and distributed computations. However, most scientific workflow management systems have not been built using existing workflow technology; rather they have been designed and developed from scratch. Due to the lack of generality of early scientific workflow systems, many domain-specific workflow systems have been developed. Generally speaking, those domain-specific approaches lack common acceptance and tool support and offer lower robustness compared to business workflow systems. In this thesis, the use of the industry standard BPEL, a workflow language for modeling business processes, is proposed for the modeling and the execution of scientific workflows. Due to the widespread use of BPEL in enterprises, a number of stable and mature software products exist. The language is expressive (Turingcomplete) and not restricted to specific applications. BPEL is well suited for the modeling of scientific workflows, but existing implementations of the standard lack important features that are necessary for the execution of scientific workflows. This work presents components that extend an existing implementation of the BPEL standard and eliminate the identified weaknesses. The components thus provide the technical basis for use of BPEL in academia. The particular focus is on so-called non-functional (Quality of Service) requirements. These requirements include scalability, reliability (fault tolerance), data security, and cost (of executing a workflow). From a technical perspective, the workflow system must be able to interface with the middleware systems that are commonly used by the scientific workflow community to allow access to heterogeneous, distributed resources (especially Grid and Cloud resources). The major components cover exactly these requirements: Cloud Resource Provisioner Scalability of the workflow system is achieved by automatically adding additional (Cloud) resources to the workflow system’s resource pool when the workflow system is heavily loaded. Fault Tolerance Module High reliability is achieved via continuous monitoring of workflow execution and corrective interventions, such as re-execution of a failed workflow step or replacement of the faulty resource. Cost Aware Data Flow Aware Scheduler The majority of scientific workflow systems only take the performance and utilization of resources for the execution of workflow steps into account when making scheduling decisions. The presented workflow system goes beyond that. By defining preference values for the weighting of costs and the anticipated workflow execution time, workflow users may influence the resource selection process. The developed multiobjective scheduling algorithm respects the defined weighting and makes both efficient and advantageous decisions using a heuristic approach. Security Extensions Because it supports various encryption, signature and authentication mechanisms (e.g., Grid Security Infrastructure), the workflow system guarantees data security in the transfer of workflow data. Furthermore, this work identifies the need to equip workflow developers with workflow modeling tools that can be used intuitively. This dissertation presents two modeling tools that support users with different needs. The first tool, DAVO (domain-adaptable, Visual BPEL Orchestrator), operates at a low level of abstraction and allows users with knowledge of BPEL to use the full extent of the language. DAVO is a software that offers extensibility and customizability for different application domains. These features are used in the implementation of the second tool, SimpleBPEL Composer. SimpleBPEL is aimed at users with little or no background in computer science and allows for quick and intuitive development of BPEL workflows based on predefined components

A component-based collaboration infrastructure

Groupware applications allow geographically distributed users to collaborate on shared tasks. However, it is widely recognized that groupware applications are expensive to build due to coordination services and group dynamics, neither of which is present in single-user applications. Previous collaboration transparency systems reuse existing single-user applications as a whole for collaborative work, often at the price of inflexible coordination. Previous collaboration awareness systems, on the other hand, provide reusable coordination services and multi-user widgets, but often with two weaknesses: (1) the multi-user widgets provided are special-purpose and limited in number, while no guidelines are provided for developing multi-user interface components in general; and (2) they often fail to reach the desired level of flexibility in coordination by tightly binding shared data and coordination services. In this dissertation, we propose a component-based approach to developing group- ware applications that addresses the above two problems. To address the first prob- lem, we propose a shared component model for modeling data and graphic user inter- face(GUI) components of groupware applications. As a result, the myriad of existing single-user components can be re-purposed as shared GUI or data components. An adaptation tool is developed to assist the adaptation process. To address the second problem, we propose a coordination service framework which systematically model the interaction between user, data, and coordination protocols. Due to the clean separation of data and control and the capability to dynamically "glue" them together, the framework provides reusable services such as data distribution, persistence, and adaptable consistency control. The association between data and coordination services can be dynamically changed at runtime. An Evolvable and eXtensible Environment for Collaboration (EXEC) is built to evaluate the proposed approach. In our experiments, we demonstrate two benefits of our approach: (1) a group of common groupware features adapted from existing single- user components are plugged in to extend the functionalities of the environment itself; and (2)coordination services can be dynamically attached to and detached from these shared components at different granules to support evolving collaboration needs

Analysis of Software Design Patterns in Human Cognitive Performance Experiments

As Air Force operations continue to move toward the use of more autonomous systems and more human-machine teaming in general, there is a corresponding need to swiftly evaluate systems with these capabilities. We support this development through software design improvements of the execution of human cognitive performance experiments. This thesis sought to answer the following two research questions addressing the core functionality that these experiments rely on for execution and analysis: 1) What data infrastructure software requirements are necessary to execute the experimental design of human cognitive performance experiments? 2) How effectively does a central data mediator design pattern meet the time-alignment requirements of human cognitive performance studies? To answer these questions, this research contributes an exploration of establishing design patterns to reduce the cost of conducting human cognitive performance studies. The activities included in this exploration were a method for requirements gathering, a meta-study of recent experiments, and a design pattern evaluation all focused on the experimental design domain