Operator State Estimation for Adaptive Aiding in Uninhabited Combat Air Vehicles

This research demonstrated the first closed-loop implementation of adaptive automation using operator functional state in an operationally relevant environment. In the Uninhabited Combat Air Vehicle (UCAV) environment, operators can become cognitively overloaded and their performance may decrease during mission critical events. This research demonstrates an unprecedented closed-loop system, one that adaptively aids UCAV operators based on their cognitive functional state A series of experiments were conducted to 1) determine the best classifiers for estimating operator functional state, 2) determine if physiological measures can be used to develop multiple cognitive models based on information processing demands and task type, 3) determine the salient psychophysiological measures in operator functional state, and 4) demonstrate the benefits of intelligent adaptive aiding using operator functional state. Aiding the operator actually improved performance and increased mission effectiveness by 67%

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 359)

This bibliography lists 164 reports, articles and other documents introduced into the NASA Scientific and Technical Information System during Jan. 1992. Subject coverage includes: aerospace medicine and physiology, life support systems and man/system technology, protective clothing, exobiology and extraterrestrial life, planetary biology, and flight crew behavior and performance

Passive BCI in operational environments: insights, recent advances and future trends

this mini-review aims to highlight recent important aspects to consider and evaluate when passive Brain-Computer Interface (pBCI) systems would be developed and used in operational environments, and remarks future directions of their applications

Breaking Down the Barriers To Operator Workload Estimation: Advancing Algorithmic Handling of Temporal Non-Stationarity and Cross-Participant Differences for EEG Analysis Using Deep Learning

This research focuses on two barriers to using EEG data for workload assessment: day-to-day variability, and cross- participant applicability. Several signal processing techniques and deep learning approaches are evaluated in multi-task environments. These methods account for temporal, spatial, and frequential data dependencies. Variance of frequency- domain power distributions for cross-day workload classification is statistically significant. Skewness and kurtosis are not significant in an environment absent workload transitions, but are salient with transitions present. LSTMs improve day- to-day feature stationarity, decreasing error by 59% compared to previous best results. A multi-path convolutional recurrent model using bi-directional, residual recurrent layers significantly increases predictive accuracy and decreases cross-participant variance. Deep learning regression approaches are applied to a multi-task environment with workload transitions. Accounting for temporal dependence significantly reduces error and increases correlation compared to baselines. Visualization techniques for LSTM feature saliency are developed to understand EEG analysis model biases

Human Factors and Neurophysiological Metrics in Air Traffic Control: a Critical Review

International audienceThis article provides the reader a focused and organised review of the research progresses on neurophysiological indicators, also called “neurometrics”, to show how neurometrics could effectively address some of the most important Human Factors (HFs) needs in the Air Traffic Management (ATM) field. The state of the art on the most involved HFs and related cognitive processes (e.g. mental workload, cognitive training) is presented together with examples of possible applications in the current and future ATM scenarios, in order to better understand and highlight the available opportunities of such neuroscientific applications. Furthermore, the paper will discuss the potential enhancement that further research and development activities could bring to the efficiency and safety of the ATM service

Pilot workload and fatigue: A critical survey of concepts and assessment techniques

The principal unresolved issues in conceptualizing and measuring pilot workload and fatigue are discussed. These issues are seen as limiting the development of more useful working concepts and techniques and their application to systems engineering and management activities. A conceptual analysis of pilot workload and fatigue, an overview and critique of approaches to the assessment of these phenomena, and a discussion of current trends in the management of unwanted workload and fatigue effects are presented. Refinements and innovations in assessment methods are recommended for enhancing the practical significance of workload and fatigue studies

A new perspective for the training assessment: Machine learning-based neurometric for augmented user's evaluation

Inappropriate training assessment might have either high social costs and economic impacts, especially in high risks categories, such as Pilots, Air Traffic Controllers, or Surgeons. One of the current limitations of the standard training assessment procedures is the lack of information about the amount of cognitive resources requested by the user for the correct execution of the proposed task. In fact, even if the task is accomplished achieving the maximum performance, by the standard training assessment methods, it would not be possible to gather and evaluate information about cognitive resources available for dealing with unexpected events or emergency conditions. Therefore, a metric based on the brain activity (neurometric) able to provide the Instructor such a kind of information should be very important. As a first step in this direction, the Electroencephalogram (EEG) and the performance of 10 participants were collected along a training period of 3 weeks, while learning the execution of a new task. Specific indexes have been estimated from the behavioral and EEG signal to objectively assess the users' training progress. Furthermore, we proposed a neurometric based on a machine learning algorithm to quantify the user's training level within each session by considering the level of task execution, and both the behavioral and cognitive stabilities between consecutive sessions. The results demonstrated that the proposed methodology and neurometric could quantify and track the users' progresses, and provide the Instructor information for a more objective evaluation and better tailoring of training programs. © 2017 Borghini, Aricò, Di Flumeri, Sciaraffa, Colosimo, Herrero, Bezerianos, Thakor and Babiloni

Using near infrared spectroscopy and heart rate variability to detect mental overload

Mental workload is a key factor influencing the occurrence of human error, especially during piloting and remotely operated vehicle (ROV) operations, where safety depends on the ability of pilots to act appropriately. In particular, excessively high or low mental workload can lead operators to neglect critical information. The objective of the present study is to investigate the potential of functional Near Infrared Spectroscopy (fNIRS) – a non-invasive method of measuring prefrontal cortex activity – in combination with measurements of heart rate variability (HRV), to predict mental workload during a simulated piloting task, with particular regard to task engagement and disengagement. Twelve volunteers performed a computer-based piloting task in which they were asked to follow a dynamic target with their aircraft, a task designed to replicate key cognitive demands associated with real life ROV operating tasks. In order to cover a wide range of mental workload levels, task difficulty was manipulated in terms of processing load and difficulty of control – two critical sources of workload associated with piloting and remotely operating a vehicle. Results show that both fNIRS and HRV are sensitive to different levels of mental workload; notably, lower prefrontal activation as well as a lower LF/HF ratio at the highest level of difficulty, suggest that these measures are suitable for mental overload detection. Moreover, these latter measurements point towards the existence of a quadratic model of mental workload

Technical approaches for measurement of human errors

Human error is a significant contributing factor in a very high proportion of civil transport, general aviation, and rotorcraft accidents. The technical details of a variety of proven approaches for the measurement of human errors in the context of the national airspace system are presented. Unobtrusive measurements suitable for cockpit operations and procedures in part of full mission simulation are emphasized. Procedure, system performance, and human operator centered measurements are discussed as they apply to the manual control, communication, supervisory, and monitoring tasks which are relevant to aviation operations

Selection of Psychophysiological Features across Subjects for Classifying Workload Using Artificial Neural Networks

The issue of pilot workload is important to the United States Air Force because pilot overload or task saturation leads to decreases in mission effectiveness. Additionally, in the most extreme cases, pilot overload may lead to the loss of aircraft and crewmember lives. Current research efforts are utilizing psychophysiological data including electroencephalography (EEG), cardiac, eye-blink, and respiration measures in an attempt to identify workload levels. The primary focus of this effort is to determine if a single parsimonious set of psychophysiological features exists for accurately classifying workload levels between multiple test subjects. To accomplish this objective, the signal-to-noise (SNR) saliency measure is used to determine the usefulness of psychophysiological features in feedforward artificial neural networks (ANN). The SNR saliency measure determines the saliency, or relative value, of a feature by comparing it to a feature of injected noise. For this effort, 36 psychophysiological features were derived from the data collected as each subject completed simulated crewmember tasks using the Multi-Attribute Task Battery developed by NASA. These tasks were randomly presented to the subjects in blocks with three distinct levels: low, medium, and an overload level in which subjects could not complete all tasks