Selection of Psychophysiological Features across Subjects for Classifying Workload Using Artificial Neural Networks

The issue of pilot workload is important to the United States Air Force because pilot overload or task saturation leads to decreases in mission effectiveness. Additionally, in the most extreme cases, pilot overload may lead to the loss of aircraft and crewmember lives. Current research efforts are utilizing psychophysiological data including electroencephalography (EEG), cardiac, eye-blink, and respiration measures in an attempt to identify workload levels. The primary focus of this effort is to determine if a single parsimonious set of psychophysiological features exists for accurately classifying workload levels between multiple test subjects. To accomplish this objective, the signal-to-noise (SNR) saliency measure is used to determine the usefulness of psychophysiological features in feedforward artificial neural networks (ANN). The SNR saliency measure determines the saliency, or relative value, of a feature by comparing it to a feature of injected noise. For this effort, 36 psychophysiological features were derived from the data collected as each subject completed simulated crewmember tasks using the Multi-Attribute Task Battery developed by NASA. These tasks were randomly presented to the subjects in blocks with three distinct levels: low, medium, and an overload level in which subjects could not complete all tasks

Feature Fusion-Based Capsule Network for Cross-Subject Mental Workload Classification

In a complex human-computer interaction system, estimating mental workload based on electroencephalogram (EEG) plays a vital role in the system adaption in accordance with users’ mental state. Compared to within-subject classification, cross-subject classification is more challenging due to larger variation across subjects. In this paper, we targeted the cross-subject mental workload classification and attempted to improve the performance. A capsule network capturing structural relationships between features of power spectral density and brain connectivity was proposed. The comparison results showed that it achieved a cross-subject classification accuracy of 45.11%, which was superior to the compared methods (e.g., convolutional neural network and support vector machine). The results also demonstrated feature fusion positively contributed to the cross-subject workload classification. Our study could benefit the future development of a real-time workload detection system unspecific to subjects

Operator State Estimation for Adaptive Aiding in Uninhabited Combat Air Vehicles

This research demonstrated the first closed-loop implementation of adaptive automation using operator functional state in an operationally relevant environment. In the Uninhabited Combat Air Vehicle (UCAV) environment, operators can become cognitively overloaded and their performance may decrease during mission critical events. This research demonstrates an unprecedented closed-loop system, one that adaptively aids UCAV operators based on their cognitive functional state A series of experiments were conducted to 1) determine the best classifiers for estimating operator functional state, 2) determine if physiological measures can be used to develop multiple cognitive models based on information processing demands and task type, 3) determine the salient psychophysiological measures in operator functional state, and 4) demonstrate the benefits of intelligent adaptive aiding using operator functional state. Aiding the operator actually improved performance and increased mission effectiveness by 67%

Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks

Applying deep learning methods to electroencephalograph (EEG) data for cognitive state assessment has yielded improvements over previous modeling methods. However, research focused on cross-participant cognitive workload modeling using these techniques is underrepresented. We study the problem of cross-participant state estimation in a non-stimulus-locked task environment, where a trained model is used to make workload estimates on a new participant who is not represented in the training set. Using experimental data from the Multi-Attribute Task Battery (MATB) environment, a variety of deep neural network models are evaluated in the trade-space of computational efficiency, model accuracy, variance and temporal specificity yielding three important contributions: (1) The performance of ensembles of individually-trained models is statistically indistinguishable from group-trained methods at most sequence lengths. These ensembles can be trained for a fraction of the computational cost compared to group-trained methods and enable simpler model updates. (2) While increasing temporal sequence length improves mean accuracy, it is not sufficient to overcome distributional dissimilarities between individuals’ EEG data, as it results in statistically significant increases in cross-participant variance. (3) Compared to all other networks evaluated, a novel convolutional-recurrent model using multi-path subnetworks and bi-directional, residual recurrent layers resulted in statistically significant increases in predictive accuracy and decreases in cross-participant variance

Enhancing video game performance through an individualized biocybernetic system

Biocybernetic systems are physiological software systems that explicitly utilize physiological signals to control or adapt software functionality (Pope et al., 1995.) These systems have tremendous potential for innovation in human computer interaction by using physiological signals to infer a user\u27s emotional and mental states (Allanson & Fairclough, 2004; Fairclough, 2008). Nevertheless, development of these systems has been ultimately hindered by two fundamental challenges. First, these systems make generalizations about physiological indicators of cognitive states across populations when, in fact, relationships between physiological responses and cognitive states are specific to each individual (Andreassi, 2006). Second, they often employ largely inconsistent retrospective techniques to subjectively infer user\u27s mental state (Fairclough, 2008). An individualized biocybernetic system was developed to address the fundamental challenges of biocybernetic research. This system was used to adapt video game difficulty through real-time classifications of physiological responses to subjective appraisals. A study was conducted to determine the system\u27s ability to improve player\u27s performance. The results provide evidence of significant task performance increase and higher attained task difficulty when players interacted with the game using the system than without. This work offers researchers with an alternative method for software adaptation by conforming to the individual characteristics of each user

Aerospace Medicine and Biology

This bibliography lists 184 reports, articles and other documents introduced into the NASA Scientific and Technical Information System during October 1989. Subject coverage includes: aerospace medicine and psychology, life support systems and controlled environments, safety equipment, exobiology and extraterrestrial life, and flight crew behavior and performance

Johnson Space Center Research and Technology 1993 Annual Report

Johnson Space Center research and technology accomplishments during fiscal year 1993 are described and principle researchers and technologists are identified as contacts for further information. Each of the four sections gives a summary of overall progress in a major discipline, followed by detailed, illustrated descriptions of significant tasks. The four disciplines are Life Sciences, Human Support Technology, Solar Systems Sciences, and Space Systems Technology. The report is intended for technical and management audiences throughout the NASA and worldwide aerospace community. An index lists project titles, funding codes, and principal investigators

Human Fatigue Predictions in Complex Aviation Crew Operational Impact Conditions

In this last decade, several regulatory frameworks across the world in all modes of transportation had brought fatigue and its risk management in operations to the forefront. Of all transportation modes air travel has been the safest means of transportation. Still as part of continuous improvement efforts, regulators are insisting the operators to adopt strong fatigue science and its foundational principles to reinforce safety risk assessment and management. Fatigue risk management is a data driven system that finds a realistic balance between safety and productivity in an organization. This work discusses the effects of mathematical modeling of fatigue and its quantification in the context of fatigue risk management for complex global logistics operations. A new concept called Duty DNA is designed within the system that helps to predict and forecast sleep, duty deformations and fatigue. The need for a robust structure of elements to house the components to measure and manage fatigue risk in operations is also debated. By operating on the principles of fatigue management, new science-based predictive, proactive and reactive approaches were designed for an industry leading fatigue risk management program Accurately predicting sleep is very critical to predicting fatigue and alertness. Mathematical models are being developed to track the biological processes quantitatively and predicting temporal profile of fatigue given a person’s sleep history, planned work schedule including night and day exposure. As these models are being continuously worked to improve, a new limited deep learning machine learning based approach is attempted to predict fatigue for a duty in isolation without knowing much of work schedule history. The model within also predicts the duty disruptions and predicted fatigue at the end state of duty

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 359)

This bibliography lists 164 reports, articles and other documents introduced into the NASA Scientific and Technical Information System during Jan. 1992. Subject coverage includes: aerospace medicine and physiology, life support systems and man/system technology, protective clothing, exobiology and extraterrestrial life, planetary biology, and flight crew behavior and performance

Cognitive Decay And Memory Recall During Long Duration Spaceflight

This dissertation aims to advance the efficacy of Long-Duration Space Flight (LDSF) pre-flight and in-flight training programs, acknowledging existing knowledge gaps in NASA\u27s methodologies. The research\u27s objective is to optimize the cognitive workload of LDSF crew members, enhance their neurocognitive functionality, and provide more meaningful work experiences, particularly for Mars missions.The study addresses identified shortcomings in current training and learning strategies and simulation-based training systems, focusing on areas requiring quantitative measures for astronaut proficiency and training effectiveness assessment. The project centers on understanding cognitive decay and memory loss under LDSF-related stressors, seeking to establish when such cognitive decline exceeds acceptable performance levels throughout mission phases. The research acknowledges the limitations of creating a near-orbit environment due to resource constraints and the need to develop engaging tasks for test subjects. Nevertheless, it underscores the potential impact on future space mission training and other high-risk professions. The study further explores astronaut training complexities, the challenges encountered in LDSF missions, and the cognitive processes involved in such demanding environments. The research employs various cognitive and memory testing events, integrating neuroimaging techniques to understand cognition\u27s neural mechanisms and memory. It also explores Rasmussen\u27s S-R-K behaviors and Brain Network Theory’s (BNT) potential for measuring forgetting, cognition, and predicting training needs. The multidisciplinary approach of the study reinforces the importance of integrating insights from cognitive psychology, behavior analysis, and brain connectivity research. Research experiments were conducted at the University of North Dakota\u27s Integrated Lunar Mars Analog Habitat (ILMAH), gathering data from selected subjects via cognitive neuroscience tools and Electroencephalography (EEG) recordings to evaluate neurocognitive performance. The data analysis aimed to assess brain network activations during mentally demanding activities and compare EEG power spectra across various frequencies, latencies, and scalp locations. Despite facing certain challenges, including inadequacies of the current adapter boards leading to analysis failure, the study provides crucial lessons for future research endeavors. It highlights the need for swift adaptation, continual process refinement, and innovative solutions, like the redesign of adapter boards for high radio frequency noise environments, for the collection of high-quality EEG data. In conclusion, while the research did not reveal statistically significant differences between the experimental and control groups, it furnished valuable insights and underscored the need to optimize astronaut performance, well-being, and mission success. The study contributes to the ongoing evolution of training methodologies, with implications for future space exploration endeavors