Breaking Down the Barriers To Operator Workload Estimation: Advancing Algorithmic Handling of Temporal Non-Stationarity and Cross-Participant Differences for EEG Analysis Using Deep Learning

This research focuses on two barriers to using EEG data for workload assessment: day-to-day variability, and cross- participant applicability. Several signal processing techniques and deep learning approaches are evaluated in multi-task environments. These methods account for temporal, spatial, and frequential data dependencies. Variance of frequency- domain power distributions for cross-day workload classification is statistically significant. Skewness and kurtosis are not significant in an environment absent workload transitions, but are salient with transitions present. LSTMs improve day- to-day feature stationarity, decreasing error by 59% compared to previous best results. A multi-path convolutional recurrent model using bi-directional, residual recurrent layers significantly increases predictive accuracy and decreases cross-participant variance. Deep learning regression approaches are applied to a multi-task environment with workload transitions. Accounting for temporal dependence significantly reduces error and increases correlation compared to baselines. Visualization techniques for LSTM feature saliency are developed to understand EEG analysis model biases

Development, Testing, and Validation of a Model-Based Tool to Predict Operator Responses in Unexpected Workload Transitions

One human factors challenge is predicting operator performance in novel situations. Approaches such as drawing on relevant previous experience, and developing computational models to predict operator performance in complex situations, offer potential methods to address this challenge. A few concerns with modeling operator performance are that models need to realistic, and they need to be tested empirically and validated. In addition, many existing human performance modeling tools are complex and require that an analyst gain significant experience to be able to develop models for meaningful data collection. This paper describes an effort to address these challenges by developing an easy to use model-based tool, using models that were developed from a review of existing human performance literature and targeted experimental studies, and performing an empirical validation of key model predictions

Vo(2) transitional response to a crossover from priming exercise

The question regarding oxygen uptake kinetics centers on the rate-limiting step. This study was designed to observe the oxygen uptake response that occurs between a crossover of modes of priming exercises. Participants completed three exercise trials. Trial 1 involved cycling from rest to a target workload, Triat 2 entailed cycling from rest to light and then to the target workload, and Trial 3 was from rest to stepping followed by cycling at the target workload. Transitions from rest had similar half-time (1/2 t) values. Transitions that occurred after a priming exercise produced longer 1/2 t to steady state regardless of the mode of exercise: cycling from low to target workload =62 seconds, cycling after stepping =76 seconds. This data suggests that when oxygen uptake kinetics is concerned, exercise transitions from rest are more efficient than transitions from a warmed up state regardless of the mode of priming exercise

Creatine kinase in energy metabolic signaling in muscle

There has been much debate on the mechanism of regulation of mitochondrial ATP synthesis to balance ATP consumption during changing cardiac workloads. A key role of creatine kinase (CK) isoenzymes in this regulation of oxidative phosphorylation and in intracellular energy transport had been proposed, but has in the mean time been disputed for many years. It was hypothesized that high-energy phosphoryl groups are obligatorily transferred via CK; this is termed the phosphocreatine shuttle. The other important role ascribed to the CK system is its ability to buffer ADP concentration in cytosol near sites of ATP hydrolysis. 

Almost all of the experiments to determine the role of CK had been done in the steady state, but recently the dynamic response of oxidative phosphorylation to quick changes in
cytosolic ATP hydrolysis has been assessed at various levels of inhibition of CK. Steady state models of CK function in energy transfer existed but were unable to explain the dynamic response with CK inhibited.

The aim of this study was to explain the mode of functioning of the CK system in heart, and in particular the role of different CK isoenzymes in the dynamic response to workload steps. For this purpose we used a mathematical model of cardiac muscle cell energy metabolism containing the kinetics of the key processes of energy production, consumption and transfer pathways. The model underscores that CK plays indeed a dual role in the cardiac cells. The buffering role of CK system is due to the activity of myofibrillar CK (MMCK) while the energy transfer role depends on the activity of mitochondrial CK (MiCK). We propose that this may lead to the differences in regulation mechanisms and energy transfer modes in species with relatively low MiCK activity such as rabbit in comparison with species with high MiCK activity such as rat.

The model needed modification to explain the new type of experimental data on the dynamic response of the mitochondria. We submit that building a Virtual Muscle Cell is not possible without continuous experimental tests to improve the model. In close interaction with experiments we are developing a model for muscle energy metabolism and transport mediated by the creatine kinase isoforms which now already can explain many different types of experiments

Deep Long Short-term Memory Structures Model Temporal Dependencies Improving Cognitive Workload Estimation

Using deeply recurrent neural networks to account for temporal dependence in electroencephalograph (EEG)-based workload estimation is shown to considerably improve day-to-day feature stationarity resulting in significantly higher accuracy (p \u3c .0001) than classifiers which do not consider the temporal dependence encoded within the EEG time-series signal. This improvement is demonstrated by training several deep Recurrent Neural Network (RNN) models including Long Short-Term Memory (LSTM) architectures, a feedforward Artificial Neural Network (ANN), and Support Vector Machine (SVM) models on data from six participants who each perform several Multi-Attribute Task Battery (MATB) sessions on five separate days spread out over a month-long period. Each participant-specific classifier is trained on the first four days of data and tested using the fifth’s. Average classification accuracy of 93.0% is achieved using a deep LSTM architecture. These results represent a 59% decrease in error compared to the best previously published results for this dataset. This study additionally evaluates the significance of new features: all combinations of mean, variance, skewness, and kurtosis of EEG frequency-domain power distributions. Mean and variance are statistically significant features, while skewness and kurtosis are not. The overall performance of this approach is high enough to warrant evaluation for inclusion in operational systems

Hysteresis Effects In Driving

This dissertation presents two studies examining the interaction between workload history and driver mental workload. The first experiment focuses on testing for the presence of a hysteresis effect in the driving task. The second experiment examines the proposition that cueing impending periods of higher task demand can reduce the impact of any such potential hysteresis effects. Thirty-two licensed drivers served as participants and all served in both studies. Using the directions provided by a Heads-Up-Display navigation system, participants followed a pre-set route in the simulated environment. At specified points within the drive, the navigation system would purposefully fail which required drivers to relay a ten digit alphanumeric error code to a remote operator in order to reset the system. Results indicated that this increase in task demand from the navigation system\u27s failure leads to a significant increase in perceived mental workload as compared to pre-failure periods. This increase in driver mental workload was not significantly reduced by the time the drive ended, indicating the presence of a hysteresis effect. In the second experiment, the navigation system provided a completely reliable visual warning before failure. Results indicate that cueing had neither an effect on perceived mental workload, nor any ameliorating effect on the hysteretic type effect seen in mental workload recovery. The conclusion of these findings being that the overall safety and efficiency of the surface transportation system would likely improve by designs which accommodate the periods immediately following a reduction in stress. Whether from leaving high demand areas such as work zones or in the period immediately after using a in-car information device such as a GPS or a cell phone, these post-high workload periods are associated with increased variability in driver inputs and levels of mental workload

Leveraging Initial Cognitive Load to Predict User Response to Complex Visual Tasks

In this study, we were able to show how cognitive load measurement during the initiation of a complex visual task can predict user response. We measured cognitive load using pupil size and microsaccade rate. The initial phase of task was defined as the first 25 percent of the trial Reaction Time (RT), which was variable up to 50 seconds. The complex visual task entailed a set of twelve words that could be grouped based into 1, 2, or 3 categories or sets, e.g., the words bed, pillow, headboard, etc. can be grouped into a single set that is bedroom. We found a significant correlation between initial cognitive load and final user response to the task. This study provides new insights into the initial cognitive processes that would have practical applications in adaptive user interface design, early warning controls, and detection in human performance

A study of mitochondrial redox state in cardiac muscle

This thesis describes the use of intrinsic fluorescence measurements as a means for examining mitochondrial function in different cardiac preparations and phenotypes. Cardiac myocytes are intrinsically fluorescent and spectroscopic analysis of rabbit ventricular myocytes indicated that the majority of this fluorescence arises from the metabolic coenzymes nicotinamide adenine dinucleotide in the reduced state (NADH) and flavin adenine dinucleotide (FAD) in the oxidised state. Calibration of the NADH and FAD fluorescence signal with the mitochondrial inhibitors sodium cyanide (NaCN) and carbonyl cyanide p- (trifluoromethoxy) phenylhydrazone (FCCP) enabled calculation of mitochondrial redox states. Redox measurements reflect the balance between reduced and oxidised forms of the NAD and FAD pools and provide an index for assessing mitochondrial function in cells and tissue. The major advantage of this technique is that the intrinsically fluorescent nature of these metabolites obviates the need for exogenous indicators of mitochondrial function, which can themselves influence mitochondrial behaviour. Mitochondrial redox state was established using a variety of fluorescence techniques. Values for NADstate represent the proportion of the NADH/NAD+ redox couple in the reduced state. Calculation of NADstate using single photon, two photon and wide-field epifluorescence microscopy revealed very similar values ranging from 0.57±0.18 to 0.59±0.17 (mean±SD). FAD fluorescence measurements were used to establish FADstate (the proportion of the FADH2/FAD redox couple in the oxidised state). However, FAD fluorescence could only be detected by epifluorescence and single photon excitation fluorescence microscopy. Once again, comparable values of 0.17±0.10 and 0.18±0.07 respectively were obtained, thus demonstrating the reproducibility of the technique. Attempts were made to perform these measurements in intact cardiac tissue preparations. However, difficulties encountered with the delivery of mitochondrial inhibitor to specific areas of tissue and problems with inner filter effects complicating the interpretation of fluorescence recordings meant that this was not possible. Measurements of intrinsic fluorescence were utilised in order to assess the mitochondrial redox response of cardiac cells to increased energy demand. Isolated rabbit ventricular myocytes were field stimulated and fractional shortening was simultaneously recorded with epifluorescence measurements of NADH and FAD. Cells were paced at 0.5Hz and the stimulation frequency step increased to 1Hz, 2Hz and 3Hz in order to increase work intensity and energy demand. Step increasing stimulation frequency resulted in a decrease in NADH fluorescence and an increase in FAD fluorescence before reaching an essentially steady state. This indicated oxidation of the cell environment, suggesting a transient mismatch between metabolite supply and demand. The magnitude of this response was related to stimulation frequency, with the biggest responses taking place at the highest work intensity. Reducing work intensity back to 0.5Hz pacing resulted in immediate recovery of metabolite fluorescence. Investigation into the redox response to increased work intensity in the stroke prone spontaneously hypertensive rat (SHRSP) model of cardiac hypertrophy found that energy supply and demand matching was in fact improved in these cardiomyocytes compared to Wistar Kyoto (WKY) control myocytes. Work intensity was increased from 1Hz to 2, 4 and 6Hz pacing and the oxidative response to increased workload was found to be significantly less in SHRSP cardiomyocytes compared to WKY myocytes (p<0.01). This was despite similar levels of contractile work being performed by the two groups and may be related to the young age of the animals (16 weeks). At this age, hypertrophy of the SHRSP hearts is likely to still be in the compensated state and mitochondrial function may indeed be improved rather than detrimentally affected at this stage

Factors Affecting Air Traffic Controller’s Weather Dissemination to Pilots

As the number of flights in the United States continues to rise steadily, an equally amplified need for reliability and safety has come to the forefront of aviation research. One of the most alarming trends is the number of general aviation (GA) accidents during severe weather events that occur yearly, with fatalities occurring in more than half of these cases. This study focuses on identifying factors influencing weather dissemination of Air Traffic Controllers (ATC) to GA pilots. Ten factors affecting controllers’ performance during severe weather events were identified through an in-depth literature review including controller mental workload, situation awareness, weather information format and accuracy, weather information needs, weather tool limitations, inaccurate assumption and bias, controller training and expereince, regulatory factor, supervisory factors, and pilot-controller relationship. Recommendation can be developed to address each factors so that aviation safety could be enhanced in severe weather situations

An Eye for the Air Traffic Controller Workload

The purpose of this paper is to outline an approach for workload measurements and optimization of air traffic systems and displays that match controller needs. Ill-designed systems and displays can cause safety hazards for aircraft by increasing controller workload and reducing situation awareness. To prevent this situation, researchers need to develop systems that allow effortless monitoring while being attentive to operator needs. Such systems, once developed, will increase operator and system efficiency and increase the safety of airline operations