Cross-participant and cross-task classification of cognitive load based on eye tracking

Cognitive load refers to the total amount of working memory resources a person is currently using. Successfully detecting the cognitive load a person is experiencing is the first important step towards applications that adapt to a user’s current load. Provided that cognitive load is estimated correctly, a system can enhance a user’s experience or increase its own efficiency by adapting to this detected load. Using digital learning environments as an example to illustrate this idea, a learning environment could tune the difficulty of presented exercises or learning material to match the learner’s current load to not underwhelm them, but also to prevent overload and frustration. Physiological sensors have great promise when cognitive load estimation is concerned as many physiological signals show distinctive signs of cognitive load. Eye tracking is an especially promising candidate as it does not require physical contact between sensor and user and is therefore very subtle. A major problem is the lack of general classifiers for cognitive load as classifiers are usually specific to a single person and do not generalize well. For adaptive interfaces based on a user’s cognitive load to be viable, a classifier that is accurate and performs well independently of user and specific task would be needed. In the current doctoral thesis, I present four studies that successively build upon each other and build up towards an eye-tracking based classifier for cognitive load that is 1) accurate, 2) robust, 3) can generalize, and 4) can operate in real-time. Each of the presented studies advances our approach’s capability to generalize one step further. Along the way, different eye-tracking features are explored and evaluated for their suitability as predictors of cognitive load and the implications for the distinction between cognitive load and perceptual load are discussed. The resulting method demonstrates a degree of generalization that no other approach has achieved and combines it with low hardware requirements and high robustness into a method that has great promise for future applications. Overall, the results presented in this thesis may serve as a foundation for the use of eye tracking in adaptive interfaces that react to a user’s cognitive load

Breaking Down the Barriers To Operator Workload Estimation: Advancing Algorithmic Handling of Temporal Non-Stationarity and Cross-Participant Differences for EEG Analysis Using Deep Learning

This research focuses on two barriers to using EEG data for workload assessment: day-to-day variability, and cross- participant applicability. Several signal processing techniques and deep learning approaches are evaluated in multi-task environments. These methods account for temporal, spatial, and frequential data dependencies. Variance of frequency- domain power distributions for cross-day workload classification is statistically significant. Skewness and kurtosis are not significant in an environment absent workload transitions, but are salient with transitions present. LSTMs improve day- to-day feature stationarity, decreasing error by 59% compared to previous best results. A multi-path convolutional recurrent model using bi-directional, residual recurrent layers significantly increases predictive accuracy and decreases cross-participant variance. Deep learning regression approaches are applied to a multi-task environment with workload transitions. Accounting for temporal dependence significantly reduces error and increases correlation compared to baselines. Visualization techniques for LSTM feature saliency are developed to understand EEG analysis model biases