Understand Group Interaction and Cognitive State in Online Collaborative Problem Solving: Leveraging Brain-to-Brain Synchrony Data

The purpose of this study aimed to analyze the process of online collaborative problem solving (CPS) via brain-to-brain synchrony (BS) at the problem-understanding and problem-solving stages. Aiming to obtain additional insights than traditional approaches (survey and observation), BS refers to the synchronization of brain activity between two or more people, as an indicator of interpersonal interaction or common attention. Thirty-six undergraduate students participated. Results indicate the problem-understanding stage showed a higher level of BS than the problem-solving stage. Moreover, the level of BS at the problem-solving stage was significantly correlated with task performance. Groups with all high CPS skill students had the highest level of BS, while some of the mixed groups could achieve the same level of BS. BS is an effective indicator of CPS to group performance and individual interaction. Implications for the online CPS design and possible supports for the process of online CPS activity are also discussed

Analyzing the Impact of Cognitive Load in Evaluating Gaze-based Typing

Gaze-based virtual keyboards provide an effective interface for text entry by eye movements. The efficiency and usability of these keyboards have traditionally been evaluated with conventional text entry performance measures such as words per minute, keystrokes per character, backspace usage, etc. However, in comparison to the traditional text entry approaches, gaze-based typing involves natural eye movements that are highly correlated with human brain cognition. Employing eye gaze as an input could lead to excessive mental demand, and in this work we argue the need to include cognitive load as an eye typing evaluation measure. We evaluate three variations of gaze-based virtual keyboards, which implement variable designs in terms of word suggestion positioning. The conventional text entry metrics indicate no significant difference in the performance of the different keyboard designs. However, STFT (Short-time Fourier Transform) based analysis of EEG signals indicate variances in the mental workload of participants while interacting with these designs. Moreover, the EEG analysis provides insights into the user's cognition variation for different typing phases and intervals, which should be considered in order to improve eye typing usability.Comment: 6 pages, 4 figures, IEEE CBMS 201

The Effect of Bilingualism on Perceptual Processing in Adults

abstract: The experience of language can, as any other experience, change the way that the human brain is organized and connected. Fluency in more than one language should, in turn, change the brain in the same way. Recent research has focused on the differences in processing between bilinguals and monolinguals, and has even ventured into using different neuroimaging techniques to study why these differences exist. What previous research has failed to identify is the mechanism that is responsible for the difference in processing. In an attempt to gather information about these effects, this study explores the possibility that bilingual individuals utilize lower signal strength (and by comparison less biological energy) to complete the same tasks that monolingual individuals do. Using an electroencephalograph (EEG), signal strength is retrieved during two perceptual tasks, the Landolt C and the critical flicker fusion threshold, as well as one executive task (the Stroop task). Most likely due to small sample size, bilingual participants did not perform better than monolingual participants on any of the tasks they were given, but they did show a lower EEG signal strength during the Landolt C task than monolingual participants. Monolingual participants showed a lower EEG signal strength during the Stroop task, which stands to support the idea that a linguistic processing task adds complexity to the bilingual brain. Likewise, analysis revealed a significantly lower signal strength during the critical flicker fusion task for monolingual participants than for bilingual participants. Monolingual participants also had a significantly different variability during the critical flicker fusion threshold task, suggesting that becoming bilingual creates an entirely separate population of individuals. Future research should perform analysis with the addition of a prefrontal cortex electrode to determine if less collaboration during processing is present for bilinguals, and if signal complexity in the prefrontal cortex is lower than other electrodes.Dissertation/ThesisMasters Thesis Psychology 201

Deep Long Short-term Memory Structures Model Temporal Dependencies Improving Cognitive Workload Estimation

Using deeply recurrent neural networks to account for temporal dependence in electroencephalograph (EEG)-based workload estimation is shown to considerably improve day-to-day feature stationarity resulting in significantly higher accuracy (p \u3c .0001) than classifiers which do not consider the temporal dependence encoded within the EEG time-series signal. This improvement is demonstrated by training several deep Recurrent Neural Network (RNN) models including Long Short-Term Memory (LSTM) architectures, a feedforward Artificial Neural Network (ANN), and Support Vector Machine (SVM) models on data from six participants who each perform several Multi-Attribute Task Battery (MATB) sessions on five separate days spread out over a month-long period. Each participant-specific classifier is trained on the first four days of data and tested using the fifth’s. Average classification accuracy of 93.0% is achieved using a deep LSTM architecture. These results represent a 59% decrease in error compared to the best previously published results for this dataset. This study additionally evaluates the significance of new features: all combinations of mean, variance, skewness, and kurtosis of EEG frequency-domain power distributions. Mean and variance are statistically significant features, while skewness and kurtosis are not. The overall performance of this approach is high enough to warrant evaluation for inclusion in operational systems

Analysis of electrogenesis’ changes in mental retardation persons by using computer electroencephalography

Justification. Mental retardation is a persistent decrease in human cognitive activity against the background of organic damage to the central nervous system. Neurophysiological diagnostics, in particular electroencephalography (EEG), most adequately reflects the morpho-functional state of the central nervous system, which is the basis of the mechanisms of mental activity, and the originality of the bioelectrical activity of the brain can be considered as the main indicator that determines a decrease in the level of intellectual development and, thereby, characterizes this state. This provision actualizes the search for highly informative indicators of the originality of the bioelectrical activity of the brain in children with intellectual disabilities. Purspose. With the use of periodometric analysis investigate EEG’s indicators and interhemispheric asymmetry of rhythms amplitudes in MR patients. Materials and methods. The EEG was recorded in a state of calm wakefulness with closed eyes with Neuron-Spectrum-2 electroencephalograph. Differences in indicators were tracked using the calculation of the coefficient of compliance (CC), EEG functional asymmetry coefficients in amplitude were determined, too. Results. It was revealed that in MR patients the amplitudes of the rhythms were greater than in healthy subjects. The greatest increase was determined in theta rhythm in the anterior temporal and posterior temporal leads in the left hemispheres. Duration indices in the delta, theta and alpha ranges of the EEG in mental retardation compared with the control group were increased, and the indices of the duration of beta rhythms - decreased. When analyzing FMPA in MR persons it turned out that in right-handers the negativeness of FMPA indices increased, and in left-handers there was an increase in the positivity of FMPA indices. Conclusions 1. With mental retardation, the amplitudes of the rhythms were greater than in healthy people. The greatest increase was determined in theta rhythm in the anterior temporal and posterior temporal leads in the left hemispheres. 2. The indices of duration in the delta, theta and alpha ranges of the EEG of MR subjects were increased, and the indices of the duration of beta rhythms – decreased. 3. When analyzing FMPA in MR persons, it turned out that in right-handers the negativeness of FMPA indices increased, and in left-handers there was an increase in the positivity of FMPA indices

Working memory impairments imitate age-related behaviors in children using visual stimulation based on event-related potentials

The aim of this study is to examine the working memory impairments imitate age-related between 7 to 12 years old using Event-Related Potentials (ERP) signal. 97 normal children were selected to a visual stimuli assessment (Phase 1 and Phase 2) while their working memory response was recorded using Electroencephalograph (EEG) machine. Raw EEG signal were segmented and averaged into the ERP signal according to the event stimulus occur. Discrete Wavelet Transform technique is preferred to decompose the ERP signal into different frequency band. ERP signal at alpha frequency is used because of alpha is the most prominent component of brain waves activity. The necessary features were extracted as an input for the Logistic Regression (LR) and Support Vector Machine (SVM) classifier. Consequence indicated that the accuracy and mean performance results were significant in predicting either a child had working memory impairment or not. 7 years old have lower accuracy compared to other groups with 60% for LR and 86% for SVM. In conclusion, the study proposed that age-related changes and increasing level of visual stimuli affect working memory impaired. Thus, this study has provided empirical evidence in support for the assumption that younger children have working memory impaired through visual stimuli assessment

Optimal set of EEG features for emotional state classification and trajectory visualization in Parkinson's disease

In addition to classic motor signs and symptoms, individuals with Parkinson's disease (PD) are characterized by emotional deficits. Ongoing brain activity can be recorded by electroencephalograph (EEG) to discover the links between emotional states and brain activity. This study utilized machine-learning algorithms to categorize emotional states in PD patients compared with healthy controls (HC) using EEG. Twenty non-demented PD patients and 20 healthy age-, gender-, and education level-matched controls viewed happiness, sadness, fear, anger, surprise, and disgust emotional stimuli while fourteen-channel EEG was being recorded. Multimodal stimulus (combination of audio and visual) was used to evoke the emotions. To classify the EEG-based emotional states and visualize the changes of emotional states over time, this paper compares four kinds of EEG features for emotional state classification and proposes an approach to track the trajectory of emotion changes with manifold learning. From the experimental results using our EEG data set, we found that (a) bispectrum feature is superior to other three kinds of features, namely power spectrum, wavelet packet and nonlinear dynamical analysis; (b) higher frequency bands (alpha, beta and gamma) play a more important role in emotion activities than lower frequency bands (delta and theta) in both groups and; (c) the trajectory of emotion changes can be visualized by reducing subject-independent features with manifold learning. This provides a promising way of implementing visualization of patient's emotional state in real time and leads to a practical system for noninvasive assessment of the emotional impairments associated with neurological disorders

Master of Science

thesisToday, the majority of people in the United States reside in dynamic urban environments and navigate multiple, complex situations in their daily routines. These factors of modern life often require prolonged appropriation of cognitive energy to both external stimuli and a continuous series of tasks, resulting in directed attention fatigue. Directed attention fatigue is marked by a diminishment in an individual’s physiological state (alterations in neural activity), cognitive state (a decrease in motivation, reduction in the capacity to focus attention, and difficulties ignoring irrelevant information), and affective state (changes in emotional responses). The depletion of this resource results in, among other things, reduced task performance, which carries a potential for drastic, negative consequences. Increasingly, mobile phones are having a significant impact on these states. As of 2013, an estimated 91% of adults owned a mobile phone and most frequently use it for texting. Emerging trends involve the changing relationship between user and device, as a growing number of smartphone owners exhibit behaviors of over-use, dependency, and even addiction. Given the near constant presence of mobile phones and their increasing use for personal and professional purposes, their ability to constantly place demands on directed attention is cause for concern. Exposure to nature-rich surroundings, however, has been shown to activate alternate attentional networks, forcing the deactivation and restoration of the directed attention network. It was the purpose of this pilot study to determine to what extent directed attention is activated or deactivated in a nature-based environment when an individual is aware of the potentially distracting presence of their mobile phone. To this end, electroencephalograph recordings, a Recognition Memory Task, and the Positive and Negative Affect Schedule were utilized, with and compared across two participant groups â€" those completing a nature walk without a phone and those completing it while receiving text messages on their phones (though instructed not to interact with the device). Upon processing the data, no significant differences were found to exist between groups. The pilot design of this study, however, has offered insight on previously unaccounted for variables, and has the potential to inform the development of future studies

Machine Learning Methods for functional Near Infrared Spectroscopy

Identification of user state is of interest in a wide range of disciplines that fall under the umbrella of human machine interaction. Functional Near Infra-Red Spectroscopy (fNIRS) device is a relatively new device that enables inference of brain activity through non-invasively pulsing infra-red light into the brain. The fNIRS device is particularly useful as it has a better spatial resolution than the Electroencephalograph (EEG) device that is most commonly used in Human Computer Interaction studies under ecologically valid settings. But this key advantage of fNIRS device is underutilized in current literature in the fNIRS domain. We propose machine learning methods that capture this spatial nature of the human brain activity using a novel preprocessing method that uses `Region of Interest\u27 based feature extraction. Experiments show that this method outperforms the F1 score achieved previously in classifying `low\u27 vs `high\u27 valence state of a user. We further our analysis by applying a Convolutional Neural Network (CNN) to the fNIRS data, thus preserving the spatial structure of the data and treating the data similar to a series of images to be classified. Going further, we use a combination of CNN and Long Short-Term Memory (LSTM) to capture the spatial and temporal behavior of the fNIRS data, thus treating it similar to a video classification problem. We show that this method improves upon the accuracy previously obtained by valence classification methods using EEG or fNIRS devices. Finally, we apply the above model to a problem in classifying combined task-load and performance in an across-subject, across-task scenario of a Human Machine Teaming environment in order to achieve optimal productivity of the system