Brain informed transfer learning for categorizing construction hazards

A transfer learning paradigm is proposed for "knowledge" transfer between the human brain and convolutional neural network (CNN) for a construction hazard categorization task. Participants' brain activities are recorded using electroencephalogram (EEG) measurements when viewing the same images (target dataset) as the CNN. The CNN is pretrained on the EEG data and then fine-tuned on the construction scene images. The results reveal that the EEG-pretrained CNN achieves a 9 % higher accuracy compared with a network with same architecture but randomly initialized parameters on a three-class classification task. Brain activity from the left frontal cortex exhibits the highest performance gains, thus indicating high-level cognitive processing during hazard recognition. This work is a step toward improving machine learning algorithms by learning from human-brain signals recorded via a commercially available brain-computer interface. More generalized visual recognition systems can be effectively developed based on this approach of "keep human in the loop"

Sensorimotor Modulations by Cognitive Processes During Accurate Speech Discrimination: An EEG Investigation of Dorsal Stream Processing

Internal models mediate the transmission of information between anterior and posterior regions of the dorsal stream in support of speech perception, though it remains unclear how this mechanism responds to cognitive processes in service of task demands. The purpose of the current study was to identify the influences of attention and working memory on sensorimotor activity across the dorsal stream during speech discrimination, with set size and signal clarity employed to modulate stimulus predictability and the time course of increased task demands, respectively. Independent Component Analysis of 64–channel EEG data identified bilateral sensorimotor mu and auditory alpha components from a cohort of 42 participants, indexing activity from anterior (mu) and posterior (auditory) aspects of the dorsal stream. Time frequency (ERSP) analysis evaluated task-related changes in focal activation patterns with phase coherence measures employed to track patterns of information flow across the dorsal stream. ERSP decomposition of mu clusters revealed event-related desynchronization (ERD) in beta and alpha bands, which were interpreted as evidence of forward (beta) and inverse (alpha) internal modeling across the time course of perception events. Stronger pre-stimulus mu alpha ERD in small set discrimination tasks was interpreted as more efficient attentional allocation due to the reduced sensory search space enabled by predictable stimuli. Mu-alpha and mu-beta ERD in peri- and post-stimulus periods were interpreted within the framework of Analysis by Synthesis as evidence of working memory activity for stimulus processing and maintenance, with weaker activity in degraded conditions suggesting that covert rehearsal mechanisms are sensitive to the quality of the stimulus being retained in working memory. Similar ERSP patterns across conditions despite the differences in stimulus predictability and clarity, suggest that subjects may have adapted to tasks. In light of this, future studies of sensorimotor processing should consider the ecological validity of the tasks employed, as well as the larger cognitive environment in which tasks are performed. The absence of interpretable patterns of mu-auditory coherence modulation across the time course of speech discrimination highlights the need for more sensitive analyses to probe dorsal stream connectivity

Real-time fMRI neurofeedback and smartphone-based interventions to modulate mental functions

Our brains are constantly changing on a molecular level depending on the demands thrown at them by our environments, behavior, and thoughts. This neuronal plasticity allows us to voluntarily influence mental functions. Taking conscious control over mental functions goes potentially back millenia, but it was psychotherapy since the early 20th century which moulded this concept into a concrete form to target specific mental disorders. Mental disorders constitute a large burden on modern societies. Stress-related disorders like anxiety and depression particularly make up a large part of this burden and new ways to treat or prevent them are highly desirable, since traditional approaches are not equally helpful to every person affected. This might be because the infrastructure is not available where the person lives, their schedules and obligations or financial means do not enable them to seek help or they simply do not respond to traditional forms of treatment. Technological advances bring forth new potential approaches to modulate mental functions and allow using additional information to tailor an intervention better to an individual patient. The focus of this dissertation lies on two promising approaches to cognitively intervene and modulate mental functions: real-time functional magnetic resonance imaging neurofeedback (rtfMRInf) on one hand and smartphone-based interventions (SBIs) on the other. To investigate various aspects of both these methods in the context of stress and in relation to personalized interventions, we designed and conducted two experiments with a main rtfMRInf intervention, and also with ambulatory training of mental strategies, which participants accessed on their mobile phones. The four publication this thesis entails, are related to this topic as follows: The first publication focuses on rtfMRInf effects on the physiological stress response, exploring whether neurofeedback could reduce stress-related changes in brain activity and blood pressure. The second publication focuses on rtfMRInf effects on psychological measures related to the stress response, namely on arousal and mood, based on data from self-report by the participants. The third publication focuses on rtfMRInf methodology itself, looking at complex connectivity data between major neural networks. Finally, the fourth publication focuses on personalized prediction of intervention success of an SBI using data from previous training days

EEG-based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications.

Brain-Computer interfaces (BCIs) enhance the capability of human brain activities to interact with the environment. Recent advancements in technology and machine learning algorithms have increased interest in electroencephalographic (EEG)-based BCI applications. EEG-based intelligent BCI systems can facilitate continuous monitoring of fluctuations in human cognitive states under monotonous tasks, which is both beneficial for people in need of healthcare support and general researchers in different domain areas. In this review, we survey the recent literature on EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensating for the gaps in the systematic summary of the past five years. Specifically, we first review the current status of BCI and signal sensing technologies for collecting reliable EEG signals. Then, we demonstrate state-of-the-art computational intelligence techniques, including fuzzy models and transfer learning in machine learning and deep learning algorithms, to detect, monitor, and maintain human cognitive states and task performance in prevalent applications. Finally, we present a couple of innovative BCI-inspired healthcare applications and discuss future research directions in EEG-based BCI research