916 research outputs found
Trends in Machine Learning and Electroencephalogram (EEG): A Review for Undergraduate Researchers
This paper presents a systematic literature review on Brain-Computer
Interfaces (BCIs) in the context of Machine Learning. Our focus is on
Electroencephalography (EEG) research, highlighting the latest trends as of
2023. The objective is to provide undergraduate researchers with an accessible
overview of the BCI field, covering tasks, algorithms, and datasets. By
synthesizing recent findings, our aim is to offer a fundamental understanding
of BCI research, identifying promising avenues for future investigations.Comment: 14 pages, 1 figure, HCI International 2023 Conferenc
Extrapolatable Transformer Pre-training for Ultra Long Time-Series Forecasting
Large-scale pre-trained models (PTMs) such as BERT and GPT have recently
achieved great success in Natural Language Processing and Computer Vision
domains. However, the development of PTMs on time-series data is lagging
behind. This underscores the limitations of the existing transformer-based
architectures, particularly their scalability to handle large-scale data and
ability to capture long-term temporal dependencies. In this study, we present
Timely Generative Pre-trained Transformer (TimelyGPT). TimelyGPT employs an
extrapolatable position (xPos) embedding to encode trend and periodic patterns
into time-series representations. It also integrates recurrent attention and
temporal convolution modules to effectively capture global-local temporal
dependencies. Our experiments show that TimelyGPT excels in modeling
continuously monitored biosignals and irregularly-sampled time series data
commonly observed in longitudinal electronic health records (EHRs). In
ultra-long-term forecasting experiment, TimelyGPT achieves accurate
extrapolation up to 6,000 timesteps of body temperature during the sleep stage
transition given a short look-up window (i.e., prompt) containing only 2,000
timesteps. We further demonstrated TimelyGPT's forecasting capabilities on a
preprocessed longitudinal healthcare administrative database called PopHR
consisting of 489,000 patients randomly sampled from Montreal population.
Together, we envision TimelyGPT to be useful in a broad spectrum of health
domains including long-term patient health state forecasting and patient risk
trajectory prediction
Emotion Recognition with Pre-Trained Transformers Using Multimodal Signals
In this paper, we address the problem of multimodal emotion recognition from
multiple physiological signals. We demonstrate that a Transformer-based
approach is suitable for this task. In addition, we present how such models may
be pretrained in a multimodal scenario to improve emotion recognition
performances. We evaluate the benefits of using multimodal inputs and
pre-training with our approach on a state-ofthe-art dataset
EVOKE: Emotion Enabled Virtual Avatar Mapping Using Optimized Knowledge Distillation
As virtual environments continue to advance, the demand for immersive and
emotionally engaging experiences has grown. Addressing this demand, we
introduce Emotion enabled Virtual avatar mapping using Optimized KnowledgE
distillation (EVOKE), a lightweight emotion recognition framework designed for
the seamless integration of emotion recognition into 3D avatars within virtual
environments. Our approach leverages knowledge distillation involving
multi-label classification on the publicly available DEAP dataset, which covers
valence, arousal, and dominance as primary emotional classes. Remarkably, our
distilled model, a CNN with only two convolutional layers and 18 times fewer
parameters than the teacher model, achieves competitive results, boasting an
accuracy of 87% while demanding far less computational resources. This
equilibrium between performance and deployability positions our framework as an
ideal choice for virtual environment systems. Furthermore, the multi-label
classification outcomes are utilized to map emotions onto custom-designed 3D
avatars.Comment: Presented at IEEE 42nd International Conference on Consumer
Electronics (ICCE) 202
Large-scale Foundation Models and Generative AI for BigData Neuroscience
Recent advances in machine learning have made revolutionary breakthroughs in
computer games, image and natural language understanding, and scientific
discovery. Foundation models and large-scale language models (LLMs) have
recently achieved human-like intelligence thanks to BigData. With the help of
self-supervised learning (SSL) and transfer learning, these models may
potentially reshape the landscapes of neuroscience research and make a
significant impact on the future. Here we present a mini-review on recent
advances in foundation models and generative AI models as well as their
applications in neuroscience, including natural language and speech, semantic
memory, brain-machine interfaces (BMIs), and data augmentation. We argue that
this paradigm-shift framework will open new avenues for many neuroscience
research directions and discuss the accompanying challenges and opportunities
Transformer-based Self-supervised Multimodal Representation Learning for Wearable Emotion Recognition
Recently, wearable emotion recognition based on peripheral physiological
signals has drawn massive attention due to its less invasive nature and its
applicability in real-life scenarios. However, how to effectively fuse
multimodal data remains a challenging problem. Moreover, traditional
fully-supervised based approaches suffer from overfitting given limited labeled
data. To address the above issues, we propose a novel self-supervised learning
(SSL) framework for wearable emotion recognition, where efficient multimodal
fusion is realized with temporal convolution-based modality-specific encoders
and a transformer-based shared encoder, capturing both intra-modal and
inter-modal correlations. Extensive unlabeled data is automatically assigned
labels by five signal transforms, and the proposed SSL model is pre-trained
with signal transformation recognition as a pretext task, allowing the
extraction of generalized multimodal representations for emotion-related
downstream tasks. For evaluation, the proposed SSL model was first pre-trained
on a large-scale self-collected physiological dataset and the resulting encoder
was subsequently frozen or fine-tuned on three public supervised emotion
recognition datasets. Ultimately, our SSL-based method achieved
state-of-the-art results in various emotion classification tasks. Meanwhile,
the proposed model proved to be more accurate and robust compared to
fully-supervised methods on low data regimes.Comment: Accepted IEEE Transactions On Affective Computin
Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)
How does the brain represent different modes of information? Can we design a
system that automatically understands what the user is thinking? Such questions
can be answered by studying brain recordings like functional magnetic resonance
imaging (fMRI). As a first step, the neuroscience community has contributed
several large cognitive neuroscience datasets related to passive
reading/listening/viewing of concept words, narratives, pictures and movies.
Encoding and decoding models using these datasets have also been proposed in
the past two decades. These models serve as additional tools for basic research
in cognitive science and neuroscience. Encoding models aim at generating fMRI
brain representations given a stimulus automatically. They have several
practical applications in evaluating and diagnosing neurological conditions and
thus also help design therapies for brain damage. Decoding models solve the
inverse problem of reconstructing the stimuli given the fMRI. They are useful
for designing brain-machine or brain-computer interfaces. Inspired by the
effectiveness of deep learning models for natural language processing, computer
vision, and speech, recently several neural encoding and decoding models have
been proposed. In this survey, we will first discuss popular representations of
language, vision and speech stimuli, and present a summary of neuroscience
datasets. Further, we will review popular deep learning based encoding and
decoding architectures and note their benefits and limitations. Finally, we
will conclude with a brief summary and discussion about future trends. Given
the large amount of recently published work in the `computational cognitive
neuroscience' community, we believe that this survey nicely organizes the
plethora of work and presents it as a coherent story.Comment: 16 pages, 10 figure
- …