190 research outputs found
A hierarchy of linguistic predictions during natural language comprehension
Understanding spoken language requires transforming ambiguous acoustic streams into a hierarchy of representations, from phonemes to meaning. It has been suggested that the brain uses prediction to guide the interpretation of incoming input. However, the role of prediction in language processing remains disputed, with disagreement about both the ubiquity and representational nature of predictions. Here, we address both issues by analyzing brain recordings of participants listening to audiobooks, and using a deep neural network (GPT-2) to precisely quantify contextual predictions. First, we establish that brain responses to words are modulated by ubiquitous predictions. Next, we disentangle model-based predictions into distinct dimensions, revealing dissociable neural signatures of predictions about syntactic category (parts of speech), phonemes, and semantics. Finally, we show that high-level (word) predictions inform low-level (phoneme) predictions, supporting hierarchical predictive processing. Together, these results underscore the ubiquity of prediction in language processing, showing that the brain spontaneously predicts upcoming language at multiple levels of abstraction
Non-Redundant Spectral Dimensionality Reduction
Spectral dimensionality reduction algorithms are widely used in numerous
domains, including for recognition, segmentation, tracking and visualization.
However, despite their popularity, these algorithms suffer from a major
limitation known as the "repeated Eigen-directions" phenomenon. That is, many
of the embedding coordinates they produce typically capture the same direction
along the data manifold. This leads to redundant and inefficient
representations that do not reveal the true intrinsic dimensionality of the
data. In this paper, we propose a general method for avoiding redundancy in
spectral algorithms. Our approach relies on replacing the orthogonality
constraints underlying those methods by unpredictability constraints.
Specifically, we require that each embedding coordinate be unpredictable (in
the statistical sense) from all previous ones. We prove that these constraints
necessarily prevent redundancy, and provide a simple technique to incorporate
them into existing methods. As we illustrate on challenging high-dimensional
scenarios, our approach produces significantly more informative and compact
representations, which improve visualization and classification tasks
Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognition
Over the past few years, deep learning methods have shown remarkable results
in many face-related tasks including automatic facial expression recognition
(FER) in-the-wild. Meanwhile, numerous models describing the human emotional
states have been proposed by the psychology community. However, we have no
clear evidence as to which representation is more appropriate and the majority
of FER systems use either the categorical or the dimensional model of affect.
Inspired by recent work in multi-label classification, this paper proposes a
novel multi-task learning (MTL) framework that exploits the dependencies
between these two models using a Graph Convolutional Network (GCN) to recognize
facial expressions in-the-wild. Specifically, a shared feature representation
is learned for both discrete and continuous recognition in a MTL setting.
Moreover, the facial expression classifiers and the valence-arousal regressors
are learned through a GCN that explicitly captures the dependencies between
them. To evaluate the performance of our method under real-world conditions we
perform extensive experiments on the AffectNet and Aff-Wild2 datasets. The
results of our experiments show that our method is capable of improving the
performance across different datasets and backbone architectures. Finally, we
also surpass the previous state-of-the-art methods on the categorical model of
AffectNet.Comment: 9 pages, 8 figures, 5 tables, revised submission to the 16th IEEE
International Conference on Automatic Face and Gesture Recognitio
- …