181 research outputs found
Multi-view Multi-label Fine-grained Emotion Decoding from Human Brain Activity
Decoding emotional states from human brain activity plays an important role
in brain-computer interfaces. Existing emotion decoding methods still have two
main limitations: one is only decoding a single emotion category from a brain
activity pattern and the decoded emotion categories are coarse-grained, which
is inconsistent with the complex emotional expression of human; the other is
ignoring the discrepancy of emotion expression between the left and right
hemispheres of human brain. In this paper, we propose a novel multi-view
multi-label hybrid model for fine-grained emotion decoding (up to 80 emotion
categories) which can learn the expressive neural representations and
predicting multiple emotional states simultaneously. Specifically, the
generative component of our hybrid model is parametrized by a multi-view
variational auto-encoder, in which we regard the brain activity of left and
right hemispheres and their difference as three distinct views, and use the
product of expert mechanism in its inference network. The discriminative
component of our hybrid model is implemented by a multi-label classification
network with an asymmetric focal loss. For more accurate emotion decoding, we
first adopt a label-aware module for emotion-specific neural representations
learning and then model the dependency of emotional states by a masked
self-attention mechanism. Extensive experiments on two visually evoked
emotional datasets show the superiority of our method.Comment: Accepted by IEEE Transactions on Neural Networks and Learning System
DORIS-MAE: Scientific Document Retrieval using Multi-level Aspect-based Queries
In scientific research, the ability to effectively retrieve relevant
documents based on complex, multifaceted queries is critical. Existing
evaluation datasets for this task are limited, primarily due to the high cost
and effort required to annotate resources that effectively represent complex
queries. To address this, we propose a novel task, Scientific DOcument
Retrieval using Multi-level Aspect-based quEries (DORIS-MAE), which is designed
to handle the complex nature of user queries in scientific research. We
developed a benchmark dataset within the field of computer science, consisting
of 100 human-authored complex query cases. For each complex query, we assembled
a collection of 100 relevant documents and produced annotated relevance scores
for ranking them. Recognizing the significant labor of expert annotation, we
also introduce Anno-GPT, a scalable framework for validating the performance of
Large Language Models (LLMs) on expert-level dataset annotation tasks. LLM
annotation of the DORIS-MAE dataset resulted in a 500x reduction in cost,
without compromising quality. Furthermore, due to the multi-tiered structure of
these complex queries, the DORIS-MAE dataset can be extended to over 4,000
sub-query test cases without requiring additional annotation. We evaluated 17
recent retrieval methods on DORIS-MAE, observing notable performance drops
compared to traditional datasets. This highlights the need for better
approaches to handle complex, multifaceted queries in scientific research. Our
dataset and codebase are available at
https://github.com/Real-Doris-Mae/Doris-Mae-Dataset.Comment: To appear in NeurIPS 2023 Datasets and Benchmarks Trac
BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection
While most recent autonomous driving system focuses on developing perception
methods on ego-vehicle sensors, people tend to overlook an alternative approach
to leverage intelligent roadside cameras to extend the perception ability
beyond the visual range. We discover that the state-of-the-art vision-centric
bird's eye view detection methods have inferior performances on roadside
cameras. This is because these methods mainly focus on recovering the depth
regarding the camera center, where the depth difference between the car and the
ground quickly shrinks while the distance increases. In this paper, we propose
a simple yet effective approach, dubbed BEVHeight, to address this issue. In
essence, instead of predicting the pixel-wise depth, we regress the height to
the ground to achieve a distance-agnostic formulation to ease the optimization
process of camera-only perception methods. On popular 3D detection benchmarks
of roadside cameras, our method surpasses all previous vision-centric methods
by a significant margin. The code is available at
{\url{https://github.com/ADLab-AutoDrive/BEVHeight}}.Comment: Accepted by CVPR 202
BEVHeight++: Toward Robust Visual Centric 3D Object Detection
While most recent autonomous driving system focuses on developing perception
methods on ego-vehicle sensors, people tend to overlook an alternative approach
to leverage intelligent roadside cameras to extend the perception ability
beyond the visual range. We discover that the state-of-the-art vision-centric
bird's eye view detection methods have inferior performances on roadside
cameras. This is because these methods mainly focus on recovering the depth
regarding the camera center, where the depth difference between the car and the
ground quickly shrinks while the distance increases. In this paper, we propose
a simple yet effective approach, dubbed BEVHeight++, to address this issue. In
essence, we regress the height to the ground to achieve a distance-agnostic
formulation to ease the optimization process of camera-only perception methods.
By incorporating both height and depth encoding techniques, we achieve a more
accurate and robust projection from 2D to BEV spaces. On popular 3D detection
benchmarks of roadside cameras, our method surpasses all previous
vision-centric methods by a significant margin. In terms of the ego-vehicle
scenario, our BEVHeight++ possesses superior over depth-only methods.
Specifically, it yields a notable improvement of +1.9% NDS and +1.1% mAP over
BEVDepth when evaluated on the nuScenes validation set. Moreover, on the
nuScenes test set, our method achieves substantial advancements, with an
increase of +2.8% NDS and +1.7% mAP, respectively.Comment: arXiv admin note: substantial text overlap with arXiv:2303.0849
- …