25 research outputs found
Advancing Vision Transformers with Group-Mix Attention
Vision Transformers (ViTs) have been shown to enhance visual recognition
through modeling long-range dependencies with multi-head self-attention (MHSA),
which is typically formulated as Query-Key-Value computation. However, the
attention map generated from the Query and Key captures only token-to-token
correlations at one single granularity. In this paper, we argue that
self-attention should have a more comprehensive mechanism to capture
correlations among tokens and groups (i.e., multiple adjacent tokens) for
higher representational capacity. Thereby, we propose Group-Mix Attention (GMA)
as an advanced replacement for traditional self-attention, which can
simultaneously capture token-to-token, token-to-group, and group-to-group
correlations with various group sizes. To this end, GMA splits the Query, Key,
and Value into segments uniformly and performs different group aggregations to
generate group proxies. The attention map is computed based on the mixtures of
tokens and group proxies and used to re-combine the tokens and groups in Value.
Based on GMA, we introduce a powerful backbone, namely GroupMixFormer, which
achieves state-of-the-art performance in image classification, object
detection, and semantic segmentation with fewer parameters than existing
models. For instance, GroupMixFormer-L (with 70.3M parameters and 384^2 input)
attains 86.2% Top-1 accuracy on ImageNet-1K without external data, while
GroupMixFormer-B (with 45.8M parameters) attains 51.2% mIoU on ADE20K
MetaBEV: Solving Sensor Failures for BEV Detection and Map Segmentation
Perception systems in modern autonomous driving vehicles typically take
inputs from complementary multi-modal sensors, e.g., LiDAR and cameras.
However, in real-world applications, sensor corruptions and failures lead to
inferior performances, thus compromising autonomous safety. In this paper, we
propose a robust framework, called MetaBEV, to address extreme real-world
environments involving overall six sensor corruptions and two extreme
sensor-missing situations. In MetaBEV, signals from multiple sensors are first
processed by modal-specific encoders. Subsequently, a set of dense BEV queries
are initialized, termed meta-BEV. These queries are then processed iteratively
by a BEV-Evolving decoder, which selectively aggregates deep features from
either LiDAR, cameras, or both modalities. The updated BEV representations are
further leveraged for multiple 3D prediction tasks. Additionally, we introduce
a new M2oE structure to alleviate the performance drop on distinct tasks in
multi-task joint learning. Finally, MetaBEV is evaluated on the nuScenes
dataset with 3D object detection and BEV map segmentation tasks. Experiments
show MetaBEV outperforms prior arts by a large margin on both full and
corrupted modalities. For instance, when the LiDAR signal is missing, MetaBEV
improves 35.5% detection NDS and 17.7% segmentation mIoU upon the vanilla
BEVFusion model; and when the camera signal is absent, MetaBEV still achieves
69.2% NDS and 53.7% mIoU, which is even higher than previous works that perform
on full-modalities. Moreover, MetaBEV performs fairly against previous methods
in both canonical perception and multi-task learning settings, refreshing
state-of-the-art nuScenes BEV map segmentation with 70.4% mIoU.Comment: Project page: https://chongjiange.github.io/metabev.htm
DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Driving
Safety is the primary priority of autonomous driving. Nevertheless, no
published dataset currently supports the direct and explainable safety
evaluation for autonomous driving. In this work, we propose DeepAccident, a
large-scale dataset generated via a realistic simulator containing diverse
accident scenarios that frequently occur in real-world driving. The proposed
DeepAccident dataset contains 57K annotated frames and 285K annotated samples,
approximately 7 times more than the large-scale nuScenes dataset with 40k
annotated samples. In addition, we propose a new task, end-to-end motion and
accident prediction, based on the proposed dataset, which can be used to
directly evaluate the accident prediction ability for different autonomous
driving algorithms. Furthermore, for each scenario, we set four vehicles along
with one infrastructure to record data, thus providing diverse viewpoints for
accident scenarios and enabling V2X (vehicle-to-everything) research on
perception and prediction tasks. Finally, we present a baseline V2X model named
V2XFormer that demonstrates superior performance for motion and accident
prediction and 3D object detection compared to the single-vehicle model
Speed Co-Augmentation for Unsupervised Audio-Visual Pre-training
This work aims to improve unsupervised audio-visual pre-training. Inspired by
the efficacy of data augmentation in visual contrastive learning, we propose a
novel speed co-augmentation method that randomly changes the playback speeds of
both audio and video data. Despite its simplicity, the speed co-augmentation
method possesses two compelling attributes: (1) it increases the diversity of
audio-visual pairs and doubles the size of negative pairs, resulting in a
significant enhancement in the learned representations, and (2) it changes the
strict correlation between audio-visual pairs but introduces a partial
relationship between the augmented pairs, which is modeled by our proposed
SoftInfoNCE loss to further boost the performance. Experimental results show
that the proposed method significantly improves the learned representations
when compared to vanilla audio-visual contrastive learning.Comment: Published at the CVPR 2023 Sight and Sound worksho
AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation
Despite the considerable progress in automatic abdominal multi-organ
segmentation from CT/MRI scans in recent years, a comprehensive evaluation of
the models' capabilities is hampered by the lack of a large-scale benchmark
from diverse clinical scenarios. Constraint by the high cost of collecting and
labeling 3D medical data, most of the deep learning models to date are driven
by datasets with a limited number of organs of interest or samples, which still
limits the power of modern deep models and makes it difficult to provide a
fully comprehensive and fair estimate of various methods. To mitigate the
limitations, we present AMOS, a large-scale, diverse, clinical dataset for
abdominal organ segmentation. AMOS provides 500 CT and 100 MRI scans collected
from multi-center, multi-vendor, multi-modality, multi-phase, multi-disease
patients, each with voxel-level annotations of 15 abdominal organs, providing
challenging examples and test-bed for studying robust segmentation algorithms
under diverse targets and scenarios. We further benchmark several
state-of-the-art medical segmentation models to evaluate the status of the
existing methods on this new challenging dataset. We have made our datasets,
benchmark servers, and baselines publicly available, and hope to inspire future
research. Information can be found at https://amos22.grand-challenge.org
Dynamic doping and Cottrell atmosphere optimize the thermoelectric performance of n-type PbTe
High thermoelectric energy conversion efficiency requires a large
figure-of-merit, zT, over a broad temperature range. To achieve this, we
optimize the carrier concentrations of n-type PbTe from room up to hot-end
temperatures by co-doping Bi and Ag. Bi is an efficient n-type dopant in PbTe,
often leading to excessive carrier concentration at room temperature. As
revealed by density functional theory calculations, the formation of Bi and Ag
defect complexes is exploited to optimize the room temperature carrier
concentration. At elevated temperatures, we demonstrate the dynamic dissolution
of Ag2Te precipitates in PbTe in situ by heating in a scanning transmission
electron microscope. The release of n-type Ag interstitials with increasing
temperature fulfills the requirement of higher carrier concentrations at the
hot end. Moreover, as characterized by atom probe tomography, Ag atoms
aggregate along parallel dislocation arrays to form Cottrell atmospheres. This
results in enhanced phonon scattering and leads to a low lattice thermal
conductivity. As a result of the synergy of dynamic doping and phonon
scattering at decorated dislocations, an average zT of 1.0 is achieved in
n-type Bi/Ag-codoped PbTe between 400 and 825 K. Introducing dopants with
temperature-dependent solubility and strong interaction with dislocation cores
enables simultaneous optimization of the average power factor and thermal
conductivity, providing a new concept to exploit in the field of
thermoelectrics
Associations of risk factor burden and genetic predisposition with the 10-year risk of atrial fibrillation: observations from a large prospective study of 348,904 participants
BackgroundUnderstanding the effects of risk factor burden and genetic predisposition on the long-term risk of atrial fibrillation (AF) is important to improve public health initiatives. However, the 10-year risk of AF considering risk factor burden and genetic predisposition is unknown.MethodsA total of 348,904 genetically unrelated participants without AF at baseline from the UK were categorized into three groups: index ages 45 years (n = 84,206), 55 years (n=117,520), and 65 years (n=147,178). Optimal, borderline, or elevated risk factor burden was determined by body mass index, blood pressure, diabetes mellitus, alcohol consumption, smoking status, and history of myocardial infarction or heart failure. Genetic predisposition was estimated using the polygenic risk score (PRS), constructed using 165 predefined genetic risk variants. The combined effects of risk factor burden and PRS on the risk of incident AF in 10 years were estimated for each index age. Fine and Gray models were developed to predict the 10-year risk of AF.ResultsThe overall 10-year risk of AF was 0.67% (95% CI: 0.61-0.73%) for index age 45 years, 2.05% (95% CI: 1.96-2.13%) for index age 55 years, and 6.34% (95% CI: 6.21-6.46%) for index age 65 years, respectively. An optimal risk factor burden was associated with later AF onset regardless of genetic predisposition and sex (P ConclusionsRisk factor burden together with a genetic predisposition is associated with the 10-year risk of AF. Our results may be helpful in selecting high-risk individuals for primary prevention of AF and facilitating subsequent health interventions