14 research outputs found
Multi-scale Recurrent LSTM and Transformer Network for Depth Completion
Lidar depth completion is a new and hot topic of depth estimation. In this
task, it is the key and difficult point to fuse the features of color space and
depth space. In this paper, we migrate the classic LSTM and Transformer modules
from NLP to depth completion and redesign them appropriately. Specifically, we
use Forget gate, Update gate, Output gate, and Skip gate to achieve the
efficient fusion of color and depth features and perform loop optimization at
multiple scales. Finally, we further fuse the deep features through the
Transformer multi-head attention mechanism. Experimental results show that
without repetitive network structure and post-processing steps, our method can
achieve state-of-the-art performance by adding our modules to a simple
encoder-decoder network structure. Our method ranks first on the current
mainstream autonomous driving KITTI benchmark dataset. It can also be regarded
as a backbone network for other methods, which likewise achieves
state-of-the-art performance
RoSAS: Deep Semi-Supervised Anomaly Detection with Contamination-Resilient Continuous Supervision
Semi-supervised anomaly detection methods leverage a few anomaly examples to
yield drastically improved performance compared to unsupervised models.
However, they still suffer from two limitations: 1) unlabeled anomalies (i.e.,
anomaly contamination) may mislead the learning process when all the unlabeled
data are employed as inliers for model training; 2) only discrete supervision
information (such as binary or ordinal data labels) is exploited, which leads
to suboptimal learning of anomaly scores that essentially take on a continuous
distribution. Therefore, this paper proposes a novel semi-supervised anomaly
detection method, which devises \textit{contamination-resilient continuous
supervisory signals}. Specifically, we propose a mass interpolation method to
diffuse the abnormality of labeled anomalies, thereby creating new data samples
labeled with continuous abnormal degrees. Meanwhile, the contaminated area can
be covered by new data samples generated via combinations of data with correct
labels. A feature learning-based objective is added to serve as an optimization
constraint to regularize the network and further enhance the robustness w.r.t.
anomaly contamination. Extensive experiments on 11 real-world datasets show
that our approach significantly outperforms state-of-the-art competitors by
20%-30% in AUC-PR and obtains more robust and superior performance in settings
with different anomaly contamination levels and varying numbers of labeled
anomalies. The source code is available at https://github.com/xuhongzuo/rosas/.Comment: Accepted by Information Processing and Management (IP&M
Rethinking Class Relations: Absolute-relative Supervised and Unsupervised Few-shot Learning
The majority of existing few-shot learning methods describe image relations
with binary labels. However, such binary relations are insufficient to teach
the network complicated real-world relations, due to the lack of decision
smoothness. Furthermore, current few-shot learning models capture only the
similarity via relation labels, but they are not exposed to class concepts
associated with objects, which is likely detrimental to the classification
performance due to underutilization of the available class labels. To
paraphrase, children learn the concept of tiger from a few of actual examples
as well as from comparisons of tiger to other animals. Thus, we hypothesize
that in fact both similarity and class concept learning must be occurring
simultaneously. With these observations at hand, we study the fundamental
problem of simplistic class modeling in current few-shot learning methods. We
rethink the relations between class concepts, and propose a novel
Absolute-relative Learning paradigm to fully take advantage of label
information to refine the image representations and correct the relation
understanding in both supervised and unsupervised scenarios. Our proposed
paradigm improves the performance of several the state-of-the-art models on
publicly available datasets.Comment: IEEE/CVF Conference on Computer Vision and Pattern Recognition 202
Learning Multimodal Representations by Symmetrically Transferring Local Structures
Multimodal representations play an important role in multimodal learning tasks, including cross-modal retrieval and intra-modal clustering. However, existing multimodal representation learning approaches focus on building one common space by aligning different modalities and ignore the complementary information across the modalities, such as the intra-modal local structures. In other words, they only focus on the object-level alignment and ignore structure-level alignment. To tackle the problem, we propose a novel symmetric multimodal representation learning framework by transferring local structures across different modalities, namely MTLS. A customized soft metric learning strategy and an iterative parameter learning process are designed to symmetrically transfer local structures and enhance the cluster structures in intra-modal representations. The bidirectional retrieval loss based on multi-layer neural networks is utilized to align two modalities. MTLS is instantiated with image and text data and shows its superior performance on image-text retrieval and image clustering. MTLS outperforms the state-of-the-art multimodal learning methods by up to 32% in terms of R@1 on text-image retrieval and 16.4% in terms of AMI onclustering
Representation Learning with Multiple Lipschitz-Constrained Alignments on Partially-Labeled Cross-Domain Data
The cross-domain representation learning plays an important role in tasks including domain adaptation and transfer learning. However, existing cross-domain representation learning focuses on building one shared space and ignores the unlabeled data in the source domain, which cannot effectively capture the distribution and structure heterogeneities in cross-domain data. To address this challenge, we propose a new cross-domain representation learning approach: MUltiple Lipschitz-constrained AligNments (MULAN) on partially-labeled cross-domain data. MULAN produces two representation spaces: a common representation space to incorporate knowledge from the source domain and a complementary representation space to complement the common representation with target local topological information by Lipschitz-constrained representation transformation. MULAN utilizes both unlabeled and labeled data in the source and target domains to address distribution heterogeneity by Lipschitz-constrained adversarial distribution alignment and structure heterogeneity by cluster assumption-based class alignment while keeping the target local topological information in complementary representation by self alignment. Moreover, MULAN is effectively equipped with a customized learning process and an iterative parameter updating process. MULAN shows its superior performance on partially-labeled semi-supervised domain adaptation and few-shot domain adaptation and outperforms the state-of-the-art visual domain adaptation models by up to 12.1%
Embedding-based representation of categorical data by hierarchical value coupling learning
Learning the representation of categorical data with hierarchical value coupling relationships is very challenging but critical for the effective analysis and learning of such data. This paper proposes a novel coupled unsupervised categorical data representation (CURE) framework and its instantiation, i.e., a coupled data embedding (CDE) method, for representing categorical data by hierarchical valueto-value cluster coupling learning. Unlike existing embedding- and similarity-based representation methods which can capture only a part or none of these complex couplings, CDE explicitly incorporates the hierarchical couplings into its embedding representation. CDE first learns two complementary feature value couplings which are then used to cluster values with different granularities. It further models the couplings in value clusters within the same granularity and with different granularities to embed feature values into a new numerical space with independent dimensions. Substantial experiments show that CDE significantly outperforms three popular unsupervised embedding methods and three state-of-the-art similarity-based representation methods