1,676 research outputs found
Learning Partial Correlation based Deep Visual Representation for Image Classification
Visual representation based on covariance matrix has demonstrates its
efficacy for image classification by characterising the pairwise correlation of
different channels in convolutional feature maps. However, pairwise correlation
will become misleading once there is another channel correlating with both
channels of interest, resulting in the ``confounding'' effect. For this case,
``partial correlation'' which removes the confounding effect shall be estimated
instead. Nevertheless, reliably estimating partial correlation requires to
solve a symmetric positive definite matrix optimisation, known as sparse
inverse covariance estimation (SICE). How to incorporate this process into CNN
remains an open issue. In this work, we formulate SICE as a novel structured
layer of CNN. To ensure end-to-end trainability, we develop an iterative method
to solve the above matrix optimisation during forward and backward propagation
steps. Our work obtains a partial correlation based deep visual representation
and mitigates the small sample problem often encountered by covariance matrix
estimation in CNN. Computationally, our model can be effectively trained with
GPU and works well with a large number of channels of advanced CNNs.
Experiments show the efficacy and superior classification performance of our
deep visual representation compared to covariance matrix based counterparts.Comment: This paper is published at CVPR 202
A Review of Panoptic Segmentation for Mobile Mapping Point Clouds
3D point cloud panoptic segmentation is the combined task to (i) assign each
point to a semantic class and (ii) separate the points in each class into
object instances. Recently there has been an increased interest in such
comprehensive 3D scene understanding, building on the rapid advances of
semantic segmentation due to the advent of deep 3D neural networks. Yet, to
date there is very little work about panoptic segmentation of outdoor
mobile-mapping data, and no systematic comparisons. The present paper tries to
close that gap. It reviews the building blocks needed to assemble a panoptic
segmentation pipeline and the related literature. Moreover, a modular pipeline
is set up to perform comprehensive, systematic experiments to assess the state
of panoptic segmentation in the context of street mapping. As a byproduct, we
also provide the first public dataset for that task, by extending the NPM3D
dataset to include instance labels. That dataset and our source code are
publicly available. We discuss which adaptations are need to adapt current
panoptic segmentation methods to outdoor scenes and large objects. Our study
finds that for mobile mapping data, KPConv performs best but is slower, while
PointNet++ is fastest but performs significantly worse. Sparse CNNs are in
between. Regardless of the backbone, Instance segmentation by clustering
embedding features is better than using shifted coordinates
PANet: LiDAR Panoptic Segmentation with Sparse Instance Proposal and Aggregation
Reliable LiDAR panoptic segmentation (LPS), including both semantic and
instance segmentation, is vital for many robotic applications, such as
autonomous driving. This work proposes a new LPS framework named PANet to
eliminate the dependency on the offset branch and improve the performance on
large objects, which are always over-segmented by clustering algorithms.
Firstly, we propose a non-learning Sparse Instance Proposal (SIP) module with
the ``sampling-shifting-grouping" scheme to directly group thing points into
instances from the raw point cloud efficiently. More specifically, balanced
point sampling is introduced to generate sparse seed points with more uniform
point distribution over the distance range. And a shift module, termed bubble
shifting, is proposed to shrink the seed points to the clustered centers. Then
we utilize the connected component label algorithm to generate instance
proposals. Furthermore, an instance aggregation module is devised to integrate
potentially fragmented instances, improving the performance of the SIP module
on large objects. Extensive experiments show that PANet achieves
state-of-the-art performance among published works on the SemanticKITII
validation and nuScenes validation for the panoptic segmentation task.Comment: 8 pages, 3 figures, IROS202
Distribution Consistency based Self-Training for Graph Neural Networks with Sparse Labels
Few-shot node classification poses a significant challenge for Graph Neural
Networks (GNNs) due to insufficient supervision and potential distribution
shifts between labeled and unlabeled nodes. Self-training has emerged as a
widely popular framework to leverage the abundance of unlabeled data, which
expands the training set by assigning pseudo-labels to selected unlabeled
nodes. Efforts have been made to develop various selection strategies based on
confidence, information gain, etc. However, none of these methods takes into
account the distribution shift between the training and testing node sets. The
pseudo-labeling step may amplify this shift and even introduce new ones,
hindering the effectiveness of self-training. Therefore, in this work, we
explore the potential of explicitly bridging the distribution shift between the
expanded training set and test set during self-training. To this end, we
propose a novel Distribution-Consistent Graph Self-Training (DC-GST) framework
to identify pseudo-labeled nodes that are both informative and capable of
redeeming the distribution discrepancy and formulate it as a differentiable
optimization task. A distribution-shift-aware edge predictor is further adopted
to augment the graph and increase the model's generalizability in assigning
pseudo labels. We evaluate our proposed method on four publicly available
benchmark datasets and extensive experiments demonstrate that our framework
consistently outperforms state-of-the-art baselines.Comment: Accepted by WSDM 202
- …