143 research outputs found
Learning-Based Biharmonic Augmentation for Point Cloud Classification
Point cloud datasets often suffer from inadequate sample sizes in comparison
to image datasets, making data augmentation challenging. While traditional
methods, like rigid transformations and scaling, have limited potential in
increasing dataset diversity due to their constraints on altering individual
sample shapes, we introduce the Biharmonic Augmentation (BA) method. BA is a
novel and efficient data augmentation technique that diversifies point cloud
data by imposing smooth non-rigid deformations on existing 3D structures. This
approach calculates biharmonic coordinates for the deformation function and
learns diverse deformation prototypes. Utilizing a CoefNet, our method predicts
coefficients to amalgamate these prototypes, ensuring comprehensive
deformation. Moreover, we present AdvTune, an advanced online augmentation
system that integrates adversarial training. This system synergistically
refines the CoefNet and the classification network, facilitating the automated
creation of adaptive shape deformations contingent on the learner status.
Comprehensive experimental analysis validates the superiority of Biharmonic
Augmentation, showcasing notable performance improvements over prevailing point
cloud augmentation techniques across varied network designs
3D Adversarial Augmentations for Robust Out-of-Domain Predictions
Since real-world training datasets cannot properly sample the long tail of
the underlying data distribution, corner cases and rare out-of-domain samples
can severely hinder the performance of state-of-the-art models. This problem
becomes even more severe for dense tasks, such as 3D semantic segmentation,
where points of non-standard objects can be confidently associated to the wrong
class. In this work, we focus on improving the generalization to out-of-domain
data. We achieve this by augmenting the training set with adversarial examples.
First, we learn a set of vectors that deform the objects in an adversarial
fashion. To prevent the adversarial examples from being too far from the
existing data distribution, we preserve their plausibility through a series of
constraints, ensuring sensor-awareness and shapes smoothness. Then, we perform
adversarial augmentation by applying the learned sample-independent vectors to
the available objects when training a model. We conduct extensive experiments
across a variety of scenarios on data from KITTI, Waymo, and CrashD for 3D
object detection, and on data from SemanticKITTI, Waymo, and nuScenes for 3D
semantic segmentation. Despite training on a standard single dataset, our
approach substantially improves the robustness and generalization of both 3D
object detection and 3D semantic segmentation methods to out-of-domain data.Comment: 37 pages, 12 figure
Robust Machine Learning In Computer Vision
Deep neural networks have been shown to be successful in various computer vision tasks such as image classification and object detection. Although deep neural networks have exceeded human performance in many tasks, robustness and reliability are always the concerns of using deep learning models. On the one hand, degraded images and videos aggravate the performance of computer vision tasks. On the other hand, if the deep neural networks are under adversarial attacks, the networks can be broken completely. Motivated by the vulnerability of deep neural networks, I analyze and develop image restoration and adversarial defense algorithms towards a vision of robust machine learning in computer vision.
In this dissertation, I study two types of degradation making deep neural networks vulnerable. The first part of the dissertation focuses on face recognition at long range, whose performance is severely degraded by atmospheric turbulence. The theme is on improving the performance and robustness of various tasks in face recognition systems such as facial keypoints localization, feature extraction, and image restoration. The second part focuses on defending adversarial attacks in the images classification task. The theme is on exploring adversarial defense methods that can achieve good performance in standard accuracy, robustness to adversarial attacks with known threat models, and good generalization to other unseen attacks
Sample-adaptive Augmentation for Point Cloud Recognition Against Real-world Corruptions
Robust 3D perception under corruption has become an essential task for the
realm of 3D vision. While current data augmentation techniques usually perform
random transformations on all point cloud objects in an offline way and ignore
the structure of the samples, resulting in over-or-under enhancement. In this
work, we propose an alternative to make sample-adaptive transformations based
on the structure of the sample to cope with potential corruption via an
auto-augmentation framework, named as AdaptPoint. Specially, we leverage a
imitator, consisting of a Deformation Controller and a Mask Controller,
respectively in charge of predicting deformation parameters and producing a
per-point mask, based on the intrinsic structural information of the input
point cloud, and then conduct corruption simulations on top. Then a
discriminator is utilized to prevent the generation of excessive corruption
that deviates from the original data distribution. In addition, a
perception-guidance feedback mechanism is incorporated to guide the generation
of samples with appropriate difficulty level. Furthermore, to address the
paucity of real-world corrupted point cloud, we also introduce a new dataset
ScanObjectNN-C, that exhibits greater similarity to actual data in real-world
environments, especially when contrasted with preceding CAD datasets.
Experiments show that our method achieves state-of-the-art results on multiple
corruption benchmarks, including ModelNet-C, our ScanObjectNN-C, and
ShapeNet-C.Comment: Accepted by ICCV2023; code: https://github.com/Roywangj/AdaptPoin
- …