39 research outputs found
Investigate Indistinguishable Points in Semantic Segmentation of 3D Point Cloud
This paper investigates the indistinguishable points (difficult to predict
label) in semantic segmentation for large-scale 3D point clouds. The
indistinguishable points consist of those located in complex boundary, points
with similar local textures but different categories, and points in isolate
small hard areas, which largely harm the performance of 3D semantic
segmentation. To address this challenge, we propose a novel Indistinguishable
Area Focalization Network (IAF-Net), which selects indistinguishable points
adaptively by utilizing the hierarchical semantic features and enhances
fine-grained features for points especially those indistinguishable points. We
also introduce multi-stage loss to improve the feature representation in a
progressive way. Moreover, in order to analyze the segmentation performances of
indistinguishable areas, we propose a new evaluation metric called
Indistinguishable Points Based Metric (IPBM). Our IAF-Net achieves the
comparable results with state-of-the-art performance on several popular 3D
point cloud datasets e.g. S3DIS and ScanNet, and clearly outperforms other
methods on IPBM.Comment: AAAI202
Relational Learning between Multiple Pulmonary Nodules via Deep Set Attention Transformers
Diagnosis and treatment of multiple pulmonary nodules are clinically
important but challenging. Prior studies on nodule characterization use
solitary-nodule approaches on multiple nodular patients, which ignores the
relations between nodules. In this study, we propose a multiple instance
learning (MIL) approach and empirically prove the benefit to learn the
relations between multiple nodules. By treating the multiple nodules from a
same patient as a whole, critical relational information between
solitary-nodule voxels is extracted. To our knowledge, it is the first study to
learn the relations between multiple pulmonary nodules. Inspired by recent
advances in natural language processing (NLP) domain, we introduce a
self-attention transformer equipped with 3D CNN, named {NoduleSAT}, to replace
typical pooling-based aggregation in multiple instance learning. Extensive
experiments on lung nodule false positive reduction on LUNA16 database, and
malignancy classification on LIDC-IDRI database, validate the effectiveness of
the proposed method.Comment: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI
2020
MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response
Predicting clinical outcome is remarkably important but challenging. Research
efforts have been paid on seeking significant biomarkers associated with the
therapy response or/and patient survival. However, these biomarkers are
generally costly and invasive, and possibly dissatifactory for novel therapy.
On the other hand, multi-modal, heterogeneous, unaligned temporal data is
continuously generated in clinical practice. This paper aims at a unified deep
learning approach to predict patient prognosis and therapy response, with
easily accessible data, e.g., radiographics, laboratory and clinical
information. Prior arts focus on modeling single data modality, or ignore the
temporal changes. Importantly, the clinical time series is asynchronous in
practice, i.e., recorded with irregular intervals. In this study, we formalize
the prognosis modeling as a multi-modal asynchronous time series classification
task, and propose a MIA-Prognosis framework with Measurement, Intervention and
Assessment (MIA) information to predict therapy response, where a Simple
Temporal Attention (SimTA) module is developed to process the asynchronous time
series. Experiments on synthetic dataset validate the superiory of SimTA over
standard RNN-based approaches. Furthermore, we experiment the proposed method
on an in-house, retrospective dataset of real-world non-small cell lung cancer
patients under anti-PD-1 immunotherapy. The proposed method achieves promising
performance on predicting the immunotherapy response. Notably, our predictive
model could further stratify low-risk and high-risk patients in terms of
long-term survival.Comment: MICCAI 2020 (Early Accepted; Student Travel Award
Probabilistic Radiomics: Ambiguous Diagnosis with Controllable Shape Analysis
Radiomics analysis has achieved great success in recent years. However,
conventional Radiomics analysis suffers from insufficiently expressive
hand-crafted features. Recently, emerging deep learning techniques, e.g.,
convolutional neural networks (CNNs), dominate recent research in
Computer-Aided Diagnosis (CADx). Unfortunately, as black-box predictors, we
argue that CNNs are "diagnosing" voxels (or pixels), rather than lesions; in
other words, visual saliency from a trained CNN is not necessarily concentrated
on the lesions. On the other hand, classification in clinical applications
suffers from inherent ambiguities: radiologists may produce diverse diagnosis
on challenging cases. To this end, we propose a controllable and explainable
{\em Probabilistic Radiomics} framework, by combining the Radiomics analysis
and probabilistic deep learning. In our framework, 3D CNN feature is extracted
upon lesion region only, then encoded into lesion representation, by a
controllable Non-local Shape Analysis Module (NSAM) based on self-attention.
Inspired from variational auto-encoders (VAEs), an Ambiguity PriorNet is used
to approximate the ambiguity distribution over human experts. The final
diagnosis is obtained by combining the ambiguity prior sample and lesion
representation, and the whole network named is end-to-end
trainable. We apply the proposed method on lung nodule diagnosis on LIDC-IDRI
database to validate its effectiveness.Comment: MICCAI 2019 (early accept), with supplementary material
Iterative SE(3)-Transformers
When manipulating three-dimensional data, it is possible to ensure that
rotational and translational symmetries are respected by applying so-called
SE(3)-equivariant models. Protein structure prediction is a prominent example
of a task which displays these symmetries. Recent work in this area has
successfully made use of an SE(3)-equivariant model, applying an iterative
SE(3)-equivariant attention mechanism. Motivated by this application, we
implement an iterative version of the SE(3)-Transformer, an SE(3)-equivariant
attention-based model for graph data. We address the additional complications
which arise when applying the SE(3)-Transformer in an iterative fashion,
compare the iterative and single-pass versions on a toy problem, and consider
why an iterative model may be beneficial in some problem settings. We make the
code for our implementation available to the community
Neighbourhood-Insensitive Point Cloud Normal Estimation Network
We introduce a novel self-attention-based normal estimation network that is
able to focus softly on relevant points and adjust the softness by learning a
temperature parameter, making it able to work naturally and effectively within
a large neighbourhood range. As a result, our model outperforms all existing
normal estimation algorithms by a large margin, achieving 94.1% accuracy in
comparison with the previous state of the art of 91.2%, with a 25x smaller
model and 12x faster inference time. We also use point-to-plane Iterative
Closest Point (ICP) as an application case to show that our normal estimations
lead to faster convergence than normal estimations from other methods, without
manually fine-tuning neighbourhood range parameters. Code available at
https://code.active.vision.Comment: Accepted in BMVC 2020 as oral presentation. Code available at
https://code.active.vision and project page at http://ninormal.active.visio