5 research outputs found
AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation
This paper targets on learning-based novel view synthesis from a single or
limited 2D images without the pose supervision. In the viewer-centered
coordinates, we construct an end-to-end trainable conditional variational
framework to disentangle the unsupervisely learned relative-pose/rotation and
implicit global 3D representation (shape, texture and the origin of
viewer-centered coordinates, etc.). The global appearance of the 3D object is
given by several appearance-describing images taken from any number of
viewpoints. Our spatial correlation module extracts a global 3D representation
from the appearance-describing images in a permutation invariant manner. Our
system can achieve implicitly 3D understanding without explicitly 3D
reconstruction. With an unsupervisely learned viewer-centered
relative-pose/rotation code, the decoder can hallucinate the novel view
continuously by sampling the relative-pose in a prior distribution. In various
applications, we demonstrate that our model can achieve comparable or even
better results than pose/3D model-supervised learning-based novel view
synthesis (NVS) methods with any number of input views.Comment: ECCV 202
Automated interpretation of congenital heart disease from multi-view echocardiograms
Congenital heart disease (CHD) is the most common birth defect and the
leading cause of neonate death in China. Clinical diagnosis can be based on the
selected 2D key-frames from five views. Limited by the availability of
multi-view data, most methods have to rely on the insufficient single view
analysis. This study proposes to automatically analyze the multi-view
echocardiograms with a practical end-to-end framework. We collect the five-view
echocardiograms video records of 1308 subjects (including normal controls,
ventricular septal defect (VSD) patients and atrial septal defect (ASD)
patients) with both disease labels and standard-view key-frame labels.
Depthwise separable convolution-based multi-channel networks are adopted to
largely reduce the network parameters. We also approach the imbalanced class
problem by augmenting the positive training samples. Our 2D key-frame model can
diagnose CHD or negative samples with an accuracy of 95.4\%, and in negative,
VSD or ASD classification with an accuracy of 92.3\%. To further alleviate the
work of key-frame selection in real-world implementation, we propose an
adaptive soft attention scheme to directly explore the raw video data. Four
kinds of neural aggregation methods are systematically investigated to fuse the
information of an arbitrary number of frames in a video. Moreover, with a view
detection module, the system can work without the view records. Our video-based
model can diagnose with an accuracy of 93.9\% (binary classification), and
92.1\% (3-class classification) in a collected 2D video testing set, which does
not need key-frame selection and view annotation in testing. The detailed
ablation study and the interpretability analysis are provided.Comment: Published in Medical Image Analysi
Probabilistic forecast reconciliation: theory, algorithm, and applications
Hierarchical time series are common in several applied fields. Forecasts are required to be coherent, that is, to satisfy the constraints given by the hierarchy. The most popular technique to enforce coherence is called reconciliation, which adjusts the base forecasts computed for each time series. However, recent works on probabilistic reconciliation present several limitations. In this thesis, we propose a new approach based on conditioning to reconcile any type of forecast distribution. We then introduce a new algorithm, called Bottom-Up Importance Sampling, to efficiently sample from the reconciled distribution. It can be used for any base forecast distribution: discrete, continuous, or in the form of samples, providing a major speedup compared to the current methods. Experiments on several temporal hierarchies show a clear improvement over base probabilistic forecasts. We then study the effects of reconciliation on the forecast distribution, both from a theoretical viewpoint and using some examples with Bernoulli and Poisson distributions. We also present an application to count time series of extreme events on the Credit Default Swap (CDS) market. Finally, we introduce and study the p-Fourier Discrepancy Functions, a new family of metrics for comparing discrete probability measures