577 research outputs found
Joint Optical Flow and Temporally Consistent Semantic Segmentation
The importance and demands of visual scene understanding have been steadily
increasing along with the active development of autonomous systems.
Consequently, there has been a large amount of research dedicated to semantic
segmentation and dense motion estimation. In this paper, we propose a method
for jointly estimating optical flow and temporally consistent semantic
segmentation, which closely connects these two problem domains and leverages
each other. Semantic segmentation provides information on plausible physical
motion to its associated pixels, and accurate pixel-level temporal
correspondences enhance the accuracy of semantic segmentation in the temporal
domain. We demonstrate the benefits of our approach on the KITTI benchmark,
where we observe performance gains for flow and segmentation. We achieve
state-of-the-art optical flow results, and outperform all published algorithms
by a large margin on challenging, but crucial dynamic objects.Comment: 14 pages, Accepted for CVRSUAD workshop at ECCV 201
Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation
We propose a new deep learning method for tumour segmentation when dealing
with missing imaging modalities. Instead of producing one network for each
possible subset of observed modalities or using arithmetic operations to
combine feature maps, our hetero-modal variational 3D encoder-decoder
independently embeds all observed modalities into a shared latent
representation. Missing data and tumour segmentation can be then generated from
this embedding. In our scenario, the input is a random subset of modalities. We
demonstrate that the optimisation problem can be seen as a mixture sampling. In
addition to this, we introduce a new network architecture building upon both
the 3D U-Net and the Multi-Modal Variational Auto-Encoder (MVAE). Finally, we
evaluate our method on BraTS2018 using subsets of the imaging modalities as
input. Our model outperforms the current state-of-the-art method for dealing
with missing modalities and achieves similar performance to the subset-specific
equivalent networks.Comment: Accepted at MICCAI 201
Identification of Disulfide Bond Formation between MitoNEET and Glutamate Dehydrogenase 1
MitoNEET is a protein that was identified as a drug target for diabetes, but its cellular function as well as its role in diabetes remains elusive. Protein pull-down experiments identified glutamate dehydrogenase 1 (GDH1) as a potential binding partner. GDH1 is a key metabolic enzyme with emerging roles in insulin regulation. MitoNEET forms a covalent complex with GDH1 through disulfide bond formation and acts as an activator. Proteomic analysis identified the specific cysteine residues that participate in the disulfide bond. This is the first report that effectively links mitoNEET to activation of the insulin regulator GDH1
Joint 3D estimation of vehicles and scene flow
driving. While much progress has been made in recent years, imaging conditions in natural outdoor environments are still very challenging for current reconstruction and recognition methods. In this paper, we propose a novel unified approach which reasons jointly about 3D scene flow as well as the pose, shape and motion of vehicles in the scene. Towards this goal, we incorporate a deformable CAD model into a slanted-plane conditional random field for scene flow estimation and enforce shape consistency between the rendered 3D models and the parameters of all superpixels in the image. The association of superpixels to objects is established by an index variable which implicitly enables model selection. We evaluate our approach on the challenging KITTI scene flow dataset in terms of object and scene flow estimation. Our results provide a prove of concept and demonstrate the usefulness of our method. © 2015 Copernicus GmbH. All Rights Reserved
FLOT: Scene Flow on Point Clouds Guided by Optimal Transport
We propose and study a method called FLOT that estimates scene flow on point
clouds. We start the design of FLOT by noticing that scene flow estimation on
point clouds reduces to estimating a permutation matrix in a perfect world.
Inspired by recent works on graph matching, we build a method to find these
correspondences by borrowing tools from optimal transport. Then, we relax the
transport constraints to take into account real-world imperfections. The
transport cost between two points is given by the pairwise similarity between
deep features extracted by a neural network trained under full supervision
using synthetic datasets. Our main finding is that FLOT can perform as well as
the best existing methods on synthetic and real-world datasets while requiring
much less parameters and without using multiscale analysis. Our second finding
is that, on the training datasets considered, most of the performance can be
explained by the learned transport cost. This yields a simpler method,
FLOT, which is obtained using a particular choice of optimal transport
parameters and performs nearly as well as FLOT.Comment: Accepted at ECCV2
Identification of Disulfide Bond Formation between MitoNEET and Glutamate Dehydrogenase 1
MitoNEET is a protein that was identified as a drug target for diabetes, but its cellular function as well as its role in diabetes remains elusive. Protein pull-down experiments identified glutamate dehydrogenase 1 (GDH1) as a potential binding partner. GDH1 is a key metabolic enzyme with emerging roles in insulin regulation. MitoNEET forms a covalent complex with GDH1 through disulfide bond formation and acts as an activator. Proteomic analysis identified the specific cysteine residues that participate in the disulfide bond. This is the first report that effectively links mitoNEET to activation of the insulin regulator GDH1
Neural parameters estimation for brain tumor growth modeling
Understanding the dynamics of brain tumor progression is essential for
optimal treatment planning. Cast in a mathematical formulation, it is typically
viewed as evaluation of a system of partial differential equations, wherein the
physiological processes that govern the growth of the tumor are considered. To
personalize the model, i.e. find a relevant set of parameters, with respect to
the tumor dynamics of a particular patient, the model is informed from
empirical data, e.g., medical images obtained from diagnostic modalities, such
as magnetic-resonance imaging. Existing model-observation coupling schemes
require a large number of forward integrations of the biophysical model and
rely on simplifying assumption on the functional form, linking the output of
the model with the image information. In this work, we propose a learning-based
technique for the estimation of tumor growth model parameters from medical
scans. The technique allows for explicit evaluation of the posterior
distribution of the parameters by sequentially training a mixture-density
network, relaxing the constraint on the functional form and reducing the number
of samples necessary to propagate through the forward model for the estimation.
We test the method on synthetic and real scans of rats injected with brain
tumors to calibrate the model and to predict tumor progression
Deep learning-based parameter mapping for joint relaxation and diffusion tensor MR Fingerprinting
Magnetic Resonance Fingerprinting (MRF) enables the simultaneous
quantification of multiple properties of biological tissues. It relies on a
pseudo-random acquisition and the matching of acquired signal evolutions to a
precomputed dictionary. However, the dictionary is not scalable to
higher-parametric spaces, limiting MRF to the simultaneous mapping of only a
small number of parameters (proton density, T1 and T2 in general). Inspired by
diffusion-weighted SSFP imaging, we present a proof-of-concept of a novel MRF
sequence with embedded diffusion-encoding gradients along all three axes to
efficiently encode orientational diffusion and T1 and T2 relaxation. We take
advantage of a convolutional neural network (CNN) to reconstruct multiple
quantitative maps from this single, highly undersampled acquisition. We bypass
expensive dictionary matching by learning the implicit physical relationships
between the spatiotemporal MRF data and the T1, T2 and diffusion tensor
parameters. The predicted parameter maps and the derived scalar diffusion
metrics agree well with state-of-the-art reference protocols. Orientational
diffusion information is captured as seen from the estimated primary diffusion
directions. In addition to this, the joint acquisition and reconstruction
framework proves capable of preserving tissue abnormalities in multiple
sclerosis lesions
Selection on dispersal drives evolution of metabolic capacities for energy production in female wing-polymorphic sand field crickets, Gryllus firmus
Life history and metabolism covary, but the mechanisms and individual traits responsible for these linkages remain unresolved. Dispersal capability is a critical component of life histories that is constrained by metabolic capacities for energy production. Conflicting relationships between metabolism and life histories may be explained by accounting for variation in dispersal and maximal metabolic rates. We used female wing-polymorphic sand field crickets, Gryllus firmus, selected either for long wings (LW) and flight-capability or short wings (SW) and high early lifetime fecundity to test the hypothesis that selection on dispersal capability drives the evolution of metabolic capacities. While resting metabolic rates were similar, long-winged crickets reached higher maximal metabolic rates than short-winged crickets, resulting in improved running performance. We further provided insight into the mechanisms responsible for covariation between life history and metabolism by comparing mitochondrial content of tissues involved in powering locomotion and assessing function of mitochondria isolated from long- and short-winged crickets. This demonstrated that larger metabolic capacities in long-winged crickets were underpinned by increases in mitochondrial content of dorsoventral flight muscle and enhanced bioenergetic capacities of mitochondria within the fat body, a tissue responsible for fuel storage and mobilization. Thus, selection on flight-capability remodels metabolism in a trait and tissue-specific manner to enlarge metabolic capacities necessary for dispersal
RAFT: Recurrent All-Pairs Field Transforms for Optical Flow
We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network
architecture for optical flow. RAFT extracts per-pixel features, builds
multi-scale 4D correlation volumes for all pairs of pixels, and iteratively
updates a flow field through a recurrent unit that performs lookups on the
correlation volumes. RAFT achieves state-of-the-art performance. On KITTI, RAFT
achieves an F1-all error of 5.10%, a 16% error reduction from the best
published result (6.10%). On Sintel (final pass), RAFT obtains an
end-point-error of 2.855 pixels, a 30% error reduction from the best published
result (4.098 pixels). In addition, RAFT has strong cross-dataset
generalization as well as high efficiency in inference time, training speed,
and parameter count. Code is available at https://github.com/princeton-vl/RAFT.Comment: fixed a formatting issue, Eq 7. no change in conten
- …