397 research outputs found
Multiframe Temporal Estimation of Cardiac Nonrigid Motion
A robust, flexible system for tracking the point to
point nonrigid motion of the left ventricular (LV) endocardial
wall in image sequences has been developed. This system is
unique in its ability to model motion trajectories across multiple
frames. The foundation of this system is an adaptive transversal
filter based on the recursive least-squares algorithm. This filter
facilitates the integration of models for periodicity and proximal
smoothness as appropriate using a contour-based description
of the object’s boundaries. A set of correspondences between
contours and an associated set of correspondence quality measures
comprise the input to the system. Frame-to-frame relationships
from two different frames of reference are derived and analyzed
using synthetic and actual images. Two multiframe temporal
models, both based on a sum of sinusoids, are derived. Illustrative
examples of the system’s output are presented for quantitative
analysis. Validation of the system is performed by comparing
computed trajectory estimates with the trajectories of physical
markers implanted in the LV wall. Sample case studies of marker
trajectory comparisons are presented. Ensemble statistics from
comparisons with 15 marker trajectories are acquired and analyzed. A multiframe temporal model without spatial periodicity
constraints was determined to provide excellent performance with
the least computational cost. A multiframe spatiotemporal model
provided the best performance based on statistical standard
deviation, although at significant computational expense.National Heart, Lung, and Blood InstituteAir Force of Scientific ResearchNational Science FoundationOffice of Naval ResearchR01HL44803F49620-99-1-0481F49620-99-1-0067MIP-9615590N00014-98-1-054
Structure from Recurrent Motion: From Rigidity to Recurrency
This paper proposes a new method for Non-Rigid Structure-from-Motion (NRSfM)
from a long monocular video sequence observing a non-rigid object performing
recurrent and possibly repetitive dynamic action. Departing from the
traditional idea of using linear low-order or lowrank shape model for the task
of NRSfM, our method exploits the property of shape recurrency (i.e., many
deforming shapes tend to repeat themselves in time). We show that recurrency is
in fact a generalized rigidity. Based on this, we reduce NRSfM problems to
rigid ones provided that certain recurrency condition is satisfied. Given such
a reduction, standard rigid-SfM techniques are directly applicable (without any
change) to the reconstruction of non-rigid dynamic shapes. To implement this
idea as a practical approach, this paper develops efficient algorithms for
automatic recurrency detection, as well as camera view clustering via a
rigidity-check. Experiments on both simulated sequences and real data
demonstrate the effectiveness of the method. Since this paper offers a novel
perspective on rethinking structure-from-motion, we hope it will inspire other
new problems in the field.Comment: To appear in CVPR 201
Image-guided Simulation of Heterogeneous Tissue Deformation For Augmented Reality during Hepatic Surgery
International audienceThis paper presents a method for real-time augmentation of vas- cular network and tumors during minimally invasive liver surgery. Internal structures computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Com- pared to state-of-the-art methods, our method uses a real-time biomechanical model to compute a volumetric displacement field from partial three-dimensional liver surface motion. This permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Real-time augmentation results are presented on in vivo and ex vivo data and illustrate the benefits of such an approach for minimally invasive surgery
Learning Human Motion Models for Long-term Predictions
We propose a new architecture for the learning of predictive spatio-temporal
motion models from data alone. Our approach, dubbed the Dropout Autoencoder
LSTM, is capable of synthesizing natural looking motion sequences over long
time horizons without catastrophic drift or motion degradation. The model
consists of two components, a 3-layer recurrent neural network to model
temporal aspects and a novel auto-encoder that is trained to implicitly recover
the spatial structure of the human skeleton via randomly removing information
about joints during training time. This Dropout Autoencoder (D-AE) is then used
to filter each predicted pose of the LSTM, reducing accumulation of error and
hence drift over time. Furthermore, we propose new evaluation protocols to
assess the quality of synthetic motion sequences even for which no ground truth
data exists. The proposed protocols can be used to assess generated sequences
of arbitrary length. Finally, we evaluate our proposed method on two of the
largest motion-capture datasets available to date and show that our model
outperforms the state-of-the-art on a variety of actions, including cyclic and
acyclic motion, and that it can produce natural looking sequences over longer
time horizons than previous methods
Deformable Groupwise Registration Using a Locally Low-Rank Dissimilarity Metric for Myocardial Strain Estimation from Cardiac Cine MRI Images
Objective: Cardiovascular magnetic resonance-feature tracking (CMR-FT)
represents a group of methods for myocardial strain estimation from cardiac
cine MRI images. Established CMR-FT methods are mainly based on optical flow or
pairwise registration. However, these methods suffer from either inaccurate
estimation of large motion or drift effect caused by accumulative tracking
errors. In this work, we propose a deformable groupwise registration method
using a locally low-rank (LLR) dissimilarity metric for CMR-FT. Methods: The
proposed method (Groupwise-LLR) tracks the feature points by a groupwise
registration-based two-step strategy. Unlike the globally low-rank (GLR)
dissimilarity metric, the proposed LLR metric imposes low-rankness on local
image patches rather than the whole image. We quantitatively compared
Groupwise-LLR with the Farneback optical flow, a pairwise registration method,
and a GLR-based groupwise registration method on simulated and in vivo
datasets. Results: Results from the simulated dataset showed that Groupwise-LLR
achieved more accurate tracking and strain estimation compared with the other
methods. Results from the in vivo dataset showed that Groupwise-LLR achieved
more accurate tracking and elimination of the drift effect in late-diastole.
Inter-observer reproducibility of strain estimates was similar between all
studied methods. Conclusion: The proposed method estimates myocardial strains
more accurately due to the application of a groupwise registration-based
tracking strategy and an LLR-based dissimilarity metric. Significance: The
proposed CMR-FT method may facilitate more accurate estimation of myocardial
strains, especially in diastole, for clinical assessments of cardiac
dysfunction
Real-World Repetition Estimation by Div, Grad and Curl
We consider the problem of estimating repetition in video, such as performing
push-ups, cutting a melon or playing violin. Existing work shows good results
under the assumption of static and stationary periodicity. As realistic video
is rarely perfectly static and stationary, the often preferred Fourier-based
measurements is inapt. Instead, we adopt the wavelet transform to better handle
non-static and non-stationary video dynamics. From the flow field and its
differentials, we derive three fundamental motion types and three motion
continuities of intrinsic periodicity in 3D. On top of this, the 2D perception
of 3D periodicity considers two extreme viewpoints. What follows are 18
fundamental cases of recurrent perception in 2D. In practice, to deal with the
variety of repetitive appearance, our theory implies measuring time-varying
flow and its differentials (gradient, divergence and curl) over segmented
foreground motion. For experiments, we introduce the new QUVA Repetition
dataset, reflecting reality by including non-static and non-stationary videos.
On the task of counting repetitions in video, we obtain favorable results
compared to a deep learning alternative
Unsupervised Learning of Complex Articulated Kinematic Structures combining Motion and Skeleton Information
In this paper we present a novel framework for unsupervised kinematic structure learning of complex articulated objects from a single-view image sequence. In contrast to prior motion information based methods, which estimate relatively simple articulations, our method can generate arbitrarily complex kinematic structures with skeletal topology by a successive iterative merge process. The iterative merge process is guided by a skeleton distance function which is generated from a novel object boundary generation method from sparse points. Our main contributions can be summarised as follows: (i) Unsupervised complex articulated kinematic structure learning by combining motion and skeleton information. (ii) Iterative fine-to-coarse merging strategy for adaptive motion segmentation and structure smoothing. (iii) Skeleton estimation from sparse feature points. (iv) A new highly articulated object dataset containing multi-stage complexity with ground truth. Our experiments show that the proposed method out-performs state-of-the-art methods both quantitatively and qualitatively
- …