Accurate Segmentation and Estimation of Parametric Motion Fields for Object-based Video Coding using Mean Field Theory

We formulate the problem of decomposing a scene into its constituent objects as one of partitioning the current frame into objects comprising it. The motion parameter is modeled as a nonrandom but unknown quantity and the problem is posed as one of Maximum Likelihood (ML) estimation. The MRF potentials which characterize the underlying segmentation field are defined in a way that the spatio-temporal segmentation is constrained by the static image segmentation of the current frame. To compute the motion parameter vector and the segmentation simultaneously we use the Expectation Maximization (EM) algorithm. The E-step of the EM algorithm, which computes the conditional expectation of the segmentation field, now reflects interdependencies more accurately because of neighborhood interactions. We take recourse to Mean Field theory to compute the expected value of the conditional MRF. Robust M-estimation methods are used in the M- step. To allow for motions of large magnitudes image frames are represented at various scales and the EM procedure is embedded in a hierarchical coarse-to-fine framework. Our formulation results in a highly parallel algorithm that computes robust and accurate segmentations as well as motion vectors for use in low bit rate video coding. This report has been submitted as a paper to the SPIE conference on Visual Communications and Image Processing - VCIP98 to be held in San Jose, California on Jan 24- 30, 1998. </Center

Perception of Motion and Architectural Form: Computational Relationships between Optical Flow and Perspective

Perceptual geometry refers to the interdisciplinary research whose objectives focuses on study of geometry from the perspective of visual perception, and in turn, applies such geometric findings to the ecological study of vision. Perceptual geometry attempts to answer fundamental questions in perception of form and representation of space through synthesis of cognitive and biological theories of visual perception with geometric theories of the physical world. Perception of form, space and motion are among fundamental problems in vision science. In cognitive and computational models of human perception, the theories for modeling motion are treated separately from models for perception of form.Comment: 10 pages, 13 figures, submitted and accepted in DoCEIS'2012 Conference: http://www.uninova.pt/doceis/doceis12/home/home.ph

Efficient generic calibration method for general cameras with single centre of projection

Generic camera calibration is a non-parametric calibration technique that is applicable to any type of vision sensor. However, the standard generic calibration method was developed with the goal of generality and it is therefore sub-optimal for the common case of cameras with a single centre of projection (e.g. pinhole, fisheye, hyperboloidal catadioptric). This paper proposes novel improvements to the standard generic calibration method for central cameras that reduce its complexity, and improve its accuracy and robustness. Improvements are achieved by taking advantage of the geometric constraints resulting from a single centre of projection. Input data for the algorithm is acquired using active grids, the performance of which is characterised. A new linear estimation stage to the generic algorithm is proposed incorporating classical pinhole calibration techniques, and it is shown to be significantly more accurate than the linear estimation stage of the standard method. A linear method for pose estimation is also proposed and evaluated against the existing polynomial method. Distortion correction and motion reconstruction experiments are conducted with real data for a hyperboloidal catadioptric sensor for both the standard and proposed methods. Results show the accuracy and robustness of the proposed method to be superior to those of the standard method

Probabilistic Motion Estimation Based on Temporal Coherence

We develop a theory for the temporal integration of visual motion motivated by psychophysical experiments. The theory proposes that input data are temporally grouped and used to predict and estimate the motion flows in the image sequence. This temporal grouping can be considered a generalization of the data association techniques used by engineers to study motion sequences. Our temporal-grouping theory is expressed in terms of the Bayesian generalization of standard Kalman filtering. To implement the theory we derive a parallel network which shares some properties of cortical networks. Computer simulations of this network demonstrate that our theory qualitatively accounts for psychophysical experiments on motion occlusion and motion outliers.Comment: 40 pages, 7 figure

Statistical Mechanics and Visual Signal Processing

The nervous system solves a wide variety of problems in signal processing. In many cases the performance of the nervous system is so good that it apporaches fundamental physical limits, such as the limits imposed by diffraction and photon shot noise in vision. In this paper we show how to use the language of statistical field theory to address and solve problems in signal processing, that is problems in which one must estimate some aspect of the environment from the data in an array of sensors. In the field theory formulation the optimal estimator can be written as an expectation value in an ensemble where the input data act as external field. Problems at low signal-to-noise ratio can be solved in perturbation theory, while high signal-to-noise ratios are treated with a saddle-point approximation. These ideas are illustrated in detail by an example of visual motion estimation which is chosen to model a problem solved by the fly's brain. In this problem the optimal estimator has a rich structure, adapting to various parameters of the environment such as the mean-square contrast and the correlation time of contrast fluctuations. This structure is in qualitative accord with existing measurements on motion sensitive neurons in the fly's brain, and we argue that the adaptive properties of the optimal estimator may help resolve conlficts among different interpretations of these data. Finally we propose some crucial direct tests of the adaptive behavior.Comment: 34pp, LaTeX, PUPT-143

Analytical method to measure three-dimensional strain patterns in the left ventricle from single slice displacement data

Background: Displacement encoded Cardiovascular MR (CMR) can provide high spatial resolution measurements of three-dimensional (3D) Lagrangian displacement. Spatial gradients of the Lagrangian displacement field are used to measure regional myocardial strain. In general, adjacent parallel slices are needed in order to calculate the spatial gradient in the through-slice direction. This necessitates the acquisition of additional data and prolongs the scan time. The goal of this study is to define an analytic solution that supports the reconstruction of the out-of-plane components of the Lagrangian strain tensor in addition to the in-plane components from a single-slice displacement CMR dataset with high spatio-temporal resolution. The technique assumes incompressibility of the myocardium as a physical constraint. Results: The feasibility of the method is demonstrated in a healthy human subject and the results are compared to those of other studies. The proposed method was validated with simulated data and strain estimates from experimentally measured DENSE data, which were compared to the strain calculation from a conventional two-slice acquisition. Conclusion: This analytical method reduces the need to acquire data from adjacent slices when calculating regional Lagrangian strains and can effectively reduce the long scan time by a factor of two

Robust and Efficient Recovery of Rigid Motion from Subspace Constraints Solved using Recursive Identification of Nonlinear Implicit Systems

The problem of estimating rigid motion from projections may be characterized using a nonlinear dynamical system, composed of the rigid motion transformation and the perspective map. The time derivative of the output of such a system, which is also called the "motion field", is bilinear in the motion parameters, and may be used to specify a subspace constraint on either the direction of translation or the inverse depth of the observed points. Estimating motion may then be formulated as an optimization task constrained on such a subspace. Heeger and Jepson [5], who first introduced this constraint, solve the optimization task using an extensive search over the possible directions of translation. We reformulate the optimization problem in a systems theoretic framework as the the identification of a dynamic system in exterior differential form with parameters on a differentiable manifold, and use techniques which pertain to nonlinear estimation and identification theory to perform the optimization task in a principled manner. The general technique for addressing such identification problems [14] has been used successfully in addressing other problems in computational vision [13, 12]. The application of the general method [14] results in a recursive and pseudo-optimal solution of the motion problem, which has robustness properties far superior to other existing techniques we have implemented. By releasing the constraint that the visible points lie in front of the observer, we may explain some psychophysical effects on the nonrigid percept of rigidly moving shapes. Experiments on real and synthetic image sequences show very promising results in terms of robustness, accuracy and computational efficiency