Event-based Motion Segmentation with Spatio-Temporal Graph Cuts

Identifying independently moving objects is an essential task for dynamic scene understanding. However, traditional cameras used in dynamic scenes may suffer from motion blur or exposure artifacts due to their sampling principle. By contrast, event-based cameras are novel bio-inspired sensors that offer advantages to overcome such limitations. They report pixelwise intensity changes asynchronously, which enables them to acquire visual information at exactly the same rate as the scene dynamics. We develop a method to identify independently moving objects acquired with an event-based camera, i.e., to solve the event-based motion segmentation problem. We cast the problem as an energy minimization one involving the fitting of multiple motion models. We jointly solve two subproblems, namely event cluster assignment (labeling) and motion model fitting, in an iterative manner by exploiting the structure of the input event data in the form of a spatio-temporal graph. Experiments on available datasets demonstrate the versatility of the method in scenes with different motion patterns and number of moving objects. The evaluation shows state-of-the-art results without having to predetermine the number of expected moving objects. We release the software and dataset under an open source licence to foster research in the emerging topic of event-based motion segmentation

Object-based video representations: shape compression and object segmentation

Object-based video representations are considered to be useful for easing the process of multimedia content production and enhancing user interactivity in multimedia productions. Object-based video presents several new technical challenges, however. Firstly, as with conventional video representations, compression of the video data is a requirement. For object-based representations, it is necessary to compress the shape of each video object as it moves in time. This amounts to the compression of moving binary images. This is achieved by the use of a technique called context-based arithmetic encoding. The technique is utilised by applying it to rectangular pixel blocks and as such it is consistent with the standard tools of video compression. The blockbased application also facilitates well the exploitation of temporal redundancy in the sequence of binary shapes. For the first time, context-based arithmetic encoding is used in conjunction with motion compensation to provide inter-frame compression. The method, described in this thesis, has been thoroughly tested throughout the MPEG-4 core experiment process and due to favourable results, it has been adopted as part of the MPEG-4 video standard. The second challenge lies in the acquisition of the video objects. Under normal conditions, a video sequence is captured as a sequence of frames and there is no inherent information about what objects are in the sequence, not to mention information relating to the shape of each object. Some means for segmenting semantic objects from general video sequences is required. For this purpose, several image analysis tools may be of help and in particular, it is believed that video object tracking algorithms will be important. A new tracking algorithm is developed based on piecewise polynomial motion representations and statistical estimation tools, e.g. the expectationmaximisation method and the minimum description length principle

Model Selection for Geometric Fitting: Geometric Ale and Geometric MDL

Contrasting "geometric fitting", for which the noise level is taken as the asymptotic variable, with "statistical inference", for which the number of observations is taken as the asymptotic variable, we give a new definition of the "geometric AIC" and the "geometric MDL" as the counterparts of Akaike's AIC and Rissanen's MDL. We discuss various theoretical and practical problems that emerge from our analysis. Finally, we show, doing experiments using synthetic and real images, that the geometric MDL does not necessarily outperform the geometric AIC and that the two criteria have very different characteristics

Geometric BIC

The author introduced the "geometric AIC" and the "geometric MDL" as model selection criteria for geometric fitting problems. These correspond to Akaike’s "AIC" and Rissanen's "BIC", respectively, well known in the statistical estimation framework. Another criterion well known is Schwarz’ "BIC", but its counterpart for geometric fitting has been unknown. This paper introduces the corresponding criterion, which we call the "geometric BIC", and shows that it is of the same form as the geometric MDL. We present the underlying logical reasoning of Bayesian estimation

A New Automatic Method to Identify Galaxy Mergers I. Description and Application to the STAGES Survey

We present an automatic method to identify galaxy mergers using the morphological information contained in the residual images of galaxies after the subtraction of a Sersic model. The removal of the bulk signal from the host galaxy light is done with the aim of detecting the fainter minor mergers. The specific morphological parameters that are used in the merger diagnostic suggested here are the Residual Flux Fraction and the asymmetry of the residuals. The new diagnostic has been calibrated and optimized so that the resulting merger sample is very complete. However, the contamination by non-mergers is also high. If the same optimization method is adopted for combinations of other structural parameters such as the CAS system, the merger indicator we introduce yields merger samples of equal or higher statistical quality than the samples obtained through the use of other structural parameters. We explore the ability of the method presented here to select minor mergers by identifying a sample of visually classified mergers that would not have been picked up by the use of the CAS system, when using its usual limits. Given the low prevalence of mergers among the general population of galaxies and the optimization used here, we find that the merger diagnostic introduced in this work is best used as a negative merger test, i.e., it is very effective at selecting non-merging galaxies. As with all the currently available automatic methods, the sample of merger candidates selected is contaminated by non-mergers, and further steps are needed to produce a clean sample. This merger diagnostic has been developed using the HST/ACS F606W images of the A901/02 cluster (z=0.165) obtained by the STAGES team. In particular, we have focused on a mass and magnitude limited sample (log M/M_{O}>9.0, R_{Vega}<23.5mag)) which includes 905 cluster galaxies and 655 field galaxies of all morphological types.Comment: 25 pages, 14 figures, 4 tables. To appear in MNRA

Uncertainty Modeling and Geometric Inference

We investigate the meaning of "statistical methods" for geometric inference based on image feature points. Tracing back the origin of feature uncertainty to image processing operations, we discuss the implications of asymptotic analysis in reference to "geometric fitting" and "geometric model selection", We point out that a correspondence exists between the standard statistical analysis and the geometric inference problem. We also compare the capability of the "geometric AIC" and the "geometric MDL' in detecting degeneracy. Next, we review recent progress in geometric fitting techniques for linear constraints, describing the "FNS method", the "HEIV method", the "renormalization method", and other related techniques. Finally, we discuss the "Neyman-Scott problem" and "semiparametric models" in relation to geometric inference. We conclude that applications of statistical methods requires careful considerations about the nature of the problem in question

Uncertainty modeling and model selection for geometric inference

We first investigate the meaning of &#34;statistical methods&#34; for geometric inference based on image feature points. Tracing back the origin of feature uncertainty to image processing operations, we discuss the implications of asymptotic analysis in reference to &#34;geometric fitting&#34; and &#34;geometric model selection&#34; and point out that a correspondence exists between the standard statistical analysis and the geometric inference problem. Then, we derive the &#34;geometric AIC&#34; and the &#34;geometric MDL&#34; as counterparts of Akaike's AIC and Rissanen's MDL. We show by experiments that the two criteria have contrasting characteristics in detecting degeneracy. </p

Energy Based Multi-Model Fitting and Matching Problems

Feature matching and model fitting are fundamental problems in multi-view geometry. They are chicken-&-egg problems: if models are known it is easier to find matches and vice versa. Standard multi-view geometry techniques sequentially solve feature matching and model fitting as two independent problems after making fairly restrictive assumptions. For example, matching methods rely on strong discriminative power of feature descriptors, which fail for stereo images with repetitive textures or wide baseline. Also, model fitting methods assume given feature matches, which are not known a priori. Moreover, when data supports multiple models the fitting problem becomes challenging even with known matches and current methods commonly use heuristics. One of the main contributions of this thesis is a joint formulation of fitting and matching problems. We are first to introduce an objective function combining both matching and multi-model estimation. We also propose an approximation algorithm for the corresponding NP-hard optimization problem using block-coordinate descent with respect to matching and model fitting variables. For fixed models, our method uses min-cost-max-flow based algorithm to solve a generalization of a linear assignment problem with label cost (sparsity constraint). Fixed matching case reduces to multi-model fitting subproblem, which is interesting in its own right. In contrast to standard heuristic approaches, we introduce global objective functions for multi-model fitting using various forms of regularization (spatial smoothness and sparsity) and propose a graph-cut based optimization algorithm, PEaRL. Experimental results show that our proposed mathematical formulations and optimization algorithms improve the accuracy and robustness of model estimation over the state-of-the-art in computer vision