Three New Algorithms for Projective Bundle Adjustment with Minimum Parameters

Bundle adjustment is a technique used to compute the maximum likelihood estimate of structure and motion from image feature correspondences. It practice, large non-linear systems have to be solved, most of the time using an iterative optimization process starting from a sub-optimal solution obtained by using linear methods. The behaviour, in terms of convergence, and the computational cost of this process depend on the parameterization used to represent the problem, i.e. of structure and motion. In this paper, we address the problem of finding a minimal parameterization of projective structure and motion, i.e. when camera calibration is not available. Most of the existing parameterizations are either sub-optimal, in the sense that they do change the cost function, or quite complicated to implement, requiring different closed-form expressions, also called maps, to model every case. Without loss of generality, we restrict the problem to the minimal parameterization of the two-view projective motion, equivalent to the fundamental matrix. We propose to classify existing ways allowing to obtain a minimal set of parameters into three categories and present three new algorithms, one for each category. These algorithms are simple to implement and yield minimal parameterizations. We also address the problem of minimally parameterizing homogeneous entities. We present what we call mapped coordinates, a tool allowing to optimize homogeneous entities over a minimal number of parameters. We make extensive use of this tool for both structure and motion parameterization. We compare these algorithms with existing ones using both simulated data and real images. In the light of these experiments, it appears that the new algorithms perform better than existing ones in terms of computational cost while achieving equivalent performances in terms of convergence

Automated Top View Registration of Broadcast Football Videos

In this paper, we propose a novel method to register football broadcast video frames on the static top view model of the playing surface. The proposed method is fully automatic in contrast to the current state of the art which requires manual initialization of point correspondences between the image and the static model. Automatic registration using existing approaches has been difficult due to the lack of sufficient point correspondences. We investigate an alternate approach exploiting the edge information from the line markings on the field. We formulate the registration problem as a nearest neighbour search over a synthetically generated dictionary of edge map and homography pairs. The synthetic dictionary generation allows us to exhaustively cover a wide variety of camera angles and positions and reduce this problem to a minimal per-frame edge map matching procedure. We show that the per-frame results can be improved in videos using an optimization framework for temporal camera stabilization. We demonstrate the efficacy of our approach by presenting extensive results on a dataset collected from matches of football World Cup 2014

Calipso: Physics-based Image and Video Editing through CAD Model Proxies

We present Calipso, an interactive method for editing images and videos in a physically-coherent manner. Our main idea is to realize physics-based manipulations by running a full physics simulation on proxy geometries given by non-rigidly aligned CAD models. Running these simulations allows us to apply new, unseen forces to move or deform selected objects, change physical parameters such as mass or elasticity, or even add entire new objects that interact with the rest of the underlying scene. In Calipso, the user makes edits directly in 3D; these edits are processed by the simulation and then transfered to the target 2D content using shape-to-image correspondences in a photo-realistic rendering process. To align the CAD models, we introduce an efficient CAD-to-image alignment procedure that jointly minimizes for rigid and non-rigid alignment while preserving the high-level structure of the input shape. Moreover, the user can choose to exploit image flow to estimate scene motion, producing coherent physical behavior with ambient dynamics. We demonstrate Calipso's physics-based editing on a wide range of examples producing myriad physical behavior while preserving geometric and visual consistency.Comment: 11 page

Bridging the computational gap between mesoscopic and continuum modeling of red blood cells for fully resolved blood flow

We present a computational framework for the simulation of blood flow with fully resolved red blood cells (RBCs) using a modular approach that consists of a lattice Boltzmann solver for the blood plasma, a novel finite element based solver for the deformable bodies and an immersed boundary method for the fluid-solid interaction. For the RBCs, we propose a nodal projective FEM (npFEM) solver which has theoretical advantages over the more commonly used mass-spring systems (mesoscopic modeling), such as an unconditional stability, versatile material expressivity, and one set of parameters to fully describe the behavior of the body at any mesh resolution. At the same time, the method is substantially faster than other FEM solvers proposed in this field, and has an efficiency that is comparable to the one of mesoscopic models. At its core, the solver uses specially defined potential energies, and builds upon them a fast iterative procedure based on quasi-Newton techniques. For a known material, our solver has only one free parameter that demands tuning, related to the body viscoelasticity. In contrast, state-of-the-art solvers for deformable bodies have more free parameters, and the calibration of the models demands special assumptions regarding the mesh topology, which restrict their generality and mesh independence. We propose as well a modification to the potential energy proposed by Skalak et al. 1973 for the red blood cell membrane, which enhances the strain hardening behavior at higher deformations. Our viscoelastic model for the red blood cell, while simple enough and applicable to any kind of solver as a post-convergence step, can capture accurately the characteristic recovery time and tank-treading frequencies. The framework is validated using experimental data, and it proves to be scalable for multiple deformable bodies

Autocalibration with the Minimum Number of Cameras with Known Pixel Shape

In 3D reconstruction, the recovery of the calibration parameters of the cameras is paramount since it provides metric information about the observed scene, e.g., measures of angles and ratios of distances. Autocalibration enables the estimation of the camera parameters without using a calibration device, but by enforcing simple constraints on the camera parameters. In the absence of information about the internal camera parameters such as the focal length and the principal point, the knowledge of the camera pixel shape is usually the only available constraint. Given a projective reconstruction of a rigid scene, we address the problem of the autocalibration of a minimal set of cameras with known pixel shape and otherwise arbitrarily varying intrinsic and extrinsic parameters. We propose an algorithm that only requires 5 cameras (the theoretical minimum), thus halving the number of cameras required by previous algorithms based on the same constraint. To this purpose, we introduce as our basic geometric tool the six-line conic variety (SLCV), consisting in the set of planes intersecting six given lines of 3D space in points of a conic. We show that the set of solutions of the Euclidean upgrading problem for three cameras with known pixel shape can be parameterized in a computationally efficient way. This parameterization is then used to solve autocalibration from five or more cameras, reducing the three-dimensional search space to a two-dimensional one. We provide experiments with real images showing the good performance of the technique.Comment: 19 pages, 14 figures, 7 tables, J. Math. Imaging Vi