Sequential non-rigid structure from motion using physical priors

© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.We propose a new approach to simultaneously recover camera pose and 3D shape of non-rigid and potentially extensible surfaces from a monocular image sequence. For this purpose, we make use of the Extended Kalman Filter based Simultaneous Localization And Mapping (EKF-SLAM) formulation, a Bayesian optimization framework traditionally used in mobile robotics for estimating camera pose and reconstructing rigid scenarios. In order to extend the problem to a deformable domain we represent the object's surface mechanics by means of Navier's equations, which are solved using a Finite Element Method (FEM). With these main ingredients, we can further model the material's stretching, allowing us to go a step further than most of current techniques, typically constrained to surfaces undergoing isometric deformations. We extensively validate our approach in both real and synthetic experiments, and demonstrate its advantages with respect to competing methods. More specifically, we show that besides simultaneously retrieving camera pose and non-rigid shape, our approach is adequate for both isometric and extensible surfaces, does not require neither batch processing all the frames nor tracking points over the whole sequence and runs at several frames per second.Peer ReviewedPostprint (author's final draft

Real-time 3D reconstruction of non-rigid shapes with a single moving camera

© . This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/This paper describes a real-time sequential method to simultaneously recover the camera motion and the 3D shape of deformable objects from a calibrated monocular video. For this purpose, we consider the Navier-Cauchy equations used in 3D linear elasticity and solved by finite elements, to model the time-varying shape per frame. These equations are embedded in an extended Kalman filter, resulting in sequential Bayesian estimation approach. We represent the shape, with unknown material properties, as a combination of elastic elements whose nodal points correspond to salient points in the image. The global rigidity of the shape is encoded by a stiffness matrix, computed after assembling each of these elements. With this piecewise model, we can linearly relate the 3D displacements with the 3D acting forces that cause the object deformation, assumed to be normally distributed. While standard finite-element-method techniques require imposing boundary conditions to solve the resulting linear system, in this work we eliminate this requirement by modeling the compliance matrix with a generalized pseudoinverse that enforces a pre-fixed rank. Our framework also ensures surface continuity without the need for a post-processing step to stitch all the piecewise reconstructions into a global smooth shape. We present experimental results using both synthetic and real videos for different scenarios ranging from isometric to elastic deformations. We also show the consistency of the estimation with respect to 3D ground truth data, include several experiments assessing robustness against artifacts and finally, provide an experimental validation of our performance in real time at frame rate for small mapsPeer ReviewedPostprint (author's final draft

Extended Object Tracking: Introduction, Overview and Applications

This article provides an elaborate overview of current research in extended object tracking. We provide a clear definition of the extended object tracking problem and discuss its delimitation to other types of object tracking. Next, different aspects of extended object modelling are extensively discussed. Subsequently, we give a tutorial introduction to two basic and well used extended object tracking approaches - the random matrix approach and the Kalman filter-based approach for star-convex shapes. The next part treats the tracking of multiple extended objects and elaborates how the large number of feasible association hypotheses can be tackled using both Random Finite Set (RFS) and Non-RFS multi-object trackers. The article concludes with a summary of current applications, where four example applications involving camera, X-band radar, light detection and ranging (lidar), red-green-blue-depth (RGB-D) sensors are highlighted.Comment: 30 pages, 19 figure

A bayesian approach to simultaneously recover camera pose and non-rigid shape from monocular images

© . This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/In this paper we bring the tools of the Simultaneous Localization and Map Building (SLAM) problem from a rigid to a deformable domain and use them to simultaneously recover the 3D shape of non-rigid surfaces and the sequence of poses of a moving camera. Under the assumption that the surface shape may be represented as a weighted sum of deformation modes, we show that the problem of estimating the modal weights along with the camera poses, can be probabilistically formulated as a maximum a posteriori estimate and solved using an iterative least squares optimization. In addition, the probabilistic formulation we propose is very general and allows introducing different constraints without requiring any extra complexity. As a proof of concept, we show that local inextensibility constraints that prevent the surface from stretching can be easily integrated. An extensive evaluation on synthetic and real data, demonstrates that our method has several advantages over current non-rigid shape from motion approaches. In particular, we show that our solution is robust to large amounts of noise and outliers and that it does not need to track points over the whole sequence nor to use an initialization close from the ground truth.Peer ReviewedPostprint (author's final draft

Real-time Loss Estimation for Instrumented Buildings

Motivation. A growing number of buildings have been instrumented to measure and record earthquake motions and to transmit these records to seismic-network data centers to be archived and disseminated for research purposes. At the same time, sensors are growing smaller, less expensive to install, and capable of sensing and transmitting other environmental parameters in addition to acceleration. Finally, recently developed performance-based earthquake engineering methodologies employ structural-response information to estimate probabilistic repair costs, repair durations, and other metrics of seismic performance. The opportunity presents itself therefore to combine these developments into the capability to estimate automatically in near-real-time the probabilistic seismic performance of an instrumented building, shortly after the cessation of strong motion. We refer to this opportunity as (near-) real-time loss estimation (RTLE). Methodology. This report presents a methodology for RTLE for instrumented buildings. Seismic performance is to be measured in terms of probabilistic repair cost, precise location of likely physical damage, operability, and life-safety. The methodology uses the instrument recordings and a Bayesian state-estimation algorithm called a particle filter to estimate the probabilistic structural response of the system, in terms of member forces and deformations. The structural response estimate is then used as input to component fragility functions to estimate the probabilistic damage state of structural and nonstructural components. The probabilistic damage state can be used to direct structural engineers to likely locations of physical damage, even if they are concealed behind architectural finishes. The damage state is used with construction cost-estimation principles to estimate probabilistic repair cost. It is also used as input to a quantified, fuzzy-set version of the FEMA-356 performance-level descriptions to estimate probabilistic safety and operability levels. CUREE demonstration building. The procedure for estimating damage locations, repair costs, and post-earthquake safety and operability is illustrated in parallel demonstrations by CUREE and Kajima research teams. The CUREE demonstration is performed using a real 1960s-era, 7-story, nonductile reinforced-concrete moment-frame building located in Van Nuys, California. The building is instrumented with 16 channels at five levels: ground level, floors 2, 3, 6, and the roof. We used the records obtained after the 1994 Northridge earthquake to hindcast performance in that earthquake. The building is analyzed in its condition prior to the 1994 Northridge Earthquake. It is found that, while hindcasting of the overall system performance level was excellent, prediction of detailed damage locations was poor, implying that either actual conditions differed substantially from those shown on the structural drawings, or inappropriate fragility functions were employed, or both. We also found that Bayesian updating of the structural model using observed structural response above the base of the building adds little information to the performance prediction. The reason is probably that Real-Time Loss Estimation for Instrumented Buildings ii structural uncertainties have only secondary effect on performance uncertainty, compared with the uncertainty in assembly damageability as quantified by their fragility functions. The implication is that real-time loss estimation is not sensitive to structural uncertainties (saving costly multiple simulations of structural response), and that real-time loss estimation does not benefit significantly from installing measuring instruments other than those at the base of the building. Kajima demonstration building. The Kajima demonstration is performed using a real 1960s-era office building in Kobe, Japan. The building, a 7-story reinforced-concrete shearwall building, was not instrumented in the 1995 Kobe earthquake, so instrument recordings are simulated. The building is analyzed in its condition prior to the earthquake. It is found that, while hindcasting of the overall repair cost was excellent, prediction of detailed damage locations was poor, again implying either that as-built conditions differ substantially from those shown on structural drawings, or that inappropriate fragility functions were used, or both. We find that the parameters of the detailed particle filter needed significant tuning, which would be impractical in actual application. Work is needed to prescribe values of these parameters in general. Opportunities for implementation and further research. Because much of the cost of applying this RTLE algorithm results from the cost of instrumentation and the effort of setting up a structural model, the readiest application would be to instrumented buildings whose structural models are already available, and to apply the methodology to important facilities. It would be useful to study under what conditions RTLE would be economically justified. Two other interesting possibilities for further study are (1) to update performance using readily observable damage; and (2) to quantify the value of information for expensive inspections, e.g., if one inspects a connection with a modeled 50% failure probability and finds that the connect is undamaged, is it necessary to examine one with 10% failure probability

Modal space: a physics-based model for sequential estimation of time-varying shape from monocular video

The final publication is available at link.springer.comThis paper describes two sequential methods for recovering the camera pose together with the 3D shape of highly deformable surfaces from a monocular video. The nonrigid 3D shape is modeled as a linear combination of mode shapes with time-varying weights that define the shape at each frame and are estimated on-the-fly. The low-rank constraint is combined with standard smoothness priors to optimize the model parameters over a sliding window of image frames. We propose to obtain a physics-based shape basis using the initial frames on the video to code the time-varying shape along the sequence, reducing the problem from trilinear to bilinear. To this end, the 3D shape is discretized by means of a soup of elastic triangular finite elements where we apply a force balance equation. This equation is solved using modal analysis via a simple eigenvalue problem to obtain a shape basis that encodes the modes of deformation. Even though this strategy can be applied in a wide variety of scenarios, when the observations are denser, the solution can become prohibitive in terms of computational load. We avoid this limitation by proposing two efficient coarse-to-fine approaches that allow us to easily deal with dense 3D surfaces. This results in a scalable solution that estimates a small number of parameters per frame and could potentially run in real time. We show results on both synthetic and real videos with ground truth 3D data, while robustly dealing with artifacts such as noise and missing data.Peer ReviewedPostprint (author's final draft

6-DoF Navigation Systems for Autonomous Underwater Vehicles

Austri