Real-time 3D reconstruction of non-rigid shapes with a single moving camera

© . This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/This paper describes a real-time sequential method to simultaneously recover the camera motion and the 3D shape of deformable objects from a calibrated monocular video. For this purpose, we consider the Navier-Cauchy equations used in 3D linear elasticity and solved by finite elements, to model the time-varying shape per frame. These equations are embedded in an extended Kalman filter, resulting in sequential Bayesian estimation approach. We represent the shape, with unknown material properties, as a combination of elastic elements whose nodal points correspond to salient points in the image. The global rigidity of the shape is encoded by a stiffness matrix, computed after assembling each of these elements. With this piecewise model, we can linearly relate the 3D displacements with the 3D acting forces that cause the object deformation, assumed to be normally distributed. While standard finite-element-method techniques require imposing boundary conditions to solve the resulting linear system, in this work we eliminate this requirement by modeling the compliance matrix with a generalized pseudoinverse that enforces a pre-fixed rank. Our framework also ensures surface continuity without the need for a post-processing step to stitch all the piecewise reconstructions into a global smooth shape. We present experimental results using both synthetic and real videos for different scenarios ranging from isometric to elastic deformations. We also show the consistency of the estimation with respect to 3D ground truth data, include several experiments assessing robustness against artifacts and finally, provide an experimental validation of our performance in real time at frame rate for small mapsPeer ReviewedPostprint (author's final draft

Calipso: Physics-based Image and Video Editing through CAD Model Proxies

We present Calipso, an interactive method for editing images and videos in a physically-coherent manner. Our main idea is to realize physics-based manipulations by running a full physics simulation on proxy geometries given by non-rigidly aligned CAD models. Running these simulations allows us to apply new, unseen forces to move or deform selected objects, change physical parameters such as mass or elasticity, or even add entire new objects that interact with the rest of the underlying scene. In Calipso, the user makes edits directly in 3D; these edits are processed by the simulation and then transfered to the target 2D content using shape-to-image correspondences in a photo-realistic rendering process. To align the CAD models, we introduce an efficient CAD-to-image alignment procedure that jointly minimizes for rigid and non-rigid alignment while preserving the high-level structure of the input shape. Moreover, the user can choose to exploit image flow to estimate scene motion, producing coherent physical behavior with ambient dynamics. We demonstrate Calipso's physics-based editing on a wide range of examples producing myriad physical behavior while preserving geometric and visual consistency.Comment: 11 page

Discrete Geometric Structures in Homogenization and Inverse Homogenization with application to EIT

We introduce a new geometric approach for the homogenization and inverse homogenization of the divergence form elliptic operator with rough conductivity coefficients $\sigma(x)$ in dimension two. We show that conductivity coefficients are in one-to-one correspondence with divergence-free matrices and convex functions $s(x)$ over the domain $\Omega$. Although homogenization is a non-linear and non-injective operator when applied directly to conductivity coefficients, homogenization becomes a linear interpolation operator over triangulations of $\Omega$ when re-expressed using convex functions, and is a volume averaging operator when re-expressed with divergence-free matrices. Using optimal weighted Delaunay triangulations for linearly interpolating convex functions, we obtain an optimally robust homogenization algorithm for arbitrary rough coefficients. Next, we consider inverse homogenization and show how to decompose it into a linear ill-posed problem and a well-posed non-linear problem. We apply this new geometric approach to Electrical Impedance Tomography (EIT). It is known that the EIT problem admits at most one isotropic solution. If an isotropic solution exists, we show how to compute it from any conductivity having the same boundary Dirichlet-to-Neumann map. It is known that the EIT problem admits a unique (stable with respect to $G$-convergence) solution in the space of divergence-free matrices. As such we suggest that the space of convex functions is the natural space in which to parameterize solutions of the EIT problem

Shape basis interpretation for monocular deformable 3D reconstruction

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.In this paper, we propose a novel interpretable shape model to encode object non-rigidity. We first use the initial frames of a monocular video to recover a rest shape, used later to compute a dissimilarity measure based on a distance matrix measurement. Spectral analysis is then applied to this matrix to obtain a reduced shape basis, that in contrast to existing approaches, can be physically interpreted. In turn, these pre-computed shape bases are used to linearly span the deformation of a wide variety of objects. We introduce the low-rank basis into a sequential approach to recover both camera motion and non-rigid shape from the monocular video, by simply optimizing the weights of the linear combination using bundle adjustment. Since the number of parameters to optimize per frame is relatively small, specially when physical priors are considered, our approach is fast and can potentially run in real time. Validation is done in a wide variety of real-world objects, undergoing both inextensible and extensible deformations. Our approach achieves remarkable robustness to artifacts such as noisy and missing measurements and shows an improved performance to competing methods.Peer ReviewedPostprint (author's final draft

NASTRAN cyclic symmetry capability

A development for NASTRAN which facilitates the analysis of structures made up of identical segments symmetrically arranged with respect to an axis is described. The key operation in the method is the transformation of the degrees of freedom for the structure into uncoupled symmetrical components, thereby greatly reducing the number of equations which are solved simultaneously. A further reduction occurs if each segment has a plane of reflective symmetry. The only required assumption is that the problem be linear. The capability, as developed, will be available in level 16 of NASTRAN for static stress analysis, steady state heat transfer analysis, and vibration analysis. The paper includes a discussion of the theory, a brief description of the data supplied by the user, and the results obtained for two example problems. The first problem concerns the acoustic modes of a long prismatic cavity imbedded in the propellant grain of a solid rocket motor. The second problem involves the deformations of a large space antenna. The latter example is the first application of the NASTRAN Cyclic Symmetry capability to a really large problem