Learning monocular 3D reconstruction of articulated categories from motion

Monocular 3D reconstruction of articulated object categories is challenging due to the lack of training data and the inherent ill-posedness of the problem. In this work we use video self-supervision, forcing the consistency of consecutive 3D reconstructions by a motion-based cycle loss. This largely improves both optimization-based and learning-based 3D mesh reconstruction. We further introduce an interpretable model of 3D template deformations that controls a 3D surface through the displacement of a small number of local, learnable handles. We formulate this operation as a structured layer relying on mesh-laplacian regularization and show that it can be trained in an end-to-end manner. We finally introduce a per-sample numerical optimisation approach that jointly optimises over mesh displacements and cameras within a video, boosting accuracy both for training and also as test time post-processing. While relying exclusively on a small set of videos collected per category for supervision, we obtain state-of-the-art reconstructions with diverse shapes, viewpoints and textures for multiple articulated object categories.Comment: For project website see https://fkokkinos.github.io/video_3d_reconstruction

Data-driven weight optimization for real-time mesh deformation

3D model deformation has been an active research topic in geometric processing. Due to its efficiency, linear blend skinning (LBS) and its follow-up methods are widely used in practical applications as an efficient method for deforming vector images, geometric models and animated characters. LBS needs to determine the control handles and specify their influence weights, which requires expertise and is time-consuming. Further studies have proposed a method for efficiently calculating bounded biharmonic weights of given control handles which reduces user effort and produces smooth deformation results. The algorithm defines a high-order shape-aware smoothness function which tends to produce smooth deformation results, but fails to generate locally rigid deformations. To address this, we propose a novel data-driven approach to producing improved weights for handles that makes full use of available 3D model data by optimizing an energy consisting of data-driven, rigidity and sparsity terms, while maintaining its advantage of allowing handles of various forms. We further devise an efficient iterative optimization scheme. Through contrast experiments, it clearly shows that linear blend skinning based on our optimized weights better reflects the deformation characteristics of the model, leading to more accurate deformation results, outperforming existing methods. The method also retains real-time performance even with a large number of deformation examples. Our ablation experiments also show that each energy term is essential

Doctor of Philosophy in Computing

dissertationPhysics-based animation has proven to be a powerful tool for creating compelling animations for film and games. Most techniques in graphics are based on methods developed for predictive simulation for engineering applications; however, the goals for graphics applications are dramatically different than the goals of engineering applications. As a result, most physics-based animation tools are difficult for artists to work with, providing little direct control over simulation results. In this thesis, we describe tools for physics-based animation designed with artist needs and expertise in mind. Most materials can be modeled as elastoplastic: they recover from small deformations, but large deformations permanently alter their rest shape. Unfortunately, large plastic deformations, common in graphical applications, cause simulation instabilities if not addressed. Most elastoplastic simulation techniques in graphics rely on a finite-element approach where objects are discretized into a tetrahedral mesh. Using these approaches, maintaining simulation stability during large plastic flows requires remeshing, a complex and computationally expensive process. We introduce a new point-based approach that does not rely on an explicit mesh and avoids the expense of remeshing. Our approach produces comparable results with much lower implementation complexity. Points are a ubiquitous primitive for many effects, so our approach also integrates well with existing artist pipelines. Next, we introduce a new technique for animating stylized images which we call Dynamic Sprites. Artists can use our tool to create digital assets that interact in a natural, but stylized, way in virtual environments. In order to support the types of nonphysical, exaggerated motions often desired by artists, our approach relies on a heavily modified deformable body simulator, equipped with a set of new intuitive controls and an example-based deformation model. Our approach allows artists to specify how the shape of the object should change as it moves and collides in interactive virtual environments. Finally, we introduce a new technique for animating destructive scenes. Our approach is built on the insight that the most important visual aspects of destruction are plastic deformation and fracture. Like with Dynamic Sprites, we use an example-based model of deformation for intuitive artist control. Our simulator treats objects as rigid when computing dynamics but allows them to deform plastically and fracture in between timesteps based on interactions with the other objects. We demonstrate that our approach can efficiently animate the types of destructive scenes common in film and games. These animation techniques are designed to exploit artist expertise to ease creation of complex animations. By using artist-friendly primitives and allowing artists to provide characteristic deformations as input, our techniques enable artists to create more compelling animations, more easily

Computational design of an automotive twist beam

International audienceIn recent years, the automotive industry has known a remarkable development in order to satisfy the customer requirements. In this paper, we will study one of the components of the automotive which is the twist beam. The study is focused on the multicriteria design of the automotive twist beam undergoing linear elastic deformation (Hooke's law). Indeed, for the design of this automotive part, there are some criteria to be considered as the rigidity (stiffness) and the resistance to fatigue. Those two criteria are known to be conflicting, therefore, our aim is to identify the Pareto front of this problem. To do this, we used a Normal Boundary Intersection (NBI) algorithm coupling with a radial basis function (RBF) metamodel in order to reduce the high calculation time needed for solving the multicriteria design problem. Otherwise, we used the free form deformation (FFD) technique for the generation of the 3D shapes of the automotive part studied during the optimization process