Deformable and articulated 3D reconstruction from monocular video sequences

PhDThis thesis addresses the problem of deformable and articulated structure from motion from monocular uncalibrated video sequences. Structure from motion is defined as the problem of recovering information about the 3D structure of scenes imaged by a camera in a video sequence. Our study aims at the challenging problem of non-rigid shapes (e.g. a beating heart or a smiling face). Non-rigid structures appear constantly in our everyday life, think of a bicep curling, a torso twisting or a smiling face. Our research seeks a general method to perform 3D shape recovery purely from data, without having to rely on a pre-computed model or training data. Open problems in the field are the difficulty of the non-linear estimation, the lack of a real-time system, large amounts of missing data in real-world video sequences, measurement noise and strong deformations. Solving these problems would take us far beyond the current state of the art in non-rigid structure from motion. This dissertation presents our contributions in the field of non-rigid structure from motion, detailing a novel algorithm that enforces the exact metric structure of the problem at each step of the minimisation by projecting the motion matrices onto the correct deformable or articulated metric motion manifolds respectively. An important advantage of this new algorithm is its ability to handle missing data which becomes crucial when dealing with real video sequences. We present a generic bilinear estimation framework, which improves convergence and makes use of the manifold constraints. Finally, we demonstrate a sequential, frame-by-frame estimation algorithm, which provides a 3D model and camera parameters for each video frame, while simultaneously building a model of object deformation

Periodic Body-And-Bar Frameworks

Periodic body-and-bar frameworks are abstractions of crystalline structures made of rigid bodies connected by fixed-length bars and subject to the action of a lattice of translations. We give a Maxwell–Laman characterization for minimally rigid periodic body-and-bar frameworks in terms of their quotient graphs. As a consequence we obtain efficient polynomial time algorithms for their recognition based on matroid partition and pebble games

Linking Rigid Bodies Symmetrically

The mathematical theory of rigidity of body-bar and body-hinge frameworks provides a useful tool for analyzing the rigidity and flexibility of many articulated structures appearing in engineering, robotics and biochemistry. In this paper we develop a symmetric extension of this theory which permits a rigidity analysis of body-bar and body-hinge structures with point group symmetries. The infinitesimal rigidity of body-bar frameworks can naturally be formulated in the language of the exterior (or Grassmann) algebra. Using this algebraic formulation, we derive symmetry-adapted rigidity matrices to analyze the infinitesimal rigidity of body-bar frameworks with Abelian point group symmetries in an arbitrary dimension. In particular, from the patterns of these new matrices, we derive combinatorial characterizations of infinitesimally rigid body-bar frameworks which are generic with respect to a point group of the form $\mathbb{Z}/2\mathbb{Z}\times \dots \times \mathbb{Z}/2\mathbb{Z}$. Our characterizations are given in terms of packings of bases of signed-graphic matroids on quotient graphs. Finally, we also extend our methods and results to body-hinge frameworks with Abelian point group symmetries in an arbitrary dimension. As special cases of these results, we obtain combinatorial characterizations of infinitesimally rigid body-hinge frameworks with $\mathcal{C}_2$ or $\mathcal{D}_2$ symmetry - the most common symmetry groups found in proteins.Comment: arXiv:1308.6380 version 1 was split into two papers. The version 2 of arXiv:1308.6380 consists of Sections 1 - 6 of the version 1. This paper is based on the second part of the version 1 (Sections 7 and 8

Computing fast search heuristics for physics-based mobile robot motion planning

Mobile robots are increasingly being employed to assist responders in search and rescue missions. Robots have to navigate in dangerous areas such as collapsed buildings and hazardous sites, which can be inaccessible to humans. Tele-operating the robots can be stressing for the human operators, which are also overloaded with mission tasks and coordination overhead, so it is important to provide the robot with some degree of autonomy, to lighten up the task for the human operator and also to ensure robot safety. Moving robots around requires reasoning, including interpretation of the environment, spatial reasoning, planning of actions (motion), and execution. This is particularly challenging when the environment is unstructured, and the terrain is \textit{harsh}, i.e. not flat and cluttered with obstacles. Approaches reducing the problem to a 2D path planning problem fall short, and many of those who reason about the problem in 3D don't do it in a complete and exhaustive manner. The approach proposed in this thesis is to use rigid body simulation to obtain a more truthful model of the reality, i.e. of the interaction between the robot and the environment. Such a simulation obeys the laws of physics, takes into account the geometry of the environment, the geometry of the robot, and any dynamic constraints that may be in place. The physics-based motion planning approach by itself is also highly intractable due to the computational load required to perform state propagation combined with the exponential blowup of planning; additionally, there are more technical limitations that disallow us to use things such as state sampling or state steering, which are known to be effective in solving the problem in simpler domains. The proposed solution to this problem is to compute heuristics that can bias the search towards the goal, so as to quickly converge towards the solution. With such a model, the search space is a rich space, which can only contain states which are physically reachable by the robot, and also tells us enough information about the safety of the robot itself. The overall result is that by using this framework the robot engineer has a simpler job of encoding the \textit{domain knowledge} which now consists only of providing the robot geometric model plus any constraints

Vibration characteristic analysis and optimization of rotor hub for helicopters based on genetic algorithm

The aero-elastic response of the blades is affected by the structure style of the rotor hub. The fatigue life of the structure and the vibration level of the entire helicopter are directly affected by the aero-elastic response of the blades. Therefore, its vibration characteristics and aero-elastic response shall be considered emphatically during structure design. The flapping and lead-lag characteristics of the rotor hub were tested experimentally in the paper. It was shown in results that a big vibration displacement was generated from the rotor hub under two working conditions. However, a larger vibration displacement was generated from the flapping than that of the lead-lag. Therefore, it is necessary to conduct optimization design to the rotor hub. Analysis was carried out to aero-elastic dynamics of bearing-less rotor. Afterwards, a multi-objective optimization model for reducing vibration and aero-elastic dynamic of bearing-less rotor was built. Pareto front of multi-objective optimization design for bearing-less rotor was acquired by Genetic Algorithm (GA). To reduce the optimization burdens, the aero-elastic dynamic model was replaced by Radial Basis Function (RBF) model during the optimization. To decrease the computational costs, the sample points with the minimum point set center deviation of the current surrogate model were added gradually. Meanwhile, the design schemes of Pareto front were ranked by TOPSIS to select the design scheme with the maximum satisfaction. Comparison was carried out between the rotor design scheme with the maximum satisfaction and the unoptimized model in terms of modal frequency, modal shape and vibration load of the hub. The results showed that the dynamic performance of the optimized rotor was improved significantly. The method in this paper also provided an important reference for multi-objective optimization of aero-elastic dynamic and engineering design of other rotors

Human Pose Tracking from Monocular Image Sequences

This thesis proposes various novel approaches for improving the performance of automatic 2D human pose tracking system including multi-scale strategy, mid-level spatial dependencies to constrain more relations of multiple body parts, additional constraints between symmetric body parts and the left/right confusion correction by a head orientation estimator. These proposed approaches are employed to develop a complete human pose tracking system. The experimental results demonstrate significant improvements of all the proposed approaches towards accuracy and efficiency