Human Pose Estimation with Implicit Shape Models

This work presents a new approach for estimating 3D human poses based on monocular camera information only. For this, the Implicit Shape Model is augmented by new voting strategies that allow to localize 2D anatomical landmarks in the image. The actual 3D pose estimation is then formulated as a Particle Swarm Optimization (PSO) where projected 3D pose hypotheses are compared with the generated landmark vote distributions

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Human Pose Estimation with Implicit Shape Models

Diese Doktorarbeit stellt einen neuen Ansatz vor, wie 3D Posen von Personen alleine auf Basis monokularer Bildinformation geschätzt werden können. Hierzu wird das Implicit Shape Modell um neue Votingstrategien erweitert, die die Lokalisierung anatomischer Landmarken im 2D Bildraum erlauben. Das anschließende eigentliche 3D Posenschätzungsproblem wird dann im Rahmen einer Partikel-Schwarm-Optimierung auf Basis der generierten Voteverteilungen formuliert

Human Pose Estimation with Implicit Shape Models

Diese Doktorarbeit stellt einen neuen Ansatz vor, wie 3D Posen von Personen alleine auf Basis monokularer Bildinformation geschätzt werden können. Hierzu wird das Implicit Shape Modell um neue Votingstrategien erweitert, die die Lokalisierung anatomischer Landmarken im 2D Bildraum erlauben. Das anschließende eigentliche 3D Posenschätzungsproblem wird dann im Rahmen einer Partikel-Schwarm-Optimierung auf Basis der generierten Voteverteilungen formuliert

VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera

We present the first real-time method to capture the full global 3D skeletal pose of a human in a stable, temporally consistent manner using a single RGB camera. Our method combines a new convolutional neural network (CNN) based pose regressor with kinematic skeleton fitting. Our novel fully-convolutional pose formulation regresses 2D and 3D joint positions jointly in real time and does not require tightly cropped input frames. A real-time kinematic skeleton fitting method uses the CNN output to yield temporally stable 3D global pose reconstructions on the basis of a coherent kinematic skeleton. This makes our approach the first monocular RGB method usable in real-time applications such as 3D character control---thus far, the only monocular methods for such applications employed specialized RGB-D cameras. Our method's accuracy is quantitatively on par with the best offline 3D monocular RGB pose estimation methods. Our results are qualitatively comparable to, and sometimes better than, results from monocular RGB-D approaches, such as the Kinect. However, we show that our approach is more broadly applicable than RGB-D solutions, i.e. it works for outdoor scenes, community videos, and low quality commodity RGB cameras.Comment: Accepted to SIGGRAPH 201

A Survey on Evolutionary Computation for Computer Vision and Image Analysis: Past, Present, and Future Trends

Computer vision (CV) is a big and important field in artificial intelligence covering a wide range of applications. Image analysis is a major task in CV aiming to extract, analyse and understand the visual content of images. However, imagerelated tasks are very challenging due to many factors, e.g., high variations across images, high dimensionality, domain expertise requirement, and image distortions. Evolutionary computation (EC) approaches have been widely used for image analysis with significant achievement. However, there is no comprehensive survey of existing EC approaches to image analysis. To fill this gap, this paper provides a comprehensive survey covering all essential EC approaches to important image analysis tasks including edge detection, image segmentation, image feature analysis, image classification, object detection, and others. This survey aims to provide a better understanding of evolutionary computer vision (ECV) by discussing the contributions of different approaches and exploring how and why EC is used for CV and image analysis. The applications, challenges, issues, and trends associated to this research field are also discussed and summarised to provide further guidelines and opportunities for future research

NON-RIGID BODY MECHANICAL PROPERTY RECOVERY FROM IMAGES AND VIDEOS

Material property has great importance in surgical simulation and virtual reality. The mechanical properties of the human soft tissue are critical to characterize the tissue deformation of each patient. Studies have shown that the tissue stiffness described by the tissue properties may indicate abnormal pathological process. The (recovered) elasticity parameters can assist surgeons to perform better pre-op surgical planning and enable medical robots to carry out personalized surgical procedures. Traditional elasticity parameters estimation methods rely largely on known external forces measured by special devices and strain field estimated by landmarks on the deformable bodies. Or they are limited to mechanical property estimation for quasi-static deformation. For virtual reality applications such as virtual try-on, garment material capturing is of equal significance as the geometry reconstruction. In this thesis, I present novel approaches for automatically estimating the material properties of soft bodies from images or from a video capturing the motion of the deformable body. I use a coupled simulation-optimization-identification framework to deform one soft body at its original, non-deformed state to match the deformed geometry of the same object in its deformed state. The optimal set of material parameters is thereby determined by minimizing the error metric function. This method can simultaneously recover the elasticity parameters of multiple regions of soft bodies using Finite Element Method-based simulation (of either linear or nonlinear materials undergoing large deformation) and particle-swarm optimization methods. I demonstrate the effectiveness of this approach on real-time interaction with virtual organs in patient-specific surgical simulation, using parameters acquired from low-resolution medical images. With the recovered elasticity parameters and the age of the prostate cancer patients as features, I build a cancer grading and staging classifier. The classifier achieves up to 91% for predicting cancer T-Stage and 88% for predicting Gleason score. To recover the mechanical properties of soft bodies from a video, I propose a method which couples statistical graphical model with FEM simulation. Using this method, I can recover the material properties of a soft ball from a high-speed camera video that captures the motion of the ball. Furthermore, I extend the material recovery framework to fabric material identification. I propose a novel method for garment material extraction from a single-view image and a learning based cloth material recovery method from a video recording the motion of the cloth. Most recent garment capturing techniques rely on acquiring multiple views of clothing, which may not always be readily available, especially in the case of pre-existing photographs from the web. As an alternative, I propose a method that can compute a 3D model of a human body and its outfit from a single photograph with little human interaction. My proposed learning-based cloth material type recovery method exploits simulated data-set and deep neural network. I demonstrate the effectiveness of my algorithms by re-purposing the reconstructed garments for virtual try-on, garment transfer, and cloth animation on digital characters. With the recovered mechanical properties, one can construct a virtual world with soft objects exhibiting real-world behaviors.Doctor of Philosoph

On the dynamics of human locomotion and co-design of lower limb assistive devices

Recent developments in lower extremities wearable robotic devices for the assistance and rehabilitation of humans suffering from an impairment have led to several successes in the assistance of people who as a result regained a certain form of locomotive capability. Such devices are conventionally designed to be anthropomorphic. They follow the morphology of the human lower limbs. It has been shown previously that non-anthropomorphic designs can lead to increased comfort and better dynamical properties due to the fact that there is more morphological freedom in the design parameters of such a device. At the same time, exploitation of this freedom is not always intuitive and can be difficult to incorporate. In this work we strive towards a methodology aiding in the design of possible non-anthropomorphic structures for the task of human locomotion assistance by means of simulation and optimization. The simulation of such systems requires state of the art rigid body dynamics, contact dynamics and, importantly, closed loop dynamics. Through the course of our work, we first develop a novel, open and freely available, state of the art framework for the modeling and simulation of general coupled dynamical systems and show how such a framework enables the modeling of systems in a novel way. The resultant simulation environment is suitable for the evaluation of structural designs, with a specific focus on locomotion and wearable robots. To enable open-ended co-design of morphology and control, we employ population-based optimization methods to develop a novel Particle Swarm Optimization derivative specifically designed for the simultaneous optimization of solution structures (such as mechanical designs) as well as their continuous parameters. The optimizations that we aim to perform require large numbers of simulations to accommodate them and we develop another open and general framework to aid in large scale, population based optimizations in multi-user environments. Using the developed tools, we first explore the occurrence and underlying principles of natural human gait and apply our findings to the optimization of a bipedal gait of a humanoid robotic platform. Finally, we apply our developed methods to the co-design of a non-anthropomorphic, lower extremities, wearable robot in simulation, leading to an iterative co-design methodology aiding in the exploration of otherwise hard to realize morphological design