Human Pose Co-Estimation and Applications

Abstract—Most existing techniques for articulated human pose estimation consider each person independently. Here we tackle the problem in a new setting, coined Human Pose Co-estimation (PCE), where multiple persons are in a common, but unknown pose. The task of PCE is to estimate their poses jointly and to produce prototypes characterizing the shared pose. Since the poses of the individual persons should be similar to the prototype, PCE has less freedom compared to estimating each pose independently, which simplifies the problem. We demonstrate our PCE technique on two applications. The first is estimating pose of people performing the same activity synchronously, such as during aerobic, cheerleading and dancing in a group. We show that PCE improves pose estimation accuracy over estimating each person independently. The second application is learning prototype poses characterizing a pose class directly from an image search engine queried by the class name (e.g. ‘lotus pose’). We show that PCE leads to better pose estimation in such images, and it learns meaningful prototypes which can be used as priors for pose estimation in novel images. Index Terms—human pose estimation, articulated objects, multiple image correspondence, object detectio

Heterogeneous Multi-task Learning for Human Pose Estimation with Deep Convolutional Neural Network

We propose an heterogeneous multi-task learning framework for human pose estimation from monocular image with deep convolutional neural network. In particular, we simultaneously learn a pose-joint regressor and a sliding-window body-part detector in a deep network architecture. We show that including the body-part detection task helps to regularize the network, directing it to converge to a good solution. We report competitive and state-of-art results on several data sets. We also empirically show that the learned neurons in the middle layer of our network are tuned to localized body parts

Human Pose Estimation from Monocular Images : a Comprehensive Survey

Human pose estimation refers to the estimation of the location of body parts and how they are connected in an image. Human pose estimation from monocular images has wide applications (e.g., image indexing). Several surveys on human pose estimation can be found in the literature, but they focus on a certain category; for example, model-based approaches or human motion analysis, etc. As far as we know, an overall review of this problem domain has yet to be provided. Furthermore, recent advancements based on deep learning have brought novel algorithms for this problem. In this paper, a comprehensive survey of human pose estimation from monocular images is carried out including milestone works and recent advancements. Based on one standard pipeline for the solution of computer vision problems, this survey splits the problema into several modules: feature extraction and description, human body models, and modelin methods. Problem modeling methods are approached based on two means of categorization in this survey. One way to categorize includes top-down and bottom-up methods, and another way includes generative and discriminative methods. Considering the fact that one direct application of human pose estimation is to provide initialization for automatic video surveillance, there are additional sections for motion-related methods in all modules: motion features, motion models, and motion-based methods. Finally, the paper also collects 26 publicly available data sets for validation and provides error measurement methods that are frequently used

Deep Learning-Based Human Pose Estimation: A Survey

Human pose estimation aims to locate the human body parts and build human body representation (e.g., body skeleton) from input data such as images and videos. It has drawn increasing attention during the past decade and has been utilized in a wide range of applications including human-computer interaction, motion analysis, augmented reality, and virtual reality. Although the recently developed deep learning-based solutions have achieved high performance in human pose estimation, there still remain challenges due to insufficient training data, depth ambiguities, and occlusion. The goal of this survey paper is to provide a comprehensive review of recent deep learning-based solutions for both 2D and 3D pose estimation via a systematic analysis and comparison of these solutions based on their input data and inference procedures. More than 240 research papers since 2014 are covered in this survey. Furthermore, 2D and 3D human pose estimation datasets and evaluation metrics are included. Quantitative performance comparisons of the reviewed methods on popular datasets are summarized and discussed. Finally, the challenges involved, applications, and future research directions are concluded. We also provide a regularly updated project page: \url{https://github.com/zczcwh/DL-HPE

Recovering missing data from human body images

We establish critical discussion on the problem of Semantic Inpainting in still images of humans. We present a Dataset for this task as well as analyze the performance of current SOTA methods on it. We also present a novel metric based on human pose estimation quality over the reconstruction

Estimación de la pose humana 2D en imágenes estéreo

La Estimación de la Pose Humana es el proceso de obtener la configuración espacial de las partes del cuerpo en imágenes. Frente a los métodos monoculares, que recuperan la pose a partir de una sola imagen, los métodos estéreo usan un par de imágenes para realizar el proceso, siendo capaces de aprovechar la redundancia de información y así mejorar la precisión. Este trabajo de Tesis se centra en adaptar técnicas monoculares de estimación de la pose ya existentes para que sean capaces de aprovechar las ventajas del uso de información estéreo. La primera contribución de esta tesis es una nueva técnica para estimar la pose 2D de personas en imágenes estéreo basado en una restricción de similitud que permite la colaboración entre dos estimadores de pose. Nuestra propuesta mejora la precisión de las poses estimadas en comparación con técnicas monoculares de estimación de la pose ejecutadas de forma independiente en cada vista de la imagen estéreo. La segunda contribución es una base de datos para el problema de la estimación de la pose humana en imágenes estéreo. Para validar experimentalmente nuestras propuestas, hemos creado una nueva base de datos anotada de 630 imágenes estéreo que muestran personas en entornos diferentes, con ropa variada y diversa iluminación. La base de datos muestra a las personas en posición vertical con una gran variedad de poses de brazos que cubren todo el espacio de posibles configuraciones de poses. La tercera contribución es un nuevo método para estimar la pose 2D de personas en secuencias de video estéreo. El método comienza con una reducción de las posibles localizaciones de las partes del cuerpo usando información de color y de disparidad. A continuación, se utiliza información a priori para la localización de las partes del cuerpo más estructuradas. Por último, un método de recombinación de partes del cuerpo se aplica en la secuencia estéreo para obtener la mejor configuración de las partes del cuerpo. Los experimentos demuestran que la propuesta consigue mejores resultados que el actual estado del arte.Human Pose Estimation (HPE) is the task of obtaining the spatial con guration of human body parts from images. Methods recovering the human pose from a single image are called monocular approaches while those using image pairs are called stereo approaches. Stereo images provide extra information that can be employed to improve the results obtained by monocular approaches. This Thesis considers the problem of 2D human pose estimation on stereo images. To this end, three contributions are provided. The rst contribution of this thesis is a new technique to automatically detect and estimate the 2D pose of humans in stereo images. The proposed method is based on a similarity constraint that promotes a collaboration between two pose estimators. We show experimentally that our proposal improves the accuracy of the estimated poses when compared to standard HPE techniques running independently on each image. The second contribution is a dataset for the problem of human pose estimation in stereo image. To experimentally validate our approach, we have created a new annotated dataset of 630 stereo image from stereo videos depicting people in di erent backgrounds, clothing, lighting or locations in the image frames. The dataset contains upright people in a great variety of arms poses, covering the space of possible con gurations quite uniformly. The third contribution is a new method to estimate the 2D pose of humans in stereo videos sequences. The proposed pipeline starts by constraining the posible location of body joints by exploiting color and disparity information, and adding location priors to the most structured joints. Finally, a body limb recombination method is applied along the stereo sequence to obtain the best con guration of the body joints. The experiments show that our method obtains better average results than the state-of-the-art