Combination of Annealing Particle Filter and Belief Propagation for 3D Upper Body Tracking

3D upper body pose estimation is a topic greatly studied by the computer vision society because it is useful in a great number of applications, mainly for human robots interactions including communications with companion robots. However there is a challenging problem: the complexity of classical algorithms that increases exponentially with the dimension of the vectors’ state becomes too difficult to handle. To tackle this problem, we propose a new approach that combines several annealing particle filters defined independently for each limb and belief propagation method to add geometrical constraints between individual filters. Experimental results on a real human gestures sequence will show that this combined approach leads to reliable results

Modelação biomecânica do corpo humano : aplicação na análise da marcha

Dissertação de mestrado integrado em Engenharia BiomédicaWalking is a complex process, achieved through coordinated movements, which allows the displacement of the human body and therefore has been the subject of study since the beginning of time. Currently, modelling of this movement and the human body realistically, allowing recreating, simulating or analyzing human movement is still a major problem in biomedical engineering. Gait analysis allows the extraction of quantities that characterize human locomotion, allowing the evaluation of the gait pattern of a subject. Accurate measurement of movement is crucial in any technique to characterize the motion. The knowledge provided by this analysis provides geometric characteristics, physical and behavioral traits of the human body that allows the distinction between normal and pathological gait. The aim of this work involves developing an algorithm that allows the estimation of certain spatio-temporal parameters of interest, as are the frequency and period of the gait cycle, stride width, among others. This algorithm is developed in Matlab. It was also developed a model of the human body in Webots, whose function is to present the dynamics and the atual physical body in terms of length and weight. In the process of modeling, approximation and simplification of the form for each segment of the humanoid model is performed in order to meet the basic form of the human body. In the case of the study in question it‘s not necessary a visual result close to reality but a practical result of human locomotion. Thus the modeling of the human body was made using cylinders. In short, the main goal is to make suggestions that may contribute to the analysis of human movement, reproducing the same, using data on the position of the various segments of the human body obtained with the help of the Vicon software.Caminhar é um complexo processo, alcançado através de movimentos coordenados, que permite o deslocamento do corpo humano sendo, portanto, objeto de estudo desde sempre. Atualmente, é, ainda, um dos maiores problemas da engenharia biomédica, a modelação deste movimento e do corpo humano de modo realista, permitindo recriar, simular ou analisar o movimento humano. A análise da marcha possibilita a extração de quantidades que caracterizam a locomoção humana, permitindo a avaliação do padrão de marcha de um sujeito. A medição precisa do movimento é fulcral em qualquer técnica de caracterização da marcha. O conhecimento proporcionado por esta análise fornece características geométricas, físicas e comportamentais do corpo humano tornado possível a distinção entre marcha normal e patológica. O objetivo deste trabalho passa pelo desenvolvimento de um algoritmo que permite a estimação de determinados parâmetros espácio-temporais de interesse, como são a frequência e período do ciclo de marcha, amplitude de passada, entre outros. Este algoritmo é desenvolvido em ambiente Matlab. É ainda desenvolvido um modelo do corpo humano em Webots, cuja função é representar a dinâmica e o corpo humano real em termos de altura e massa. No processo de modelação, a aproximação e simplificação da forma para cada segmento do modelo humanoide é realizada de forma a ir de encontro à forma básica do corpo humano. No caso do estudo em causa, não é necessário um resultado visual aproximado à realidade mas sim um resultado prático de locomoção humana. Assim a modelação do corpo humano foi feita usando cilindros. Em suma, o grande objetivo é a apresentação de sugestões passíveis de contribuírem para a análise do movimento humano, reproduzindo o mesmo, através de dados relativos à posição dos diversos segmentos do corpo humano, obtidos com auxílio do software Vicon

Acquiring 3D Full-body Motion from Noisy and Ambiguous Input

Natural human motion is highly demanded and widely used in a variety of applications such as video games and virtual realities. However, acquisition of full-body motion remains challenging because the system must be capable of accurately capturing a wide variety of human actions and does not require a considerable amount of time and skill to assemble. For instance, commercial optical motion capture systems such as Vicon can capture human motion with high accuracy and resolution while they often require post-processing by experts, which is time-consuming and costly. Microsoft Kinect, despite its high popularity and wide applications, does not provide accurate reconstruction of complex movements when significant occlusions occur. This dissertation explores two different approaches that accurately reconstruct full-body human motion from noisy and ambiguous input data captured by commercial motion capture devices. The first approach automatically generates high-quality human motion from noisy data obtained from commercial optical motion capture systems, eliminating the need for post-processing. The second approach accurately captures a wide variety of human motion even under significant occlusions by using color/depth data captured by a single Kinect camera. The common theme that underlies two approaches is the use of prior knowledge embedded in pre-recorded motion capture database to reduce the reconstruction ambiguity caused by noisy and ambiguous input and constrain the solution to lie in the natural motion space. More specifically, the first approach constructs a series of spatial-temporal filter bases from pre-captured human motion data and employs them along with robust statistics techniques to filter noisy motion data corrupted by noise/outliers. The second approach formulates the problem in a Maximum a Posterior (MAP) framework and generates the most likely pose which explains the observations as well as consistent with the patterns embedded in the pre-recorded motion capture database. We demonstrate the effectiveness of our approaches through extensive numerical evaluations on synthetic data and comparisons against results created by commercial motion capture systems. The first approach can effectively denoise a wide variety of noisy motion data, including walking, running, jumping and swimming while the second approach is shown to be capable of accurately reconstructing a wider range of motions compared with Microsoft Kinect

Advances in Monocular Exemplar-based Human Body Pose Analysis: Modeling, Detection and Tracking

Esta tesis contribuye en el análisis de la postura del cuerpo humano a partir de secuencias de imágenes adquiridas con una sola cámara. Esta temática presenta un amplio rango de potenciales aplicaciones en video-vigilancia, video-juegos o aplicaciones biomédicas. Las técnicas basadas en patrones han tenido éxito, sin embargo, su precisión depende de la similitud del punto de vista de la cámara y de las propiedades de la escena entre las imágenes de entrenamiento y las de prueba. Teniendo en cuenta un conjunto de datos de entrenamiento capturado mediante un número reducido de cámaras fijas, paralelas al suelo, se han identificado y analizado tres escenarios posibles con creciente nivel de dificultad: 1) una cámara estática paralela al suelo, 2) una cámara de vigilancia fija con un ángulo de visión considerablemente diferente, y 3) una secuencia de video capturada con una cámara en movimiento o simplemente una sola imagen estática

Detektion und Posenerkennung von Personen mit 3D-Ansichtsmodellen

Service and assistant robots take over more and more complex duties within our everyday life. Therefore, the communication between humans and robots should be as natural as possible, in order to support an intuitive interaction. Detection and recognition of humans, as well as the estimation of their pose within camera images make a crucial contribution to this challenge. - For this purpose, an appearance based approach which uses a 3D model has been investigated within this diploma thesis. The model evaluates pose hypothesis by interpretation of color and edge features. The modeled upper body pose is determined by 13 parameters for the torso position and turning, the head twist plus the setting of the upper arms and the forearms. - An essential issue of this work is the optimization of the fit value function over the high dimensional pose space, in order to detect the most likely pose hypothesis. To achieve this, it is important that the appearance model causes a preferably smooth fit value function. Furthermore, different methods for optimization of the pose hypotheses within the pose space have been investigated. A tracking approach has been developed, which in contrast to the particle filter, allows the unrestricted motion of pose hypotheses during the optimization process. - Based on the diploma thesis of Dipl.-Ing. Daniel Dornbusch a framework is implemented, which allows the flexible combination of different subcomponents and easy testing of the corresponding parameters. The results of the analysis of the method have been validated by estimating the 3D head-torso pose of a person within a real image sequence.Assistenz- und Serviceroboter übernehmen zunehmend komplexe Aufgaben im menschlichen Alltag. Damit Personen intuitiv mit solchen Robotern interagieren können, sollte die Kommunikation zwischen Mensch und Roboter möglichst natürlich sein. Die Detektion und Wiedererkennung von Personen, sowie die Erkennung ihrer Pose auf Kamerabildern liefern dazu einen entscheidenden Beitrag. Zu diesem Zweck wird in dieser Diplomarbeit ein ansichtsbasiertes Verfahren vorgestellt, welches auf einem 3D-Modell beruht. Das Modell dient der Bewertung von Posenhypothesen auf der Basis von Farb- und Kantenmerkmalen. Die modellierten Oberkörperposen werden durch 13 Parameter für Torsoposition, und -orientierung, Kopfdrehung, sowie Ober- und Unterarmstellung beschrieben. Ein wesentlicher Kernpunkt der Arbeit ist die Optimierung dieser Posenhypothesen in dem Gütegebirge über dem hochdimensionalen Posenraum, zur Detektion der wahrscheinlichsten Hypothesen. In diesem Zusammenhang ist es von besonderer Bedeutung, dass das Gütegebirge des ansichtsbasierten 3D-Modells eine hohe Glattheit aufweist. Des Weiteren wurden verschiedene Methoden zur Optimierung der Posenhypothesen untersucht. Es wurde ein Tracking-Verfahren entwickelt, welches, im Gegensatz zum Partikel filter, die Bewegung der Hypothesen während der Optimierung uneingeschränkt zulässt. Aufbauend auf der Diplomarbeit von Dipl.-Ing. Daniel Dornbusch wurde ein Framework, zur flexiblen Kombination verschiedener Einzelkomponenten des Verfahrens und der Untersuchung unterschiedlicher Parameterkonfigurationen, implementiert. Die Analyseergebnisse, welche mittels dieses Frameworks erstellt wurden, konnten durch die Schätzung der dreidimensionalen Kopf-Torso-Pose einer Person innerhalb einer realen Bildsequenz validiert werden.Ilmenau, Techn. Univ., Diplomarbeit, 200

Upper body pose recognition and estimation towards the translation of South African sign language

Masters of ScienceRecognising and estimating gestures is a fundamental aspect towards translating from a sign language to a spoken language. It is a challenging problem and at the same time, a growing phenomenon in Computer Vision. This thesis presents two approaches, an example-based and a learning-based approach, for performing integrated detection, segmentation and 3D estimation of the human upper body from a single camera view. It investigates whether an upper body pose can be estimated from a database of exemplars with labelled poses. It also investigates whether an upper body pose can be estimated using skin feature extraction, Support Vector Machines (SVM) and a 3D human body model. The example-based and learning-based approaches obtained success rates of 64% and 88%, respectively. An analysis of the two approaches have shown that, although the learning-based system generally performs better than the example-based system, both approaches are suitable to recognise and estimate upper body poses in a South African sign language recognition and translation system.South Afric

Contour and texture for visual recognition of object categories

The recognition of categories of objects in images has become a central topic in computer vision. Automatic visual recognition systems are rapidly becoming central to applications such as image search, robotics, vehicle safety systems, and image editing. This work addresses three sub-problems of recognition: image classification, object detection, and semantic segmentation. The task of classification is to determine whether an object of a particular category is present or not. Object detection aims to localize any objects of the category. Semantic segmentation is a more complete image understanding, whereby an image is partitioned into coherent regions that are assigned meaningful class labels. This thesis proposes novel discriminative learning approaches to these problems. Our primary contributions are threefold. Firstly, we demonstrate that the contours (the outline and interior edges) of an object are, alone, sufficient for accurate visual recognition. Secondly, we propose two powerful new feature types: (i) a learned codebook of contour fragments matched with an improved oriented chamfer distance, and (ii) a set of texture-based features that simultaneously exploit local appearance, approximate shape, and appearance context. The efficacy of these new features types is evaluated on a wide variety of datasets. Thirdly, we show how, in combination, these two largely orthogonal feature types can substantially improve recognition performance above that achieved by either alone

Metaheuristic Optimization Techniques for Articulated Human Tracking

Four adaptive metaheuristic optimization algorithms are proposed and demonstrated: Adaptive Parameter Particle Swarm Optimization (AP-PSO), Modified Artificial Bat (MAB), Differential Mutated Artificial Immune System (DM-AIS) and hybrid Particle Swarm Accelerated Artificial Immune System (PSO-AIS). The algorithms adapt their search parameters on the basis of the fitness of obtained solutions such that a good fitness value favors local search, while a poor fitness value favors global search. This efficient feedback of the solution quality, imparts excellent global and local search characteristic to the proposed algorithms. The algorithms are tested on the challenging Articulated Human Tracking (AHT) problem whose objective is to infer human pose, expressed in terms of joint angles, from a continuous video stream. The Particle Filter (PF) algorithms, widely applied in generative model based AHT, suffer from the 'curse of dimensionality' and 'degeneracy' challenges. The four proposed algorithms show stable performance throughout the course of numerical experiments. DM-AIS performs best among the proposed algorithms followed in order by PSO-AIS, AP-PSO, and MBA in terms of Most Appropriate Pose (MAP) tracking error. The MAP tracking error of the proposed algorithms is compared with four heuristic approaches: generic PF, Annealed Particle Filter (APF), Partitioned Sampled Annealed Particle Filter (PSAPF) and Hierarchical Particle Swarm Optimization (HPSO). They are found to outperform generic PF with a confidence level of 95%, PSAPF and HPSO with a confidence level of 85%. While DM-AIS and PSO-AIS outperform APF with a confidence level of 80%. Further, it is noted that the proposed algorithms outperform PSAPF and HPSO using a significantly lower number of function evaluations, 2500 versus 7200. The proposed algorithms demonstrate reduced particle requirements, hence improving computational efficiency and helping to alleviate the 'curse of dimensionality'. The adaptive nature of the algorithms is found to guide the whole swarm towards the optimal solution by sharing information and exploring a wider solution space which resolves the 'degeneracy' challenge. Furthermore, the decentralized structure of the algorithms renders them insensitive to accumulation of error and allows them to recover from catastrophic failures due to loss of image data, sudden change in motion pattern or discrete instances of algorithmic failure. The performance enhancements demonstrated by the proposed algorithms, attributed to their balanced local and global search capabilities, makes real-time AHT applications feasible. Finally, the utility of the proposed algorithms in low-dimensional system identification problems as well as high-dimensional AHT problems demonstrates their applicability in various problem domains