Skeleton-aided Articulated Motion Generation

This work make the first attempt to generate articulated human motion sequence from a single image. On the one hand, we utilize paired inputs including human skeleton information as motion embedding and a single human image as appearance reference, to generate novel motion frames, based on the conditional GAN infrastructure. On the other hand, a triplet loss is employed to pursue appearance-smoothness between consecutive frames. As the proposed framework is capable of jointly exploiting the image appearance space and articulated/kinematic motion space, it generates realistic articulated motion sequence, in contrast to most previous video generation methods which yield blurred motion effects. We test our model on two human action datasets including KTH and Human3.6M, and the proposed framework generates very promising results on both datasets.Comment: ACM MM 201

Ontology-based context representation and reasoning for object tracking and scene interpretation in video

Computer vision research has been traditionally focused on the development of quantitative techniques to calculate the properties and relations of the entities appearing in a video sequence. Most object tracking methods are based on statistical methods, which often result inadequate to process complex scenarios. Recently, new techniques based on the exploitation of contextual information have been proposed to overcome the problems that these classical approaches do not solve. The present paper is a contribution in this direction: we propose a Computer Vision framework aimed at the construction of a symbolic model of the scene by integrating tracking data and contextual information. The scene model, represented with formal ontologies, supports the execution of reasoning procedures in order to: (i) obtain a high-level interpretation of the scenario; (ii) provide feedback to the low-level tracking procedure to improve its accuracy and performance. The paper describes the layered architecture of the framework and the structure of the knowledge model, which have been designed in compliance with the JDL model for Information Fusion. We also explain how deductive and abductive reasoning is performed within the model to accomplish scene interpretation and tracking improvement. To show the advantages of our approach, we develop an example of the use of the framework in a video-surveillance application.This work was supported in part by Projects CICYT TIN2008- 06742-C02-02/TSI, CICYT TEC2008-06732-C02-02/TEC, SINPROB, CAM MADRINET S-0505/TIC/0255 and DPS2008–07029-C02–02.Publicad

Modeling, Analysis, and Control of a Mobile Robot for \u3ci\u3eIn Vivo\u3c/i\u3e Fluoroscopy of Human Joints during Natural Movements

In this dissertation, the modeling, analysis and control of a multi-degree of freedom (mdof) robotic fluoroscope was investigated. A prototype robotic fluoroscope exists, and consists of a 3 dof mobile platform with two 2 dof Cartesian manipulators mounted symmetrically on opposite sides of the platform. One Cartesian manipulator positions the x-ray generator and the other Cartesian manipulator positions the x-ray imaging device. The robotic fluoroscope is used to x-ray skeletal joints of interest of human subjects performing natural movement activities. In order to collect the data, the Cartesian manipulators must keep the x-ray generation and imaging devices accurately aligned while dynamically tracking the desired skeletal joint of interest. In addition to the joint tracking, this also requires the robotic platform to move along with the subject, allowing the manipulators to operate within their ranges of motion. A comprehensive dynamic model of the robotic fluoroscope prototype was created, incorporating the dynamic coupling of the system. Empirical data collected from an RGB-D camera were used to create a human kinematic model that can be used to simulate the joint of interest target dynamics. This model was incorporated into a computer simulation that was validated by comparing the simulation results with actual prototype experiments using the same human kinematic model inputs. The computer simulation was used in a comprehensive dynamic analysis of the prototype and in the development and evaluation of sensing, control, and signal processing approaches that optimize the subject and joint tracking performance characteristics. The modeling and simulation results were used to develop real-time control strategies, including decoupling techniques that reduce tracking error on the prototype. For a normal walking activity, the joint tracking error was less than 20 mm, and the subject tracking error was less than 140 mm

Modelação biomecânica do corpo humano : aplicação na análise da marcha

Dissertação de mestrado integrado em Engenharia BiomédicaWalking is a complex process, achieved through coordinated movements, which allows the displacement of the human body and therefore has been the subject of study since the beginning of time. Currently, modelling of this movement and the human body realistically, allowing recreating, simulating or analyzing human movement is still a major problem in biomedical engineering. Gait analysis allows the extraction of quantities that characterize human locomotion, allowing the evaluation of the gait pattern of a subject. Accurate measurement of movement is crucial in any technique to characterize the motion. The knowledge provided by this analysis provides geometric characteristics, physical and behavioral traits of the human body that allows the distinction between normal and pathological gait. The aim of this work involves developing an algorithm that allows the estimation of certain spatio-temporal parameters of interest, as are the frequency and period of the gait cycle, stride width, among others. This algorithm is developed in Matlab. It was also developed a model of the human body in Webots, whose function is to present the dynamics and the atual physical body in terms of length and weight. In the process of modeling, approximation and simplification of the form for each segment of the humanoid model is performed in order to meet the basic form of the human body. In the case of the study in question it‘s not necessary a visual result close to reality but a practical result of human locomotion. Thus the modeling of the human body was made using cylinders. In short, the main goal is to make suggestions that may contribute to the analysis of human movement, reproducing the same, using data on the position of the various segments of the human body obtained with the help of the Vicon software.Caminhar é um complexo processo, alcançado através de movimentos coordenados, que permite o deslocamento do corpo humano sendo, portanto, objeto de estudo desde sempre. Atualmente, é, ainda, um dos maiores problemas da engenharia biomédica, a modelação deste movimento e do corpo humano de modo realista, permitindo recriar, simular ou analisar o movimento humano. A análise da marcha possibilita a extração de quantidades que caracterizam a locomoção humana, permitindo a avaliação do padrão de marcha de um sujeito. A medição precisa do movimento é fulcral em qualquer técnica de caracterização da marcha. O conhecimento proporcionado por esta análise fornece características geométricas, físicas e comportamentais do corpo humano tornado possível a distinção entre marcha normal e patológica. O objetivo deste trabalho passa pelo desenvolvimento de um algoritmo que permite a estimação de determinados parâmetros espácio-temporais de interesse, como são a frequência e período do ciclo de marcha, amplitude de passada, entre outros. Este algoritmo é desenvolvido em ambiente Matlab. É ainda desenvolvido um modelo do corpo humano em Webots, cuja função é representar a dinâmica e o corpo humano real em termos de altura e massa. No processo de modelação, a aproximação e simplificação da forma para cada segmento do modelo humanoide é realizada de forma a ir de encontro à forma básica do corpo humano. No caso do estudo em causa, não é necessário um resultado visual aproximado à realidade mas sim um resultado prático de locomoção humana. Assim a modelação do corpo humano foi feita usando cilindros. Em suma, o grande objetivo é a apresentação de sugestões passíveis de contribuírem para a análise do movimento humano, reproduzindo o mesmo, através de dados relativos à posição dos diversos segmentos do corpo humano, obtidos com auxílio do software Vicon