Data-driven techniques for animating virtual characters

One of the key goals of current research in data-driven computer animation is the synthesis of new motion sequences from existing motion data. This thesis presents three novel techniques for synthesising the motion of a virtual character from existing motion data and develops a framework of solutions to key character animation problems. The first motion synthesis technique presented is based on the character’s locomotion composition process. This technique examines the ability of synthesising a variety of character’s locomotion behaviours while easily specified constraints (footprints) are placed in the three-dimensional space. This is achieved by analysing existing motion data, and by assigning the locomotion behaviour transition process to transition graphs that are responsible for providing information about this process. However, virtual characters should also be able to animate according to different style variations. Therefore, a second technique to synthesise real-time style variations of character’s motion. A novel technique is developed that uses correlation between two different motion styles, and by assigning the motion synthesis process to a parameterised maximum a posteriori (MAP) framework retrieves the desire style content of the input motion in real-time, enhancing the realism of the new synthesised motion sequence. The third technique presents the ability to synthesise the motion of the character’s fingers either o↵-line or in real-time during the performance capture process. The advantage of both techniques is their ability to assign the motion searching process to motion features. The presented technique is able to estimate and synthesise a valid motion of the character’s fingers, enhancing the realism of the input motion. To conclude, this thesis demonstrates that these three novel techniques combine in to a framework that enables the realistic synthesis of virtual character movements, eliminating the post processing, as well as enabling fast synthesis of the required motion

An inertial motion capture framework for constructing body sensor networks

Motion capture is the process of measuring and subsequently reconstructing the movement of an animated object or being in virtual space. Virtual reconstructions of human motion play an important role in numerous application areas such as animation, medical science, ergonomics, etc. While optical motion capture systems are the industry standard, inertial body sensor networks are becoming viable alternatives due to portability, practicality and cost. This thesis presents an innovative inertial motion capture framework for constructing body sensor networks through software environments, smartphones and web technologies. The first component of the framework is a unique inertial motion capture software environment aimed at providing an improved experimentation environment, accompanied by programming scaffolding and a driver development kit, for users interested in studying or engineering body sensor networks. The software environment provides a bespoke 3D engine for kinematic motion visualisations and a set of tools for hardware integration. The software environment is used to develop the hardware behind a prototype motion capture suit focused on low-power consumption and hardware-centricity. Additional inertial measurement units, which are available commercially, are also integrated to demonstrate the functionality the software environment while providing the framework with additional sources for motion data. The smartphone is the most ubiquitous computing technology and its worldwide uptake has prompted many advances in wearable inertial sensing technologies. Smartphones contain gyroscopes, accelerometers and magnetometers, a combination of sensors that is commonly found in inertial measurement units. This thesis presents a mobile application that investigates whether the smartphone is capable of inertial motion capture by constructing a novel omnidirectional body sensor network. This thesis proposes a novel use for web technologies through the development of the Motion Cloud, a repository and gateway for inertial data. Web technologies have the potential to replace motion capture file formats with online repositories and to set a new standard for how motion data is stored. From a single inertial measurement unit to a more complex body sensor network, the proposed architecture is extendable and facilitates the integration of any inertial hardware configuration. The Motion Cloud’s data can be accessed through an application-programming interface or through a web portal that provides users with the functionality for visualising and exporting the motion data

Human Motion Analysis Using Very Few Inertial Measurement Units

Realistic character animation and human motion analysis have become major topics of research. In this doctoral research work, three different aspects of human motion analysis and synthesis have been explored. Firstly, on the level of better management of tens of gigabytes of publicly available human motion capture data sets, a relational database approach has been proposed. We show that organizing motion capture data in a relational database provides several benefits such as centralized access to major freely available mocap data sets, fast search and retrieval of data, annotations based retrieval of contents, entertaining data from non-mocap sensor modalities etc. Moreover, the same idea is also proposed for managing quadruped motion capture data. Secondly, a new method of full body human motion reconstruction using very sparse configuration of sensors is proposed. In this setup, two sensor are attached to the upper extremities and one sensor is attached to the lower trunk. The lower trunk sensor is used to estimate ground contacts, which are later used in the reconstruction process along with the low dimensional inputs from the sensors attached to the upper extremities. The reconstruction results of the proposed method have been compared with the reconstruction results of the existing approaches and it has been observed that the proposed method generates lower average reconstruction errors. Thirdly, in the field of human motion analysis, a novel method of estimation of human soft biometrics such as gender, height, and age from the inertial data of a simple human walk is proposed. The proposed method extracts several features from the time and frequency domains for each individual step. A random forest classifier is fed with the extracted features in order to estimate the soft biometrics of a human. The results of classification have shown that it is possible with a higher accuracy to estimate the gender, height, and age of a human from the inertial data of a single step of his/her walk

Full-body human motion reconstruction with sparse joint tracking using flexible sensors

Human motion tracking is a fundamental building block for various applications including computer animation, human-computer interaction, healthcare, etc. To reduce the burden of wearing multiple sensors, human motion prediction from sparse sensor inputs has become a hot topic in human motion tracking. However, such predictions are non-trivial as i) the widely adopted data-driven approaches can easily collapse to average poses. ii) the predicted motions contain unnatural jitters. In this work, we address the aforementioned issues by proposing a novel framework which can accurately predict the human joint moving angles from the signals of only four flexible sensors, thereby achieving the tracking of human joints in multi-degrees of freedom. Specifically, we mitigate the collapse to average poses by implementing the model with a Bi-LSTM neural network that makes full use of short-time sequence information; we reduce jitters by adding a median pooling layer to the network, which smooths consecutive motions. Although being bio-compatible and ideal for improving the wearing experience, the flexible sensors are prone to aging which increases prediction errors. Observing that the aging of flexible sensors usually results in drifts of their resistance ranges, we further propose a novel dynamic calibration technique to rescale sensor ranges, which further improves the prediction accuracy. Experimental results show that our method achieves a low and stable tracking error of 4.51 degrees across different motion types with only four sensors

Kinematic State Estimation using Multiple DGPS/MEMS-IMU Sensors

Animals have evolved over billions of years and understanding these complex and intertwined systems have potential to advance the technology in the field of sports science, robotics and more. As such, a gait analysis using Motion Capture (MOCAP) technology is the subject of a number of research and development projects aimed at obtaining quantitative measurements. Existing MOCAP technology has limited the majority of studies to the analysis of the steady-state locomotion in a controlled (indoor) laboratory environment. MOCAP systems such as the optical, non-optical acoustic and non-optical magnetic MOCAP systems require predefined capture volume and controlled environmental conditions whilst the non-optical mechanical MOCAP system impedes the motion of the subject. Although the non-optical inertial MOCAP system allows MOCAP in an outdoor environment, it suffers from measurement noise and drift and lacks global trajectory information. The accuracy of these MOCAP systems are known to decrease during the tracking of the transient locomotion. Quantifying the manoeuvrability of animals in their natural habitat to answer the question “Why are animals so manoeuvrable?” remains a challenge. This research aims to develop an outdoor MOCAP system that will allow tracking of the steady-state as well as the transient locomotion of an animal in its natural habitat outside a controlled laboratory condition. A number of researchers have developed novel MOCAP systems with the same aim of creating an outdoor MOCAP system that is aimed at tracking the motion outside a controlled laboratory (indoor) environment with unlimited capture volume. These novel MOCAP systems are either not validated against the commercial MOCAP systems or do not have comparable sub-millimetre accuracy as the commercial MOCAP systems. The developed DGPS/MEMS-IMU multi-receiver fusion MOCAP system was assessed to have global trajectory accuracy of _0:0394m, relative limb position accuracy of _0:006497m. To conclude the research, several recommendations are made to improve the developed MOCAP system and to prepare for a field-testing with a wild animal from a family of a terrestrial megafauna

Application of multibody dynamics techniques to the analysis of human gait

La tesi que es presenta tracta l’estudi cinemàtic i dinàmic de la marxa humana mitjançant tècniques de dinàmica de sistemes multisòlid. Per a aquest propòsit, s’utilitzen dos models biomecànics: un model pla format per 11 segments i 14 graus de llibertat i un model tridimensional format per 18 segments i 57 graus de llibertat. La formulació dinàmica multisòlid ha estat desenvolupada en coordenades mixtes (naturals i relatives). La marxa de l’individu s’enregistra al laboratori utilitzant un sistema de captura del moviment mitjançant el qual s’obté la posició de cadascun dels 37 marcadors situats sobre el cos del subjecte. Les dades de posició es filtren utilitzant un algorisme basat en el singular spectrum analysis (SSA) i les coordenades naturals del model es calculen mitjançant relacions algebraiques entre les posicions dels marcadors. Posteriorment, un procés de consistència cinemàtica assegura les restriccions de sòlid rígid. El processament cinemàtic continua amb l’aproximació de les posicions mitjançant corbes B-spline d’on se n’obtenen, per derivació analítica, els valors de velocitat i acceleració. En una anàlisi dinàmica inversa de la marxa humana, s’acostumen a utilitzar com a dades d’entrada els paràmetres antropomètrics (geomètrics i inercials) dels segments, les dades cinemàtiques i les mesures de les plaques de força. En contraposició al que fan la majoria d’autors, en aquesta tesi, les mesures de les plaques de força no són utilitzades directament en l’anàlisi sinó que només s’usen per solucionar el problema del repartiment del torsor resultant de les forces de contacte durant la fase de doble suport. En aquesta fase, els dos peus es recolzen sobre el terra i les mesures cinemàtiques són insuficients per determinar el torsor en cada peu. El nou mètode de repartiment que es proposa (anomenat contact force plate sharing, CFP) és una de les aportacions de la tesi i destaca pel fet que permet determinar un conjunt de forces i moments dinàmicament consistents amb el model biomecànic, sense haver de modificar-ne les coordenades cinemàtiques ni afegir forces o moments residuals en algun dels segments. Encara dins l’àmbit de l’estudi dinàmic invers, s’ha analitzat la sensitivitat dels parells articulars a errors comesos en estimar els paràmetres antropomètrics, a errors que poden contenir les mesures de les plaques de força i a errors que es poden cometre en el processament cinemàtic de les mesures. L’estudi permet concloure que els resultats són molt sensibles als errors cinemàtics i a les forces mesurades per les plaques, sent els errors en els paràmetres antropomètrics menys influents. La tesi també presenta un nou model tridimensional de contacte peu-terra basat en el contacte esfera-pla i els seus paràmetres s’estimen mitjançant dos enfocaments diferents basats en tècniques d’optimització. El model s’utilitza com un mètode alternatiu per solucionar el problema del repartiment durant la fase de doble suport en dinàmica inversa, i també s’utilitza en simulacions de dinàmica directa per estimar les forces de contacte entre el model biomecànic i el seu entorn. En l’anàlisi dinàmica directa és necessària la implementació d’un controlador que està basat, en aquest cas, en el filtre de Kalman estès. Les contribucions més importants de la tesi, en el cas de l’anàlisi dinàmica inversa, es centren en el mètode CFP i en l’ús del model de contacte per solucionar el repartiment de forces de contacte en la fase de doble suport. Referent a l’anàlisi de la influència dels errors en les dades d’entrada del problema dinàmic invers, la modelització estadística dels errors conjuntament amb la pertorbació conjunta de més d’un paràmetre antropomètric a la vegada (mantenint constant l’alçada i el pes de la persona) és també una novetat. Per altra banda, el model de contacte presentat és també una contribució original. En l’estat de l’art actual no es troben models que usin dades reals capturades al laboratori i que a la vegada s’utilitzin per solucionar el problema de repartiment en el doble suport i per simular el contacte peu-terra en una anàlisi dinàmica directa. Finalment, el fet de desenvolupar un model que s’utilitzi tant per a l’anàlisi dinàmica directa com inversa és també una de les aportacions d’aquesta tesi. Tot i que les dues anàlisis, per separat, són temes de recerca comuns en l’àmbit de la Biomecànica, es troben a faltar estudis que comprovin la validesa dels resultats que se n’obtenen. En aquesta tesi, els resultats de la dinàmica inversa s’han utilitzat com a dades d’entrada de l’anàlisi dinàmica directa, el resultat de la qual (el moviment) ha pogut ser comparat amb el que s’obté de la captura del laboratori (entrada de la dinàmica inversa). D’aquesta manera, el cercle es tanca i es pot verificar la validesa tant dels models com dels resultats obtinguts.This thesis presents the kinematic and dynamic study of human motion by means of multibody system dynamics techniques. For this purpose, two biomechanical models are used: a 2D model formed by 11 segments with 14 degrees of freedom, and a 3D model that consists of 18 segments with 57 degrees of freedom. The movement of the subject is recorded in the laboratory using a motion capture system that provides the position along time of 37 markers attached on the body of the subject. Position data are filtered using an algorithm based on singular spectrum analysis (SSA) and the natural coordinates of the model are calculated using algebraic relations between the marker positions. Afterwards, a kinematic procedure ensures the kinematic consistency and the data processing continues with the approximation of the position histories using B-spline curves and obtaining, by analytical derivation, the velocity and acceleration values. This information is used as input of an inverse dynamic analysis. Differing to most published works, in this thesis the force plates measurements are not used directly as inputs of the analysis. When both feet contact the ground, kinematic measurements are insufficient to determine the individual wrench at each foot. One of the contributions of the thesis is a new strategy that is proposed to solve the this indeterminacy (called corrected force plate sharing, CFP) based on force plates data. Using this method, a set of two contact wrenches dynamically consistent with the movement are obtained with no need neither to add residual wrenches nor to modify the original motion. Also in the IDA field, the sensitivity of the joint torques to errors in the anthropometric parameters, in the force plate measurements and to errors committed during the kinematic data processing is studied. The analysis shows that the results are very sensitive to errors in force measurements and in the kinematic processing, being the errors in the body segment parameters less influential. A new 3D foot-ground contact model is presented and its parameters are estimated using optimization techniques. The model is used as an alternative method to solve the mentioned sharing problem during the double support phase and it is also used, in a forward dynamic analysis, to estimate the contact forces between the biomechanical model and its environment. The forward dynamic simulation requires the implementation of a controller that is based, in this case, on the extended Kalman filter. The most important contributions of the thesis in IDA are focused on the CFP sharing method and regarding the analysis of the influence of errors in input data on the inverse dynamics results, the statistical modelling of the uncertainties together with the perturbation of more than one parameter at same time (remaining height and weight as a constant parameters) is also new in the literature. Moreover, the presented foot-ground contact model is also original. In the current state of the art, there are no models that use real data captured in the laboratory to solve the contact wrench sharing problem during the double support phase. Furthermore, there are few studies simulating the foot-ground interaction in a forward dynamic analysis using a continuous foot-ground contact model. Finally, developing a model that is used for both forward and inverse dynamic analysis is a relevant aspect of the methodology used. Although the two approaches separately are common research topics in the field of biomechanics, a small number of studies prove the validity of the obtained results. In this thesis, the results of the inverse dynamics are used as input data for the forward dynamic analysis, and the results of the latter (the motion) have been compared with the motion capture in the laboratory (input of the inverse dynamics analysis). Thus, the circle has been closed which allows us to validate the accuracy of both the models and the obtained results

Survey of Motion Tracking Methods Based on Inertial Sensors: A Focus on Upper Limb Human Motion

Motion tracking based on commercial inertial measurements units (IMUs) has been widely studied in the latter years as it is a cost-effective enabling technology for those applications in which motion tracking based on optical technologies is unsuitable. This measurement method has a high impact in human performance assessment and human-robot interaction. IMU motion tracking systems are indeed self-contained and wearable, allowing for long-lasting tracking of the user motion in situated environments. After a survey on IMU-based human tracking, five techniques for motion reconstruction were selected and compared to reconstruct a human arm motion. IMU based estimation was matched against motion tracking based on the Vicon marker-based motion tracking system considered as ground truth. Results show that all but one of the selected models perform similarly (about 35 mm average position estimation error)

Reconstruction and analysis of dynamic shapes

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2010.Cataloged from PDF version of thesis.Includes bibliographical references (p. 122-141).Motion capture has revolutionized entertainment and influenced fields as diverse as the arts, sports, and medicine. This is despite the limitation that it tracks only a small set of surface points. On the other hand, 3D scanning techniques digitize complete surfaces of static objects, but are not applicable to moving shapes. I present methods that overcome both limitations, and can obtain the moving geometry of dynamic shapes (such as people and clothes in motion) and analyze it in order to advance computer animation. Further understanding of dynamic shapes will enable various industries to enhance virtual characters, advance robot locomotion, improve sports performance, and aid in medical rehabilitation, thus directly affecting our daily lives. My methods efficiently recover much of the expressiveness of dynamic shapes from the silhouettes alone. Furthermore, the reconstruction quality is greatly improved by including surface orientations (normals). In order to make reconstruction more practical, I strive to capture dynamic shapes in their natural environment, which I do by using hybrid inertial and acoustic sensors. After capture, the reconstructed dynamic shapes are analyzed in order to enhance their utility. My algorithms then allow animators to generate novel motions, such as transferring facial performances from one actor onto another using multi-linear models. The presented research provides some of the first and most accurate reconstructions of complex moving surfaces, and is among the few approaches that establish a relationship between different dynamic shapes.by Daniel Vlasic.Ph.D

Human Motion Analysis with Wearable Inertial Sensors

High-resolution, quantitative data obtained by a human motion capture system can be used to better understand the cause of many diseases for effective treatments. Talking about the daily care of the aging population, two issues are critical. One is to continuously track motions and position of aging people when they are at home, inside a building or in the unknown environment; the other is to monitor their health status in real time when they are in the free-living environment. Continuous monitoring of human movement in their natural living environment potentially provide more valuable feedback than these in laboratory settings. However, it has been extremely challenging to go beyond laboratory and obtain accurate measurements of human physical activity in free-living environments. Commercial motion capture systems produce excellent in-studio capture and reconstructions, but offer no comparable solution for acquisition in everyday environments. Therefore in this dissertation, a wearable human motion analysis system is developed for continuously tracking human motions, monitoring health status, positioning human location and recording the itinerary. In this dissertation, two systems are developed for seeking aforementioned two goals: tracking human body motions and positioning a human. Firstly, an inertial-based human body motion tracking system with our developed inertial measurement unit (IMU) is introduced. By arbitrarily attaching a wearable IMU to each segment, segment motions can be measured and translated into inertial data by IMUs. A human model can be reconstructed in real time based on the inertial data by applying high efficient twists and exponential maps techniques. Secondly, for validating the feasibility of developed tracking system in the practical application, model-based quantification approaches for resting tremor and lower extremity bradykinesia in Parkinson’s disease are proposed. By estimating all involved joint angles in PD symptoms based on reconstructed human model, angle characteristics with corresponding medical ratings are employed for training a HMM classifier for quantification. Besides, a pedestrian positioning system is developed for tracking user’s itinerary and positioning in the global frame. Corresponding tests have been carried out to assess the performance of each system