Motion capture data processing, retrieval and recognition.

Character animation plays an essential role in the area of featured film and computer games. Manually creating character animation by animators is both tedious and inefficient, where motion capture techniques (MoCap) have been developed and become the most popular method for creating realistic character animation products. Commercial MoCap systems are expensive and the capturing process itself usually requires an indoor studio environment. Procedural animation creation is often lacking extensive user control during the generation progress. Therefore, efficiently and effectively reusing MoCap data can brings significant benefits, which has motivated wider research in terms of machine learning based MoCap data processing. A typical work flow of MoCap data reusing can be divided into 3 stages: data capture, data management and data reusing. There are still many challenges at each stage. For instance, the data capture and management often suffer from data quality problems. The efficient and effective retrieval method is also demanding due to the large amount of data being used. In addition, classification and understanding of actions are the fundamental basis of data reusing. This thesis proposes to use machine learning on MoCap data for reusing purposes, where a frame work of motion capture data processing is designed. The modular design of this framework enables motion data refinement, retrieval and recognition. The first part of this thesis introduces various methods used in existing motion capture processing approaches in literature and a brief introduction of relevant machine learning methods used in this framework. In general, the frameworks related to refinement, retrieval, recognition are discussed. A motion refinement algorithm based on dictionary learning will then be presented, where kinematical structural and temporal information are exploited. The designed optimization method and data preprocessing technique can ensure a smooth property for the recovered result. After that, a motion refinement algorithm based on matrix completion is presented, where the low-rank property and spatio-temporal information is exploited. Such model does not require preparing data for training. The designed optimization method outperforms existing approaches in regard to both effectiveness and efficiency. A motion retrieval method based on multi-view feature selection is also proposed, where the intrinsic relations between visual words in each motion feature subspace are discovered as a means of improving the retrieval performance. A provisional trace-ratio objective function and an iterative optimization method are also included. A non-negative matrix factorization based motion data clustering method is proposed for recognition purposes, which aims to deal with large scale unsupervised/semi-supervised problems. In addition, deep learning models are used for motion data recognition, e.g. 2D gait recognition and 3D MoCap recognition. To sum up, the research on motion data refinement, retrieval and recognition are presented in this thesis with an aim to tackle the major challenges in motion reusing. The proposed motion refinement methods aim to provide high quality clean motion data for downstream applications. The designed multi-view feature selection algorithm aims to improve the motion retrieval performance. The proposed motion recognition methods are equally essential for motion understanding. A collection of publications by the author of this thesis are noted in publications section

Modelling and analysis of hand motion in everyday activities with application to prosthetic hand technology

Upper-limb prostheses are either too expensive for many consumers or exhibit a greatly simplified choice of actions, this research aims to enable an improvement in the quality of life for recipients of these devices. Previous attempts at determining the hand shapes performed during activities of daily living (ADL) provide a limited range of tasks studied and data recorded. To avoid these limitations, motion capture systems and machine learning techniques have been utilised throughout this study. A portable motion capture system created, utilising a Leap Motion controller (LMC), has captured natural hand motions during modern ADL. Furthering the use of these data, a method applying optimisation techniques alongside a musculoskeletal model of the hand is proposed for predicting muscle excitations from kinematic data. The LMC was also employed in a device (AirGo) created to measure joint angles, aiming to provide an improvement to joint angle measurements in hand clinics. Hand movements for 22 participants were recorded during ADL over 111 hours and 20 minutes - providing a taxonomy of 40 and 24 hand shapes for the left and right hands, respectively. The predicted muscle excitations produced joint angles with an average correlation of 0.58 to those of the desired hand shapes. AirGo has been successfully employed within a hand therapy clinic to measure digit angles of 11 patients. A taxonomy of the hand shapes used in modern ADL is presented, highlighting the hand shapes currently more appropriate to consider during upper-limb prostheses development. A method for predicting the muscle excitations of the hand from kinematic data is introduced, implemented with data collected during ADL. AirGo offered improved repeatability over traditional devices used for such measurements with greater ease of use