Objective assessment of movement disabilities using wearable sensors

The research presents a series of comprehensive analyses based on inertial measurements obtained from wearable sensors to quantitatively describe and assess human kinematic performance in certain tasks that are most related to daily life activities. This is not only a direct application of human movement analysis but also very pivotal in assessing the progression of patients undergoing rehabilitation services. Moreover, the detailed analysis will provide clinicians with greater insights to capture movement disorders and unique ataxic features regarding axial abnormalities which are not directly observed by the clinicians

On the role of gestures in human-robot interaction

This thesis investigates the gestural interaction problem and in particular the usage of gestures for human-robot interaction. The lack of a clear definition of the problem statement and a common terminology resulted in a fragmented field of research where building upon prior work is rare. The scope of the research presented in this thesis, therefore, consists in laying the foundation to help the community to build a more homogeneous research field. The main contributions of this thesis are twofold: (i) a taxonomy to define gestures; and (ii) an ingegneristic definition of the gestural interaction problem. The contributions resulted is a schema to represent the existing literature in a more organic way, helping future researchers to identify existing technologies and applications, also thanks to an extensive literature review. Furthermore, the defined problem has been studied in two of its specialization: (i) direct control and (ii) teaching of a robotic manipulator, which leads to the development of technological solutions for gesture sensing, detection and classification, which can possibly be applied to other contexts

Low-Cost Sensors and Biological Signals

Many sensors are currently available at prices lower than USD 100 and cover a wide range of biological signals: motion, muscle activity, heart rate, etc. Such low-cost sensors have metrological features allowing them to be used in everyday life and clinical applications, where gold-standard material is both too expensive and time-consuming to be used. The selected papers present current applications of low-cost sensors in domains such as physiotherapy, rehabilitation, and affective technologies. The results cover various aspects of low-cost sensor technology from hardware design to software optimization

On-line Time Warping of Human Motion Sequences

Some application areas require motions to be time warped on-line as a motion is captured, aligning a partially captured motion to a complete prerecorded motion. For example movement training applications for dance and medical procedures, require on-line time warping for analysing and visually feeding back the accuracy of human motions as they are being performed. Additionally, real-time production techniques such as virtual production, in camera visual effects and the use of avatars in live stage performances, require on-line time warping to align virtual character performances to a live performer. The work in this thesis first addresses a research gap in the measurement of the alignment of two motions, proposing approaches based on rank correlation and evaluating them against existing distance based approaches to measuring motion similarity. The thesis then goes onto propose and evaluate novel methods for on-line time warping, which plot alignments in a forward direction and utilise forecasting and local continuity constraint techniques. Current studies into measuring the similarity of motions focus on distance based metrics for measuring the similarity of the motions to support motion recognition applications, leaving a research gap regarding the effectiveness of similarity metrics bases on correlation and the optimal metrics for measuring the alignment of two motions. This thesis addresses this research gap by comparing the performance of variety of similarity metrics based on distance and correlation, including novel combinations of joint parameterisation and correlation methods. The ability of each metric to measure both the similarity and alignment of two motions is independently assessed. This work provides a detailed evaluation of a variety of different approaches to using correlation within a similarity metric, testing their performance to determine which approach is optimal and comparing their performance against established distance based metrics. The results show that a correlation based metric, in which joints are parameterised using displacement vectors and correlation is measured using Kendall Tau rank correlation, is the optimal approach for measuring the alignment between two motions. The study also showed that similarity metrics based on correlation are better at measuring the alignment of two motions, which is important in motion blending and style transfer applications as well as evaluating the performance of time warping algorithms. It also showed that metrics based on distance are better at measuring the similarity of two motions, which is more relevant to motion recognition and classification applications. A number of approaches to on-line time warping have been proposed within existing research, that are based on plotting an alignment path backwards from a selected end-point within the complete motion. While these approaches work for discrete applications, such as recognising a motion, their lack of monotonic constraint between alignment of each frame, means these approaches do not support applications that require an alignment to be maintained continuously over a number of frames. For example applications involving continuous real-time visualisation, feedback or interaction. To solve this problem, a number of novel on-line time warping algorithms, based on forward plotting, motion forecasting and local continuity constraints are proposed and evaluated by applying them to human motions. Two benchmarks standards for evaluating the performance of on-line time warping algorithms are established, based on UTW time warping and compering the resulting alignment path with that produced by DTW. This work also proposes a novel approach to adapting existing local continuity constraints to a forward plotting approach. The studies within this thesis demonstrates that these time warping approaches are able to produce alignments of sufficient quality to support applications that require an alignment to be maintained continuously. The on-line time warping algorithms proposed in this study can align a previously recorded motion to a user in real-time, as they are performing the same action or an opposing action recorded at the same time as the motion being align. This solution has a variety of potential application areas including: visualisation applications, such as aligning a motion to a live performer to facilitate in camera visual effects or a live stage performance with a virtual avatar; motion feedback applications such as dance training or medical rehabilitation; and interaction applications such as working with Cobots

Seleksi Fitur untuk Pengenalan Gestur Tangan 3D secara Dinamis pada Interaksi Benda Virtual Menggunakan Hybrid GMM

Penjelajahan dan interaksi manusia dalam dunia virtual yang dilakukan berbatuan sensor 3D merambah beragam aktivitas kehidupan manusia. Khusus untuk interaksi terhadap benda virtual mengalami kemajuan pesat dalam hal ragam gestur yang digunakan. Usaha peningkatan kinerja pengenalan gestur banyak dilakukan, antara lain menggunakan ragam pengklasifikasi seperti: Naive Bayes, Quaternion Dynamic Time Warping, Hidden Conditional Neural Field, Two Stacked Long-Short Term Memory(LSTM) dan Modified LSTM melalui penambahan Reset Gate. Namun, masih terdapat celah yaitu relatif rendahnya akurasi pengenalan gestur oleh pengklasifikasi. Hal ini diakibatkan oleh tidak dimanfaatkannya seleksi fitur dalam persiapan dataset. Tujuan penelitian ini adalah menawarkan sebuah pendekatan baru untuk meningkatkan akurasi pengenalan gestur tangan melalui modifikasi seleksi fitur. Tahapan yang diusulkan dimulai dari perekaman data, menghitung simpangan baku dari tiap fitur. Kemudian, dilakukan proses pencarian korelasi positif antar fitur yang telah dihitung menggunakan fungsi probability density. Fitur berkorelasi positif dapat dilihat setelah disusun membentuk kurva gaussian mixture model(GMM) untuk kemudian digunakan dalam proses pengenalan gestur. Seluruh rangkaian proses disebut dengan HybridGMM. Pengklasifikasian menggunakan seleksi fitur Hybrid-GMM dibandingkan dengan seleksi fitur menggunakan PCA Kovarian. Diperoleh peningkatan akurasi pengenalan gestur menggunakan Hybrid-GMM pada pengklasifikasi k-NN dan SVM sebesar 99.48% dan 99.9%, mengungguli PCA Kovarian dengan selisih 0.59% dan 0.9%. Penggunaan Hybrid-GMM pada pengklasifikasi single LSTM dan tanpa penambahan Reset Gate menghasilkan akurasi 100%. ================================================================================ Exploration and interaction of humans in a virtual world that is done with a 3D sensor touch a variety of activities in human life. Especially for interactions with virtual objects experiencing rapid progress in terms of the variety of gestures used. Efforts to improve gesture recognition performance have been carried out, including using various classifiers such as Naive Bayes, Quaternion Dynamic Time Warping, Hidden Conditional Neural Fields, Two Stacked Long Short-Term Memory (LSTM) and Modified LSTM through the addition of Reset Gate. However, there is still a gap, namely the relatively low accuracy of gesture recognition by classifiers. This is caused by not using feature selection in data-set preparation. The purpose of this study is to offer a new approach to improve the accuracy of hand gesture recognition through modification of feature selection. The proposed stage starts with recording data, calculating the standard deviation of each feature. Then, the process of finding a positive correlation between features has been calculated using the probability density function. Positively correlated features can be seen after they have been arranged to form a gaussian mixture model (GMM) curve for later use in the gesture recognition process. The whole set of processes is called Hybrid-GMM. Classification using Hybrid-GMM feature selection compared to feature selection using PCA Covariance. Improved accuracy of gesture recognition using Hybrid-GMM in k-NN and SVM classifiers is 99.48% and 99.9%, ahead of PCA Covariance by 0.59% and 0.9% difference. The use of Hybrid-GMM in the single LSTM classifier and without the addition of Reset Gate produces 100% accuracy