Human gesture classification by brute-force machine learning for exergaming in physiotherapy

In this paper, a novel approach for human gesture classification on skeletal data is proposed for the application of exergaming in physiotherapy. Unlike existing methods, we propose to use a general classifier like Random Forests to recognize dynamic gestures. The temporal dimension is handled afterwards by majority voting in a sliding window over the consecutive predictions of the classifier. The gestures can have partially similar postures, such that the classifier will decide on the dissimilar postures. This brute-force classification strategy is permitted, because dynamic human gestures show sufficient dissimilar postures. Online continuous human gesture recognition can classify dynamic gestures in an early stage, which is a crucial advantage when controlling a game by automatic gesture recognition. Also, ground truth can be easily obtained, since all postures in a gesture get the same label, without any discretization into consecutive postures. This way, new gestures can be easily added, which is advantageous in adaptive game development. We evaluate our strategy by a leave-one-subject-out cross-validation on a self-captured stealth game gesture dataset and the publicly available Microsoft Research Cambridge-12 Kinect (MSRC-12) dataset. On the first dataset we achieve an excellent accuracy rate of 96.72%. Furthermore, we show that Random Forests perform better than Support Vector Machines. On the second dataset we achieve an accuracy rate of 98.37%, which is on average 3.57% better then existing methods

Smart Chair for Monitoring of Sitting Behavior

Sitting is a common behavior of human body in daily life. It is found that poor sitting postures can link to pains and other complications for people in literature. In order to avoid the adverse effects of poor sitting behavior, we have developed a highly practical design of smart chair system in this paper, which is able to monitor the sitting behavior of human body accurately and non-invasively. The pressure patterns of eight standardized sitting postures of human subjects were acquired and transmitted to the computer for the automatic sitting posture recognition with the application of artificial neural network classifier. The experimental results showed that it can recognize eight sitting postures of human subjects with high accuracy. The sitting posture monitoring in the developed smart chair system can help or promote people to achieve and maintain healthy sitting behavior, and prevent or reduce the chronic disease caused by poor sitting behavior. These promising results suggested that the presented system is feasible for sitting behavior monitoring, which can find applications in many areas including healthcare services, human-computer interactions and intelligent environment

Assentication: User Deauthentication and Lunchtime Attack Mitigation with Seated Posture Biometric

Biometric techniques are often used as an extra security factor in authenticating human users. Numerous biometrics have been proposed and evaluated, each with its own set of benefits and pitfalls. Static biometrics (such as fingerprints) are geared for discrete operation, to identify users, which typically involves some user burden. Meanwhile, behavioral biometrics (such as keystroke dynamics) are well suited for continuous, and sometimes more unobtrusive, operation. One important application domain for biometrics is deauthentication, a means of quickly detecting absence of a previously authenticated user and immediately terminating that user's active secure sessions. Deauthentication is crucial for mitigating so called Lunchtime Attacks, whereby an insider adversary takes over (before any inactivity timeout kicks in) authenticated state of a careless user who walks away from her computer. Motivated primarily by the need for an unobtrusive and continuous biometric to support effective deauthentication, we introduce PoPa, a new hybrid biometric based on a human user's seated posture pattern. PoPa captures a unique combination of physiological and behavioral traits. We describe a low cost fully functioning prototype that involves an office chair instrumented with 16 tiny pressure sensors. We also explore (via user experiments) how PoPa can be used in a typical workplace to provide continuous authentication (and deauthentication) of users. We experimentally assess viability of PoPa in terms of uniqueness by collecting and evaluating posture patterns of a cohort of users. Results show that PoPa exhibits very low false positive, and even lower false negative, rates. In particular, users can be identified with, on average, 91.0% accuracy. Finally, we compare pros and cons of PoPa with those of several prominent biometric based deauthentication techniques

Machine Learning Methods for Classifying Human Physical Activity from On-Body Accelerometers

The use of on-body wearable sensors is widespread in several academic and industrial domains. Of great interest are their applications in ambulatory monitoring and pervasive computing systems; here, some quantitative analysis of human motion and its automatic classification are the main computational tasks to be pursued. In this paper, we discuss how human physical activity can be classified using on-body accelerometers, with a major emphasis devoted to the computational algorithms employed for this purpose. In particular, we motivate our current interest for classifiers based on Hidden Markov Models (HMMs). An example is illustrated and discussed by analysing a dataset of accelerometer time series

Development of a Wearable Sensor-Based Framework for the Classification and Quantification of High Knee Flexion Exposures in Childcare

Repetitive cyclic and prolonged joint loading in high knee flexion postures has been associated with the progression of degenerative knee joint diseases and knee osteoarthritis (OA). Despite this association, high flexion postures, where the knee angle exceeds 120°, are commonly performed within occupational settings. While work related musculoskeletal disorders have been studied across many occupations, the risk of OA development associated with the adoption of high knee flexion postures in childcare workers has until recently been unexplored; and therefore, occupational childcare has not appeared in any systematic reviews seeking to prove a causal relationship between occupational exposures and the risk of knee OA development. Therefore, the overarching goal of this thesis was to explore the adoption of high flexion postures in childcare settings and to develop a means by which these could be measured using non-laboratory-based technologies. The global objectives of this thesis were to (i) identify the postural demands of occupational childcare as they relate to high flexion exposures at the knee, (ii) apply, extend, and validate sensor to segment alignment algorithms through which lower limb flexion-extension kinematics could be measured in multiple high knee flexion postures using inertial measurement units (IMUs), and (iii) develop a machine learning based classification model capable of identifying each childcare-inspired high knee flexion posture. In-line with these global objectives, four independent studies were conducted.   Study I – Characterization of Postures of High Knee Flexion and Lifting Tasks Associated with Occupational Childcare Background: High knee flexion postures, despite their association with increased incidences of osteoarthritis, are frequently adopted in occupational childcare. High flexion exposure thresholds (based on exposure frequency or cumulative daily exposure) that relate to increased incidences of OA have previously been proposed; yet our understanding of how the specific postural requirements of this childcare compare to these thresholds remains limited. Objectives: This study sought to define and quantify high flexion postures typically adopted in childcare to evaluate any increased likelihood of knee osteoarthritis development. Methods: Video data of eighteen childcare workers caring for infant, toddler, and preschool-aged children over a period of approximately 3.25 hours were obtained for this investigation from a larger cohort study conducted across five daycares in Kingston, Ontario, Canada. Each video was segmented to identify the start and end of potential high knee flexion exposures. Each identified posture was quantified by duration and frequency. An analysis of postural adoption by occupational task was subsequently performed to determine which task(s) might pose the greatest risk for cumulative joint trauma. Results: A total of ten postures involving varying degrees of knee flexion were identified, of which 8 involved high knee flexion. Childcare workers caring for children of all ages were found to adopt high knee flexion postures for durations of 1.45±0.15 hours and frequencies of 128.67±21.45 over the 3.25 hour observation period, exceeding proposed thresholds for incidences of knee osteoarthritis development. Structured activities, playing, and feeding tasks were found to demand the greatest adoption of high flexion postures. Conclusions: Based on the findings of this study, it is likely that childcare workers caring for children of all ages exceed cumulative exposure- and frequency-based thresholds associated with increased incidences of knee OA development within a typical working day. Study II – Evaluating the Robustness of Automatic IMU Calibration for Lower Extremity Motion Analysis in High Knee Flexion Postures Background: While inertial measurement units promise an out- of-the-box, minimally intrusive means of objectively measuring body segment kinematics in any setting, in practice their implementation requires complex calculations in order to align each sensor with the coordinate system of the segment to which they are attached. Objectives: This study sought to apply and extend previously proposed alignment algorithms to align inertial sensors with the segments on which they are attached in order to calculate flexion-extension angles for the ankle, knee, and hip during multiple childcare-inspired postures. Methods: The Seel joint axis algorithm and the Constrained Seel Knee Axis (CSKA) algorithm were implemented for the sensor to segment calibration of acceleration and angular velocity data from IMUs mounted on the lower limbs and pelvis, based on a series of calibration movements about each joint. Further, the Iterative Seel spherical axis (ISSA) extension to this implementation was proposed for the calibration of sensors about the ankle and hip. The performance of these algorithms was validated across fifty participants during ten childcare-inspired movements performed by comparing IMU- and gold standard optical-based flexion-extension angle estimates. Results: Strong correlations between the IMU- and optical-based angle estimates were reported for all joints during each high flexion motion with the exception of a moderate correlation reported for the ankle angle estimate during child chair sitting. Mean RMSE between protocols were found to be 6.61° ± 2.96° for the ankle, 7.55° ± 5.82° for the knee, and 14.64° ± 6.73° for the hip. Conclusions: The estimation of joint kinematics through the IMU-based CSKA and ISSA algorithms presents an effective solution for the sensor to segment calibration of inertial sensors, allowing for the calculation of lower limb flexion-extension kinematics in multiple childcare-inspired high knee flexion postures. Study III – A Multi-Dimensional Dynamic Time Warping Distance-Based Framework for the Recognition of High Knee Flexion Postures in Inertial Sensor Data Background: The interpretation of inertial measures as they relate to occupational exposures is non-trivial. In order to relate the continuously collected data to the activities or postures performed by the sensor wearer, pattern recognition and machine learning based algorithms can be applied. One difficulty in applying these techniques to real-world data lies in the temporal and scale variability of human movements, which must be overcome when seeking to classify data in the time-domain. Objectives: The objective of this study was to develop a sensor-based framework for the detection and measurement of isolated childcare-specific postures (identified in Study I). As a secondary objective, the classification accuracy movements performed under loaded and unloaded conditions were compared in order to assess the sensitivity of the developed model to potential postural variabilities accompanying the presence of a load. Methods: IMU-based joint angle estimates for the ankle, knee, and hip were time and scale normalized prior to being input to a multi-dimensional Dynamic Time Warping (DTW) distance-based Nearest Neighbour algorithm for the identification of twelve childcare inspired postures. Fifty participants performed each posture, when possible, under unloaded and loaded conditions. Angle estimates from thirty-five participants were divided into development and testing data, such that 80% of the trials were segmented into movement templates and the remaining 20% were left as continuous movement sequences. These data were then included in the model building and testing phases while the accuracy of the model was validated based on novel data from fifteen participants. Results: Overall accuracies of 82.3% and 55.6% were reached when classifying postures on testing and validation data respectively. When adjusting for the imbalances between classification groups, mean balanced accuracies increased to 86% and 74.6% for testing and validation data respectively. Sensitivity and specificity values revealed the highest rates of misclassifications occurred between flatfoot squatting, heels-up squatting, and stooping. It was also found that the model was not capable of identifying sequences of walking data based on a single step motion template. No significant differences were found between the classification of loaded and unloaded motion trials. Conclusions: A combination of DTW distances calculated between motion templates and continuous movement sequences of lower limb flexion-extension angles was found to be effective in the identification of isolated postures frequently performed in childcare. The developed model was successful at classifying data from participants both included and precluded from the algorithm building dataset and insensitive to postural variability which might be caused by the presence of a load. Study IV – Evaluating the Feasibility of Applying the Developed Multi-Dimensional Dynamic Time Warping Distance-Based Framework to the Measurement and Recognition of High Knee Flexion Postures in a Simulated Childcare Environment Background: While the simulation of high knee flexion postures in isolation (in Study III) provided a basis for the development of a multi-dimensional Dynamic Time Warping based nearest neighbour algorithm for the identification of childcare-inspired postures, it is unlikely that the postures adopted in childcare settings would be performed in isolation. Objectives: This study sought to explore the feasibility of extending the developed classification algorithm to identify and measure postures frequently adopted when performing childcare specific tasks within a simulated childcare environment. Methods: Lower limb inertial motion data was recorded from twelve participants as they interacted with their child during a series of tasks inspired by those identified in Study I as frequently occurring in childcare settings. In order to reduce the error associated with gyroscopic drift over time, joint angles for each trial were calculated over 60 second increments and concatenated across the duration of each trial. Angle estimates from ten participants were time windowed in order to create the inputs for the development and testing of two model designs wherein: (A) the model development data included all templates generated from Study III as well as continuous motion windows here collected, or (B) only the model development data included only windows of continuous motion data. The division of data into the development and testing datasets for each 5-fold cross-validated classification model was performed in one of two ways wherein the data was divided: (a) through stratified randomized partitioning of windows such that 80% were assigned to model development and the remaining 20% were reserved for testing, or (b) by partitioning all windows from a single trial of a single participant for testing while all remaining windows were assigned to the model development dataset. When the classification of continuously collected windows was tested (using division strategy b), a logic-based correction module was introduced to eliminate any erroneous predictions. Each model design (A and B) was developed and tested using both data division strategies (a and b) and subsequently their performance was evaluated based on the classification of all data windows from the two subjects reserved for validation. Results: Classification accuracies of 42.2% and 42.5% were achieved when classifying the testing data separated through stratified random partitioning (division strategy a) using models that included (model A, 159 classes) or excluded (model B, 149 classes) the templates generated from Study III, respectively. This classification accuracy was found to decrease when classifying a test partition which included all windows of a single trial (division strategy b) to 35.4% when using model A (where templates from Study III were included in the model development dataset); however, this same trial was classified with an accuracy of 80.8% when using model B (whose development dataset included only windows of continuous motion data). This accuracy was however found to be highly dependent on the motions performed in a given trial and logic-based corrections were not found to improve classification accuracies. When validating each model by identifying postures performed by novel subjects, classification accuracies of 24.0% and 26.6% were obtained using development data which included (model A) and excluded (model B) templates from Study III, respectively. Across all novel data, the highest classification accuracies were observed when identifying static postures, which is unsurprising given that windows of these postures were most prevalent in the model development datasets. Conclusions: While classification accuracies above those achievable by chance were achieved, the classification models evaluated in this study were incapable of accurately identifying the postures adopted during simulated childcare tasks to a level that could be considered satisfactory to accurately report on the postures assumed in a childcare environment. The success of the classifier was highly dependent on the number of transitions occurring between postures while in high flexion; therefore, more classifier development data is needed to create templates for these novel transition movements. Given the high variability in postural adoption when caring for and interacting with children, additional movement templates based on continuously collected data would be required for the successful identification of postures in occupational settings. Global Conclusions Childcare workers exceed previously reported thresholds for high knee flexion postures based on cumulative exposure and frequency of adoption associated with increased incidences of knee OA development within a typical working day. Inertial measurement units provide a unique means of objectively measuring postures frequently adopted when caring for children which may ultimately permit the quantification of high knee flexion exposures in childcare settings and further study of the relationship between these postures and the risk of OA development in occupational childcare. While the results of this thesis demonstrate that IMU based measures of lower limb kinematics can be used to identify these postures in isolation, further work is required to expand the classification model and enable the identification of such postures from continuously collected data

A Hybrid Hierarchical Framework for Gym Physical Activity Recognition and Measurement Using Wearable Sensors

Due to the many beneficial effects on physical and mental health and strong association with many fitness and rehabilitation programs, physical activity (PA) recognition has been considered as a key paradigm for internet of things (IoT) healthcare. Traditional PA recognition techniques focus on repeated aerobic exercises or stationary PA. As a crucial indicator in human health, it covers a range of bodily movement from aerobics to anaerobic that may all bring health benefits. However, existing PA recognition approaches are mostly designed for specific scenarios and often lack extensibility for application in other areas, thereby limiting their usefulness. In this paper, we attempt to detect more gym physical activities (GPAs) in addition to traditional PA using acceleration, A two layer recognition framework is proposed that can classify aerobic, sedentary and free weight activities, count repetitions and sets for the free weight exercises, and in the meantime, measure quantities of repetitions and sets for free weight activities. In the first layer, a one-class SVM (OC-SVM) is applied to coarsely classify free weight and non-free weight activities. In the second layer, a neural network (NN) is utilized for aerobic and sedentary activities recognition; a hidden Markov model (HMM) is to provide a further classification in free weight activities. The performance of the framework was tested on 10 healthy subjects (age: 30 ± 5; BMI: 25 ± 5.5 kg/ and compared with some typical classifiers. The results indicate the proposed framework has better performance in recognizing and measuring GPAs than other approaches. The potential of this framework can be potentially extended in supporting more types of PA recognition in complex applications