Myoelectric Human Computer Interaction Using Reliable Temporal Sequence-based Myoelectric Classification for Dynamic Hand Gestures

To put a computerized device under human control, various interface techniques have been commonly studied in the realm of Human Computer Interaction (HCI) design. What this dissertation focuses on is a myoelectric interface, which controls a device via neuromuscular electrical signals. Myoelectric interface has advanced by recognizing repeated patterns of the signal (pattern recognition-based myoelectric classification). However, when the myoelectric classification is used to extract multiple discrete states within limited muscle sites, there are robustness issues due to external conditions: limb position changes, electrode shifts, and skin condition changes. Examined in this dissertation is the robustness issue, or drop in the performance of the myoelectric classification when the limb position varies from the position where the system was trained. Two research goals outlined in this dissertation are to increase reliability of myoelectric system and to build a myoelectric HCI to manipulate a 6-DOF robot arm with a 1-DOF gripper. To tackle the robustness issue, the proposed method uses dynamic motions which change their poses and configuration over time. The method assumes that using dynamic motions is more reliable, vis-a-vis the robustness issues, than using static motions. The robustness of the method is evaluated by choosing the training sets and validation sets at different limb positions. Next, an HCI system manipulating a 6-DOF robot arm with a 1-DOF gripper is introduced. The HCI system includes an inertia measurement unit to measure the limb orientation, as well as EMG sensors to acquire muscle force and to classify dynamic motions. Muscle force and the orientation of a forearm are used to generate velocity commands. Classified dynamic motions are used to change the manipulation modes. The performance of the myoelectric interface is measured in terms of real-time classification accuracy, path efficiency, and time-related measures. In conclusion, this dissertation proposes a reliable myoelectric classification and develops a myoelectric interface using the proposed classification method for an HCI application. The robustness of the proposed myoelectric classification is verified as compared to previous myoelectric classification approaches. The usability of the developed myoelectric interface is compared to a well-known interface

Radar and RGB-depth sensors for fall detection: a review

This paper reviews recent works in the literature on the use of systems based on radar and RGB-Depth (RGB-D) sensors for fall detection, and discusses outstanding research challenges and trends related to this research field. Systems to detect reliably fall events and promptly alert carers and first responders have gained significant interest in the past few years in order to address the societal issue of an increasing number of elderly people living alone, with the associated risk of them falling and the consequences in terms of health treatments, reduced well-being, and costs. The interest in radar and RGB-D sensors is related to their capability to enable contactless and non-intrusive monitoring, which is an advantage for practical deployment and users’ acceptance and compliance, compared with other sensor technologies, such as video-cameras, or wearables. Furthermore, the possibility of combining and fusing information from The heterogeneous types of sensors is expected to improve the overall performance of practical fall detection systems. Researchers from different fields can benefit from multidisciplinary knowledge and awareness of the latest developments in radar and RGB-D sensors that this paper is discussing

Development of a Wearable Sensor-Based Framework for the Classification and Quantification of High Knee Flexion Exposures in Childcare

Repetitive cyclic and prolonged joint loading in high knee flexion postures has been associated with the progression of degenerative knee joint diseases and knee osteoarthritis (OA). Despite this association, high flexion postures, where the knee angle exceeds 120°, are commonly performed within occupational settings. While work related musculoskeletal disorders have been studied across many occupations, the risk of OA development associated with the adoption of high knee flexion postures in childcare workers has until recently been unexplored; and therefore, occupational childcare has not appeared in any systematic reviews seeking to prove a causal relationship between occupational exposures and the risk of knee OA development. Therefore, the overarching goal of this thesis was to explore the adoption of high flexion postures in childcare settings and to develop a means by which these could be measured using non-laboratory-based technologies. The global objectives of this thesis were to (i) identify the postural demands of occupational childcare as they relate to high flexion exposures at the knee, (ii) apply, extend, and validate sensor to segment alignment algorithms through which lower limb flexion-extension kinematics could be measured in multiple high knee flexion postures using inertial measurement units (IMUs), and (iii) develop a machine learning based classification model capable of identifying each childcare-inspired high knee flexion posture. In-line with these global objectives, four independent studies were conducted.   Study I – Characterization of Postures of High Knee Flexion and Lifting Tasks Associated with Occupational Childcare Background: High knee flexion postures, despite their association with increased incidences of osteoarthritis, are frequently adopted in occupational childcare. High flexion exposure thresholds (based on exposure frequency or cumulative daily exposure) that relate to increased incidences of OA have previously been proposed; yet our understanding of how the specific postural requirements of this childcare compare to these thresholds remains limited. Objectives: This study sought to define and quantify high flexion postures typically adopted in childcare to evaluate any increased likelihood of knee osteoarthritis development. Methods: Video data of eighteen childcare workers caring for infant, toddler, and preschool-aged children over a period of approximately 3.25 hours were obtained for this investigation from a larger cohort study conducted across five daycares in Kingston, Ontario, Canada. Each video was segmented to identify the start and end of potential high knee flexion exposures. Each identified posture was quantified by duration and frequency. An analysis of postural adoption by occupational task was subsequently performed to determine which task(s) might pose the greatest risk for cumulative joint trauma. Results: A total of ten postures involving varying degrees of knee flexion were identified, of which 8 involved high knee flexion. Childcare workers caring for children of all ages were found to adopt high knee flexion postures for durations of 1.45±0.15 hours and frequencies of 128.67±21.45 over the 3.25 hour observation period, exceeding proposed thresholds for incidences of knee osteoarthritis development. Structured activities, playing, and feeding tasks were found to demand the greatest adoption of high flexion postures. Conclusions: Based on the findings of this study, it is likely that childcare workers caring for children of all ages exceed cumulative exposure- and frequency-based thresholds associated with increased incidences of knee OA development within a typical working day. Study II – Evaluating the Robustness of Automatic IMU Calibration for Lower Extremity Motion Analysis in High Knee Flexion Postures Background: While inertial measurement units promise an out- of-the-box, minimally intrusive means of objectively measuring body segment kinematics in any setting, in practice their implementation requires complex calculations in order to align each sensor with the coordinate system of the segment to which they are attached. Objectives: This study sought to apply and extend previously proposed alignment algorithms to align inertial sensors with the segments on which they are attached in order to calculate flexion-extension angles for the ankle, knee, and hip during multiple childcare-inspired postures. Methods: The Seel joint axis algorithm and the Constrained Seel Knee Axis (CSKA) algorithm were implemented for the sensor to segment calibration of acceleration and angular velocity data from IMUs mounted on the lower limbs and pelvis, based on a series of calibration movements about each joint. Further, the Iterative Seel spherical axis (ISSA) extension to this implementation was proposed for the calibration of sensors about the ankle and hip. The performance of these algorithms was validated across fifty participants during ten childcare-inspired movements performed by comparing IMU- and gold standard optical-based flexion-extension angle estimates. Results: Strong correlations between the IMU- and optical-based angle estimates were reported for all joints during each high flexion motion with the exception of a moderate correlation reported for the ankle angle estimate during child chair sitting. Mean RMSE between protocols were found to be 6.61° ± 2.96° for the ankle, 7.55° ± 5.82° for the knee, and 14.64° ± 6.73° for the hip. Conclusions: The estimation of joint kinematics through the IMU-based CSKA and ISSA algorithms presents an effective solution for the sensor to segment calibration of inertial sensors, allowing for the calculation of lower limb flexion-extension kinematics in multiple childcare-inspired high knee flexion postures. Study III – A Multi-Dimensional Dynamic Time Warping Distance-Based Framework for the Recognition of High Knee Flexion Postures in Inertial Sensor Data Background: The interpretation of inertial measures as they relate to occupational exposures is non-trivial. In order to relate the continuously collected data to the activities or postures performed by the sensor wearer, pattern recognition and machine learning based algorithms can be applied. One difficulty in applying these techniques to real-world data lies in the temporal and scale variability of human movements, which must be overcome when seeking to classify data in the time-domain. Objectives: The objective of this study was to develop a sensor-based framework for the detection and measurement of isolated childcare-specific postures (identified in Study I). As a secondary objective, the classification accuracy movements performed under loaded and unloaded conditions were compared in order to assess the sensitivity of the developed model to potential postural variabilities accompanying the presence of a load. Methods: IMU-based joint angle estimates for the ankle, knee, and hip were time and scale normalized prior to being input to a multi-dimensional Dynamic Time Warping (DTW) distance-based Nearest Neighbour algorithm for the identification of twelve childcare inspired postures. Fifty participants performed each posture, when possible, under unloaded and loaded conditions. Angle estimates from thirty-five participants were divided into development and testing data, such that 80% of the trials were segmented into movement templates and the remaining 20% were left as continuous movement sequences. These data were then included in the model building and testing phases while the accuracy of the model was validated based on novel data from fifteen participants. Results: Overall accuracies of 82.3% and 55.6% were reached when classifying postures on testing and validation data respectively. When adjusting for the imbalances between classification groups, mean balanced accuracies increased to 86% and 74.6% for testing and validation data respectively. Sensitivity and specificity values revealed the highest rates of misclassifications occurred between flatfoot squatting, heels-up squatting, and stooping. It was also found that the model was not capable of identifying sequences of walking data based on a single step motion template. No significant differences were found between the classification of loaded and unloaded motion trials. Conclusions: A combination of DTW distances calculated between motion templates and continuous movement sequences of lower limb flexion-extension angles was found to be effective in the identification of isolated postures frequently performed in childcare. The developed model was successful at classifying data from participants both included and precluded from the algorithm building dataset and insensitive to postural variability which might be caused by the presence of a load. Study IV – Evaluating the Feasibility of Applying the Developed Multi-Dimensional Dynamic Time Warping Distance-Based Framework to the Measurement and Recognition of High Knee Flexion Postures in a Simulated Childcare Environment Background: While the simulation of high knee flexion postures in isolation (in Study III) provided a basis for the development of a multi-dimensional Dynamic Time Warping based nearest neighbour algorithm for the identification of childcare-inspired postures, it is unlikely that the postures adopted in childcare settings would be performed in isolation. Objectives: This study sought to explore the feasibility of extending the developed classification algorithm to identify and measure postures frequently adopted when performing childcare specific tasks within a simulated childcare environment. Methods: Lower limb inertial motion data was recorded from twelve participants as they interacted with their child during a series of tasks inspired by those identified in Study I as frequently occurring in childcare settings. In order to reduce the error associated with gyroscopic drift over time, joint angles for each trial were calculated over 60 second increments and concatenated across the duration of each trial. Angle estimates from ten participants were time windowed in order to create the inputs for the development and testing of two model designs wherein: (A) the model development data included all templates generated from Study III as well as continuous motion windows here collected, or (B) only the model development data included only windows of continuous motion data. The division of data into the development and testing datasets for each 5-fold cross-validated classification model was performed in one of two ways wherein the data was divided: (a) through stratified randomized partitioning of windows such that 80% were assigned to model development and the remaining 20% were reserved for testing, or (b) by partitioning all windows from a single trial of a single participant for testing while all remaining windows were assigned to the model development dataset. When the classification of continuously collected windows was tested (using division strategy b), a logic-based correction module was introduced to eliminate any erroneous predictions. Each model design (A and B) was developed and tested using both data division strategies (a and b) and subsequently their performance was evaluated based on the classification of all data windows from the two subjects reserved for validation. Results: Classification accuracies of 42.2% and 42.5% were achieved when classifying the testing data separated through stratified random partitioning (division strategy a) using models that included (model A, 159 classes) or excluded (model B, 149 classes) the templates generated from Study III, respectively. This classification accuracy was found to decrease when classifying a test partition which included all windows of a single trial (division strategy b) to 35.4% when using model A (where templates from Study III were included in the model development dataset); however, this same trial was classified with an accuracy of 80.8% when using model B (whose development dataset included only windows of continuous motion data). This accuracy was however found to be highly dependent on the motions performed in a given trial and logic-based corrections were not found to improve classification accuracies. When validating each model by identifying postures performed by novel subjects, classification accuracies of 24.0% and 26.6% were obtained using development data which included (model A) and excluded (model B) templates from Study III, respectively. Across all novel data, the highest classification accuracies were observed when identifying static postures, which is unsurprising given that windows of these postures were most prevalent in the model development datasets. Conclusions: While classification accuracies above those achievable by chance were achieved, the classification models evaluated in this study were incapable of accurately identifying the postures adopted during simulated childcare tasks to a level that could be considered satisfactory to accurately report on the postures assumed in a childcare environment. The success of the classifier was highly dependent on the number of transitions occurring between postures while in high flexion; therefore, more classifier development data is needed to create templates for these novel transition movements. Given the high variability in postural adoption when caring for and interacting with children, additional movement templates based on continuously collected data would be required for the successful identification of postures in occupational settings. Global Conclusions Childcare workers exceed previously reported thresholds for high knee flexion postures based on cumulative exposure and frequency of adoption associated with increased incidences of knee OA development within a typical working day. Inertial measurement units provide a unique means of objectively measuring postures frequently adopted when caring for children which may ultimately permit the quantification of high knee flexion exposures in childcare settings and further study of the relationship between these postures and the risk of OA development in occupational childcare. While the results of this thesis demonstrate that IMU based measures of lower limb kinematics can be used to identify these postures in isolation, further work is required to expand the classification model and enable the identification of such postures from continuously collected data