Human Activity Recognition and Control of Wearable Robots

abstract: Wearable robotics has gained huge popularity in recent years due to its wide applications in rehabilitation, military, and industrial fields. The weakness of the skeletal muscles in the aging population and neurological injuries such as stroke and spinal cord injuries seriously limit the abilities of these individuals to perform daily activities. Therefore, there is an increasing attention in the development of wearable robots to assist the elderly and patients with disabilities for motion assistance and rehabilitation. In military and industrial sectors, wearable robots can increase the productivity of workers and soldiers. It is important for the wearable robots to maintain smooth interaction with the user while evolving in complex environments with minimum effort from the user. Therefore, the recognition of the user's activities such as walking or jogging in real time becomes essential to provide appropriate assistance based on the activity. This dissertation proposes two real-time human activity recognition algorithms intelligent fuzzy inference (IFI) algorithm and Amplitude omega ($A \omega$) algorithm to identify the human activities, i.e., stationary and locomotion activities. The IFI algorithm uses knee angle and ground contact forces (GCFs) measurements from four inertial measurement units (IMUs) and a pair of smart shoes. Whereas, the $A \omega$ algorithm is based on thigh angle measurements from a single IMU. This dissertation also attempts to address the problem of online tuning of virtual impedance for an assistive robot based on real-time gait and activity measurement data to personalize the assistance for different users. An automatic impedance tuning (AIT) approach is presented for a knee assistive device (KAD) in which the IFI algorithm is used for real-time activity measurements. This dissertation also proposes an adaptive oscillator method known as amplitude omega adaptive oscillator ($A\omega AO$) method for HeSA (hip exoskeleton for superior augmentation) to provide bilateral hip assistance during human locomotion activities. The $A \omega$ algorithm is integrated into the adaptive oscillator method to make the approach robust for different locomotion activities. Experiments are performed on healthy subjects to validate the efficacy of the human activities recognition algorithms and control strategies proposed in this dissertation. Both the activity recognition algorithms exhibited higher classification accuracy with less update time. The results of AIT demonstrated that the KAD assistive torque was smoother and EMG signal of Vastus Medialis is reduced, compared to constant impedance and finite state machine approaches. The $A\omega AO$ method showed real-time learning of the locomotion activities signals for three healthy subjects while wearing HeSA. To understand the influence of the assistive devices on the inherent dynamic gait stability of the human, stability analysis is performed. For this, the stability metrics derived from dynamical systems theory are used to evaluate unilateral knee assistance applied to the healthy participants.Dissertation/ThesisDoctoral Dissertation Aerospace Engineering 201

Motor control-based assessment of therapy effects in individuals post-stroke: implications for prediction of response and subject-specific modifications

Producing a coordinated motion such as walking is, at its root, the result of healthy communication pathways between the central nervous system and the musculoskeletal system. The central nervous system produces an electrical signal responsible for the excitation of a muscle, and the musculoskeletal system contains the necessary equipment for producing a movement-driving force to achieve a desired motion. Motor control refers to the ability an individual has to produce a desired motion, and the complexity of motor control is a mathematical concept stemming from how the electrical signals from the central nervous system translate to muscle activations. Exercising a high-level complexity of motor control is critical to producing a smooth motion. However, the occurrence of a sudden, detrimental neurological event like a stroke damages these connecting pathways between these two systems, and the result is a motion that is uncoordinated and energy-inefficient due to diminished motor control complexity. Stroke is a leading cause of disability with nearly 800,000 stroke victims each year in the U.S. alone, amounting to an estimated cost of $45.5B. Impaired mobility following a stroke is a widespread effect, with more than half of survivors over the age of 65 affected in this way, and up to 80% of survivors at some point experiencing hemiparesis during post-stroke recovery. As such, given the importance of independent mobility for quality of life, improving gait mechanics and mobility of stroke survivors has been the goal of rehabilitation efforts for decades. In this work, we mold together the forefronts of statistics and computational physics-based modeling to obtain insight and information about post-stroke hemiparetic gait mechanics and what drives them that would otherwise be unavailable. We expand upon previous work to quantify motor control complexity as it relates to the health of the neuromuscular system and analyze the effect of a specific therapy on motor control of individuals post-stroke. Secondly, we aim to develop a predictive model to conclude whether an individual will respond to the therapy based on kinematic and dynamic features from pre-therapy recordings. Lastly, we will determine how to individually tailor this therapy in order to achieve maximum improvement in motor control complexity in order to improve gait mechanics in individuals post-stroke

Humanoid Robots

For many years, the human being has been trying, in all ways, to recreate the complex mechanisms that form the human body. Such task is extremely complicated and the results are not totally satisfactory. However, with increasing technological advances based on theoretical and experimental researches, man gets, in a way, to copy or to imitate some systems of the human body. These researches not only intended to create humanoid robots, great part of them constituting autonomous systems, but also, in some way, to offer a higher knowledge of the systems that form the human body, objectifying possible applications in the technology of rehabilitation of human beings, gathering in a whole studies related not only to Robotics, but also to Biomechanics, Biomimmetics, Cybernetics, among other areas. This book presents a series of researches inspired by this ideal, carried through by various researchers worldwide, looking for to analyze and to discuss diverse subjects related to humanoid robots. The presented contributions explore aspects about robotic hands, learning, language, vision and locomotion

Bio-Inspired Robotics

Modern robotic technologies have enabled robots to operate in a variety of unstructured and dynamically-changing environments, in addition to traditional structured environments. Robots have, thus, become an important element in our everyday lives. One key approach to develop such intelligent and autonomous robots is to draw inspiration from biological systems. Biological structure, mechanisms, and underlying principles have the potential to provide new ideas to support the improvement of conventional robotic designs and control. Such biological principles usually originate from animal or even plant models, for robots, which can sense, think, walk, swim, crawl, jump or even fly. Thus, it is believed that these bio-inspired methods are becoming increasingly important in the face of complex applications. Bio-inspired robotics is leading to the study of innovative structures and computing with sensory–motor coordination and learning to achieve intelligence, flexibility, stability, and adaptation for emergent robotic applications, such as manipulation, learning, and control. This Special Issue invites original papers of innovative ideas and concepts, new discoveries and improvements, and novel applications and business models relevant to the selected topics of ``Bio-Inspired Robotics''. Bio-Inspired Robotics is a broad topic and an ongoing expanding field. This Special Issue collates 30 papers that address some of the important challenges and opportunities in this broad and expanding field

Foot Motion-Based Falling Risk Evaluation for Patients with Parkinson’s Disease

Parkinson’s disease (PD) affects motor functionalities, which are closely associated with increased risks of falling and decreased quality of life. However, there is no easy-to-use definitive tools for PD patients to quantify their falling risks at home. To address this, in this dissertation, we develop Monitoring Insoles (MONI) with advanced data processing techniques to score falling risks of PD patients following Falling Risk Questionnaire (FRQ) developed by the U.S. Centers for Disease Control and Prevention (CDC). To achieve this, we extract motion tasks from daily activities and select the most representative features associated with PD that facilitate accurate falling risk scoring. To address the challenge in uncontrolled daily life environments and to identify the most representative features associated with PD and falling risks, the proposed data processing method firstly recognizes foot motions such as walking and toe tapping from continuous movements with stride detection and fast labeling framework, and then extracts time-axis and acceleration-axis features from the motion tasks, at the end provides a score of falling risks using regression. The data processing method can be integrated into a mobile game to be used at home with MONI. The main contributions of this dissertation includes: (i) developing MONI as a low power solution for daily life use; (ii) utilizing stride detection and developing fast labeling framework for motion recognition that improves recognition accuracy for daily life applications; (iii) analyzing two walking and two toe tapping tasks that are close to real life scenarios and identifying important features associated with PD and falling risks; (iv) providing falling scores as quantitative evaluation to PD patients in daily life through simple foot motion tasks and setups

Biomechatronics: Harmonizing Mechatronic Systems with Human Beings

This eBook provides a comprehensive treatise on modern biomechatronic systems centred around human applications. A particular emphasis is given to exoskeleton designs for assistance and training with advanced interfaces in human-machine interaction. Some of these designs are validated with experimental results which the reader will find very informative as building-blocks for designing such systems. This eBook will be ideally suited to those researching in biomechatronic area with bio-feedback applications or those who are involved in high-end research on manmachine interfaces. This may also serve as a textbook for biomechatronic design at post-graduate level

Wearable Sensors and Machine Learning based Human Movement Analysis – Applications in Sports and Medicine

Die Analyse menschlicher Bewegung außerhalb des Labors unter realen Bedingungen ist in den letzten Jahren sowohl in sportlichen als auch in medizinischen Anwendungen zunehmend bedeutender geworden. Mobile Sensoren, welche am Körper getragen werden, haben sich in diesem Zusammenhang als wertvolle Messinstrumente etabliert. Auf Grund des Umfangs, der Komplexität, der Heterogenität und der Störanfälligkeit der Daten werden vielseitige Analysemethoden eingesetzt, um die Daten zu verarbeiten und auszuwerten. Zudem sind häufig Modellierungsansätze notwendig, da die gemessenen Größen nicht auf direktem Weg aussagekräftige biomechanische Variablen liefern. Seit wenigen Jahren haben sich hierfür Methoden des maschinellen Lernens als vielversprechende Instrumente zur Ermittlung von Zielvariablen, wie beispielsweise der Gelenkwinkel, herausgestellt. Aktuell befindet sich die Forschung an der Schnittstelle aus Biomechanik, mobiler Sensoren und maschinellem Lernen noch am Anfang. Der Bereich birgt grundsätzlich ein erhebliches Potenzial, um einerseits das Spektrum an mobilen Anwendungen im Sport, insbesondere in Sportarten mit komplexen Bewegungsanforderungen, wie beispielsweise dem Eishockey, zu erweitern. Andererseits können Methoden des maschinellen Lernens zur Abschätzung von Belastungen auf Körperstrukturen mittels mobiler Sensordaten genutzt werden. Vor allem die Anwendung mobiler Sensoren in Kombination mit Prädiktionsmodellen zur Ermittlung der Kniegelenkbelastung, wie beispielsweise der Gelenkmomente, wurde bisher nur unzureichend erforscht. Gleichwohl kommt der mobilen Erfassung von Gelenkbelastungen in der Diagnostik und Rehabilitation von Verletzungen sowie Muskel-Skelett-Erkrankungen eine zentrale Bedeutung zu. Das übergeordnete Ziel dieser Dissertation ist es, festzustellen inwieweit tragbare Sensoren und Verfahren des maschinellen Lernens zur Quantifizierung sportlicher Bewegungsmerkmale sowie zur Ermittlung der Belastung von Körperstrukturen bei der Ausführung von Alltags- und Sportbewegungen eingesetzt werden können. Die Dissertation basiert auf vier Studien, welche in internationalen Fachzeitschriften mit Peer-Review-Prozess erschienen sind. Die ersten beiden Studien konzentrieren sich zum einen auf die automatisierte Erkennung von zeitlichen Events und zum anderen auf die mobile Leistungsanalyse während des Schlittschuhlaufens im Eishockey. Die beiden weiteren Studien präsentieren jeweils einen neuartigen Ansatz zur Schätzung von Belastungen im Kniegelenk mittels künstlich neuronalen Netzen. Zwei mobile Sensoren, welche in eine Kniebandage integriert sind, dienen hierbei als Datenbasis zur Ermittlung von Kniegelenkskräften während unterschiedlicher Sportbewegungen sowie von Kniegelenksmomenten während verschiedener Lokomotionsaufgaben. Studie I zeigt eine präzise, effiziente und einfache Methode zur zeitlichen Analyse des Schlittschuhlaufens im Eishockey mittels einem am Schlittschuh befestigten Beschleunigungssensor. Die Validierung des neuartigen Ansatzes erfolgt anhand synchroner Messungen des plantaren Fußdrucks. Der mittlere Unterschied zwischen den beiden Erfassungsmethoden liegt sowohl für die Standphasendauer als auch der Gangzyklusdauer unter einer Millisekunde. Studie II zeigt das Potenzial von Beschleunigungssensoren zur Technik- und Leistungsanalyse des Schlittschuhlaufens im Eishockey. Die Ergebnisse zeigen für die Standphasendauer und Schrittintensität sowohl Unterschiede zwischen beschleunigenden Schritten und Schritten bei konstanter Geschwindigkeit als auch zwischen Teilnehmern unterschiedlichen Leistungsniveaus. Eine Korrelationsanalyse offenbart, insbesondere für die Schrittintensität, einen starken Zusammenhang mit der sportlichen Leistung des Schlittschuhlaufens im Sinne einer verkürzten Sprintzeit. Studie III präsentiert ein tragbares System zur Erfassung von Belastungen im Kniegelenk bei verschiedenen sportlichen Bewegungen auf Basis zweier mobiler Sensoren. Im Speziellen werden unterschiedliche lineare Bewegungen, Richtungswechsel und Sprünge betrachtet. Die mittels künstlich neuronalem Netz ermittelten dreidimensionalen Kniegelenkskräfte zeigen, mit Ausnahme der mediolateralen Kraftkomponente, für die meisten analysierten Bewegungen eine gute Übereinstimmung mit invers-dynamisch berechneten Referenzdaten. Die abschließende Studie IV stellt eine Erweiterung des in Studie III entwickelten tragbaren Systems zur Ermittlung von Belastungen im Kniegelenk dar. Die ambulante Beurteilung der Gelenkbelastung bei Kniearthrose steht hierbei im Fokus. Die entwickelten Prädiktionsmodelle zeigen für das Knieflexionsmoment eine gute Übereinstimmung mit invers-dynamisch berechneten Referenzdaten für den Großteil der analysierten Bewegungen. Demgegenüber ist bei der Ermittlung des Knieadduktionsmoments mittels künstlichen neuronalen Netzen Vorsicht geboten. Je nach Bewegung, kommt es zu einer schwachen bis starken Übereinstimmung zwischen der mittels Prädiktionsmodell bestimmten Belastung und dem Referenzwert. Zusammenfassend tragen die Ergebnisse von Studie I und Studie II zur sportartspezifischen Leistungsanalyse im Eishockey bei. Zukünftig können sowohl die Trainingsqualität als auch die gezielte Verbesserung sportlicher Leistung durch den Einsatz von am Körper getragener Sensoren in hohem Maße profitieren. Die methodischen Neuerungen und Erkenntnisse aus Studie III und Studie IV ebnen den Weg für die Entwicklung neuartiger Technologien im Gesundheitsbereich. Mit Blick in die Zukunft können mobile Sensoren zur intelligenten Analyse menschlicher Bewegungen sinnvoll eingesetzt werden. Die vorliegende Dissertation zeigt, dass die mobile Bewegungsanalyse zur Erleichterung der sportartspezifischen Leistungsdiagnostik unter Feldbedingungen beiträgt. Zudem zeigt die Arbeit, dass die mobile Bewegungsanalyse einen wichtigen Beitrag zur Verbesserung der Gesundheitsdiagnostik und Rehabilitation nach akuten Verletzungen oder bei chronischen muskuloskelettalen Erkrankungen leistet

IntoxiGait Deep Learning

Alcohol abuse has been a pervasive problem worldwide, causing 88,000 annual deaths. Recently, several projects have attempted to estimate a users level of intoxication by measuring gait using mobile sensors. The goal of this project was to compare a deep learning approach to previous methods to predict the blood alcohol concentration of a user by training a convolutional neural network and creating a mobile app which could accurately determine intoxication level. We gathered data from 38 participants over the course of 12 weeks, collecting accelerometer and gyroscope data simultaneously from both a smartwatch and smartphone. Our neural networks accuracy is roughly 64% on the test set and 69% on the training set into 5 BAC ranges for an input containing two seconds of data

Collaborative Artificial Intelligence Development for Social Robots

The main aim of this doctoral thesis was to investigate on how to involve a community for collaborative artificial intelligence (AI) development of a social robot. The work was initiated by the author’s personal interest in developing the Sony AIBO robots that have been unavailable on the retail markets, however, user communities with special interests in these robots remained on the internet. At first, to attract people’s attention, the author developed three specific features for the robot. These consisted of teaching the robot 1) sound event recognition in order to react to environmental audio stimuli, 2) a method to detect the underlying surface under the robot, and 3) of how to recognize its own body states. As this AI development proved to be very challenging, the author decided to start a community project for artificial intelligence development. Community involvement has a long history in open-source software projects and some robotics companies tried to benefit from their userbase in product development. An active online community of Sony AIBO owners was approached to investigate factors to engage its members in the creative processes. For this purpose, 78 Sony AIBO owners were recruited online to fill a questionnaire and their data were analyzed with respect to age, gender, culture, length of ownership, user contribution, and model preference. The results revealed the motives to own these robots for many years and how these heavy users perceived their social robots after a long period in the robot acceptance phase. For example, female participants tended to have more emotional relation to their robots than male who had more technically oriented long-term engagement motivation. The user expectations were also explored by analyzing the answers to this questionnaire to discover the key needs of this user group. The results revealed that the most-wanted skills were the interaction with humans and the autonomous operation. The integration with the AI agents and Internet services was important, but the long-term memory and learning capabilities were not so relevant for the participants. The diverse preferences for robot skills led to creating a prioritized recommendation list to complement the design guidelines for social robots in the literature. In sum, the findings of this thesis showed that developing AI features for an outdated robot is possible but takes a lot of time and shared community efforts. To involve a specific community, one needs first to build up trust by working with and for the community. Also, the trust for the long-term endurance of the development project was found as a precondition for the community commitment. The discoveries of this thesis can be applied to similar types of collaborative AI developments in the future. There are significant contributions in this dissertation to robotics. First, the long-term robot usage was not studied on a years-long scale before and the most extended human-robot interactions analyzed test subjects for only a few months. A questionnaire investigated the robot owners with 1-10+ years-long ownership in this work and their attitude towards robot acceptance. The survey results helped to understand the viable strategies to engage users for a long time. Second, innovative ways were explored to involve online communities in robotics development. The past approaches introduced the community ideas and opinions into product design and innovation iterations. The community in this dissertation tested the developed AI engine, provided inputs for further development directions, created content for the actual AI and gave their feedback about product quality. These contributions advance the social robotics field