An intuitive visual interface for a real-time monitoring system for human gait using IMUs

A Matlab Graphical User Interface (GUI) to help monitor the human gait in real time, and to enable to better understand the information of the used Inertial Measurement Units (IMUs) system is here presented. The interface not only improved the speed of data processing, but also facilitated the interaction between the user and the system, providing a clearer change of variables and information exchange. In order to ensure a clearer and more reliable data processing an algorithmic state machine (ASM) was implemented. Data from sensors can be used to estimate the orientation of each module through the use of a complementary filter (representation achieved in Quaternions and/or Euler angles), based on previous work. Furthermore, in this article it will be presented the list of requirements built initially for the interface creation as well as the Matlab GUI, its functionalities, and its assessments. Results show quality assessment of the developed interface based on an inquiry made in a group with thirty-six participants, and the assessment of the communication protocol.This work is supported by the FCT - Fundação para a Ciência e Tecnologia - with the scholarship reference SFRH/BD/108309/2015, with the reference project UID/EEA/04436/2013, by FEDER funds through the COMPETE 2020 - Programa Operacional Competitividade e Internacionalizacao (POCI) - with the reference project POCI-01-0145-FEDER-006941

IMUs: validation, gait analysis and system’s implementation

Dissertação de mestrado integrado em Engenharia Biomédica (área de especialização em Eletrónica Médica)Falls are a prevalent problem in actual society. The number of falls has been increasing greatly in the last fifteen years. Some falls result in injuries and the cost associated with their treatment is high. However, this is a complex problem that requires several steps in order to be tackled. Namely, it is crucial to develop strategies that recognize the mode of locomotion, indicating the state of the subject in various situations, namely normal gait, step before fall (pre-fall) and fall situation. Thus, this thesis aims to develop a strategy capable of identifying these situations based on a wearable system that collects information and analyses the human gait. The strategy consists, essentially, in the construction and use of Associative Skill Memories (ASMs) as tools for recognizing the locomotion modes. Consequently, at an early stage, the capabilities of the ASMs for the different modes of locomotion were studied. Then, a classifier was developed based on a set of ASMs. Posteriorly, a neural network classifier based on deep learning was used to classify, in a similar way, the same modes of locomotion. Deep learning is a technique actually widely used in data classification. These classifiers were implemented and compared, providing for a tool with a good accuracy in recognizing the modes of locomotion. In order to implement this strategy, it was previously necessary to carry out extremely important support work. An inertial measurement units’ (IMUs) system was chosen due to its extreme potential to monitor outpatient activities in the home environment. This system, which combines inertial and magnetic sensors and is able to perform the monitoring of gait parameters in real time, was validated and calibrated. Posteriorly, this system was used to collect data from healthy subjects that mimicked Fs. Results have shown that the accuracy of the classifiers was quite acceptable, and the neural networks based classifier presented the best results with 92.71% of accuracy. As future work, it is proposed to apply these strategies in real time in order to avoid the occurrence of falls.As quedas são um problema predominante na sociedade atual. O número de quedas tem aumentado bastante nos últimos quinze anos. Algumas quedas resultam em lesões e o custo associado ao seu tratamento é alto. No entanto, trata-se de um problema complexo que requer várias etapas a serem abordadas. Ou seja, é crucial desenvolver estratégias que reconheçam o modo de locomoção, indicando o estado do sujeito em várias situações, nomeadamente, marcha normal, passo antes da queda (pré-queda) e situação de queda. Assim, esta tese tem como objetivo desenvolver uma estratégia capaz de identificar essas situações com base num sistema wearable que colete informações e analise a marcha humana. A estratégia consiste, essencialmente, na construção e utilização de Associative Skill Memories (ASMs) como ferramenta para reconhecimento dos modos de locomoção. Consequentemente, numa fase inicial, foram estudadas as capacidades das ASMs para os diferentes modos de locomoção. Depois, foi desenvolvido um classificador baseado em ASMs. Posteriormente, um classificador de redes neuronais baseado em deep learning foi utilizado para classificar, de forma semelhante, os mesmos modos de locomoção. Deep learning é uma técnica bastante utilizada em classificação de dados. Estes classificadores foram implementados e comparados, fornecendo a uma ferramenta com uma boa precisão no reconhecimento dos modos de locomoção. Para implementar esta estratégia, era necessário realizar previamente um trabalho de suporte extremamente importante. Um sistema de unidades de medição inercial (IMUs), foi escolhido devido ao seu potencial extremo para monitorizar as atividades ambulatórias no ambiente domiciliar. Este sistema que combina sensores inerciais e magnéticos e é capaz de efetuar a monitorização de parâmetros da marcha em tempo real, foi validado e calibrado. Posteriormente, este Sistema foi usado para adquirir dados da marcha de indivíduos saudáveis que imitiram quedas. Os resultados mostraram que a precisão dos classificadores foi bastante aceitável e o classificador baseado em redes neuronais apresentou os melhores resultados com 92.71% de precisão. Como trabalho futuro, propõe-se a aplicação destas estratégias em tempo real de forma a evitar a ocorrência de quedas

An inertial motion capture framework for constructing body sensor networks

Motion capture is the process of measuring and subsequently reconstructing the movement of an animated object or being in virtual space. Virtual reconstructions of human motion play an important role in numerous application areas such as animation, medical science, ergonomics, etc. While optical motion capture systems are the industry standard, inertial body sensor networks are becoming viable alternatives due to portability, practicality and cost. This thesis presents an innovative inertial motion capture framework for constructing body sensor networks through software environments, smartphones and web technologies. The first component of the framework is a unique inertial motion capture software environment aimed at providing an improved experimentation environment, accompanied by programming scaffolding and a driver development kit, for users interested in studying or engineering body sensor networks. The software environment provides a bespoke 3D engine for kinematic motion visualisations and a set of tools for hardware integration. The software environment is used to develop the hardware behind a prototype motion capture suit focused on low-power consumption and hardware-centricity. Additional inertial measurement units, which are available commercially, are also integrated to demonstrate the functionality the software environment while providing the framework with additional sources for motion data. The smartphone is the most ubiquitous computing technology and its worldwide uptake has prompted many advances in wearable inertial sensing technologies. Smartphones contain gyroscopes, accelerometers and magnetometers, a combination of sensors that is commonly found in inertial measurement units. This thesis presents a mobile application that investigates whether the smartphone is capable of inertial motion capture by constructing a novel omnidirectional body sensor network. This thesis proposes a novel use for web technologies through the development of the Motion Cloud, a repository and gateway for inertial data. Web technologies have the potential to replace motion capture file formats with online repositories and to set a new standard for how motion data is stored. From a single inertial measurement unit to a more complex body sensor network, the proposed architecture is extendable and facilitates the integration of any inertial hardware configuration. The Motion Cloud’s data can be accessed through an application-programming interface or through a web portal that provides users with the functionality for visualising and exporting the motion data

Master of Science

thesisComputing and data acquisition have become an integral part of everyday life. From reading emails on cell phones to kids playing with motion sensing game consoles, we are surrounded with sensors and mobile computing devices. As the availability of powerful computing devices increases, applications in previously limited environments become possible. Training devices in rehabilitation are becoming increasingly common and more mobile. Community based rehabilitative devices are emerging that embrace these mobile advances. To further the flexibility of devices used in rehabilitation, research has explored the use of smartphones as a means to process data and provide feedback to the user. In combination with sensor embedded insoles, smartphones provide a powerful tool for the clinician in gathering data and as a standalone training tool in rehabilitation. This thesis presents the continuing research of sensor based insoles, feedback systems and increasing the capabilities of the Adaptive Real-Time Instrumentation System for Tread Imbalance Correction, or ARTISTIC, with the introduction of ARTISTIC 2.0. To increase the capabilities of the ARTISTIC an Inertial Measurement Unit (IMU) was added, which gave the system the ability to quantify the motion of the gait cycle and, more specifically, determine stride length. The number of sensors in the insole was increased from two to ten, as well as placing the microprocessor and a vibratory motor in the insole. The transmission box weight was reduced by over 50 percent and the volume by over 60 percent. Stride length was validated against a motion capture system and found the average stride length to be within 2.7 ± 6.9 percent. To continue the improvement of the ARTISTIC 2.0, future work will include implementing real-time stride length feedback

Fused mechanomyography and inertial measurement for human-robot interface

Human-Machine Interfaces (HMI) are the technology through which we interact with the ever-increasing quantity of smart devices surrounding us. The fundamental goal of an HMI is to facilitate robot control through uniting a human operator as the supervisor with a machine as the task executor. Sensors, actuators, and onboard intelligence have not reached the point where robotic manipulators may function with complete autonomy and therefore some form of HMI is still necessary in unstructured environments. These may include environments where direct human action is undesirable or infeasible, and situations where a robot must assist and/or interface with people. Contemporary literature has introduced concepts such as body-worn mechanical devices, instrumented gloves, inertial or electromagnetic motion tracking sensors on the arms, head, or legs, electroencephalographic (EEG) brain activity sensors, electromyographic (EMG) muscular activity sensors and camera-based (vision) interfaces to recognize hand gestures and/or track arm motions for assessment of operator intent and generation of robotic control signals. While these developments offer a wealth of future potential their utility has been largely restricted to laboratory demonstrations in controlled environments due to issues such as lack of portability and robustness and an inability to extract operator intent for both arm and hand motion. Wearable physiological sensors hold particular promise for capture of human intent/command. EMG-based gesture recognition systems in particular have received significant attention in recent literature. As wearable pervasive devices, they offer benefits over camera or physical input systems in that they neither inhibit the user physically nor constrain the user to a location where the sensors are deployed. Despite these benefits, EMG alone has yet to demonstrate the capacity to recognize both gross movement (e.g. arm motion) and finer grasping (e.g. hand movement). As such, many researchers have proposed fusing muscle activity (EMG) and motion tracking e.g. (inertial measurement) to combine arm motion and grasp intent as HMI input for manipulator control. However, such work has arguably reached a plateau since EMG suffers from interference from environmental factors which cause signal degradation over time, demands an electrical connection with the skin, and has not demonstrated the capacity to function out of controlled environments for long periods of time. This thesis proposes a new form of gesture-based interface utilising a novel combination of inertial measurement units (IMUs) and mechanomyography sensors (MMGs). The modular system permits numerous configurations of IMU to derive body kinematics in real-time and uses this to convert arm movements into control signals. Additionally, bands containing six mechanomyography sensors were used to observe muscular contractions in the forearm which are generated using specific hand motions. This combination of continuous and discrete control signals allows a large variety of smart devices to be controlled. Several methods of pattern recognition were implemented to provide accurate decoding of the mechanomyographic information, including Linear Discriminant Analysis and Support Vector Machines. Based on these techniques, accuracies of 94.5% and 94.6% respectively were achieved for 12 gesture classification. In real-time tests, accuracies of 95.6% were achieved in 5 gesture classification. It has previously been noted that MMG sensors are susceptible to motion induced interference. The thesis also established that arm pose also changes the measured signal. This thesis introduces a new method of fusing of IMU and MMG to provide a classification that is robust to both of these sources of interference. Additionally, an improvement in orientation estimation, and a new orientation estimation algorithm are proposed. These improvements to the robustness of the system provide the first solution that is able to reliably track both motion and muscle activity for extended periods of time for HMI outside a clinical environment. Application in robot teleoperation in both real-world and virtual environments were explored. With multiple degrees of freedom, robot teleoperation provides an ideal test platform for HMI devices, since it requires a combination of continuous and discrete control signals. The field of prosthetics also represents a unique challenge for HMI applications. In an ideal situation, the sensor suite should be capable of detecting the muscular activity in the residual limb which is naturally indicative of intent to perform a specific hand pose and trigger this post in the prosthetic device. Dynamic environmental conditions within a socket such as skin impedance have delayed the translation of gesture control systems into prosthetic devices, however mechanomyography sensors are unaffected by such issues. There is huge potential for a system like this to be utilised as a controller as ubiquitous computing systems become more prevalent, and as the desire for a simple, universal interface increases. Such systems have the potential to impact significantly on the quality of life of prosthetic users and others.Open Acces

Kvantitativna i kvalitativna procena obrasca hoda kod bolesnika sa Parkinsonovom bolešću

Background/Aim. Postural impairments and gait disorders in Parkinson's disease (PD) affect limits of stability, impaire postural adjustment, and evoke poor responses to perturbation. In the later stage of the disease, some patients can suffer from episodic features such as freezing of gait (FOG). Objective gait assessment and monitoring progress of the disease can give clinicians and therapist important information about changes in gait pattern and potential gait deviations, in order to prevent concomitant falls. The aim of this study was to propose a method for identification of freezing episodes and gait disturbances in patients with PD. A wireless inertial sensor system can be used to provide follow-up of the treatment effects or progress of the disease. Methods. The system is simple for mounting a subject, comfortable, simple for installing and recording, reliable and provides high-quality sensor data. A total of 12 patients were recorded and tested. Software calculates various gait parameters that could be estimated. User friendly visual tool provides information about changes in gait characteristics, either in a form of spectrogram or by observing spatiotemporal parameters. Based on these parameters, the algorithm performs classification of strides and identification of FOG types. Results. The described stride classification was merged with an algorithm for stride reconstruction resulting in a useful graphical tool that allows clinicians to inspect and analyze subject's movements. Conclusion. The described gait assessment system can be used for detection and categorization of gait disturbances by applying rule-based classification based on stride length, stride time, and frequency of the shank segment movements. The method provides an valuable graphical interface which is easy to interpret and provides clinicians and therapists with valuable information regarding the temporal changes in gait.Uvod/Cilj. Poremećaji hoda i ravnoteže kod bolesnika sa Parkinsonovom bolešću (PD) uključuju i poremećaje stabilnosti, održavanja ravnoteže prilikom hoda i nemogućnost adekvatne reakcije na iznenadne perturbacije. U kasnijim fazama bolesti neki bolesnici razvijaju i epizode motornog bloka, odnosno 'frizing' tokom hoda. Objektivno praćenje i merenje karakteristika hoda i promena obrasca hoda tokom progresije bolesti mogu pomoći kliničarima jer ukazuju na promene koje bi dovele do padova i ugrozile bolesnika. Cilj rada bio je razvoj metode koja bi identifikovala ovakve epizode kod bolesnika sa Parkinsonovom bolesti. Razvijeni bežični sistem sa senzorima mogao bi se koristiti za posmatranje efekata terapije ili progresije bolesti. Metode. U radu je prikazan sistem za objektivnu procenu obrasca hoda. Korišćenjem bežičnog senzorskog sistema koji koristi akcelerometre, žiroskope i senzore sile, moguće je dobiti procenu parametara hoda, ali i identifikovati 'frizing' epizode karakteristične za PD. Uz pomoć ovog sistema snimljeno je 12 bolesnika, te je na osnovu snimljenih signala razvijen novi softverski alat koji omogućava praćenje parametara hoda. Rezultati. Na osnovu dužine koraka, trajanja koraka i frekvencije pokreta, razvijen je algoritam za klasifikaciju tipova koraka i uočavanje promena frekvencija pokreta tokom hoda. Prikaz rezultata ovog sistema je dat kroz primer jednog bolesnika. Zaključak. Opisani sistem za procenu hoda može biti korišćen za kategorizaciju poremećaja hoda kroz posmatranje promena u dužini i trajanju koraka, kao i frekvencija segmenata noge. Razvijeni metod omogućava iliustrativni prikaz i grafički interfejs koji je jednostavan za interpretaciju i omogućava dobijanje informacija koje kliničarima mogu ukazati na trenutne promene u obrascu hoda

Smartphone Based Personalized Balance Training Platform

ME450 Capstone Design and Manufacturing Experience: Winter 2021Older adults are at high risk of falls, mainly due to the loss of balance control. It is important for them to regain balance control through balance training exercises for quality living. These exercises are conventionally done in a clinic-based setting under the supervision of a physical therapist (PT). However, this method comes with limitations such as cost, insurance reimbursement policies, and travel. Thus, there is a need for a portable balance training platform that can be used by older adults at home. Our team is developing a platform as such that can not only provide balance training to our users but can also measure kinematic data from multiple body parts and capture self-performance ratings after exercises are performed - these data are uploaded to a secure cloud account. The platform can also support a machine learning framework that generates a list of recommended exercises and simulated PT ratings for the users based on their performance during the balance training exercise sessions.Jamie Ferris, Safa Jabri, Christopher DiCesare, Xun Huan: Sienko Research Grouphttp://deepblue.lib.umich.edu/bitstream/2027.42/167652/1/Team_8-Smartphone_Based_Personalized_Balance_Training_Platform.pd

Human Activity Recognition and Control of Wearable Robots

abstract: Wearable robotics has gained huge popularity in recent years due to its wide applications in rehabilitation, military, and industrial fields. The weakness of the skeletal muscles in the aging population and neurological injuries such as stroke and spinal cord injuries seriously limit the abilities of these individuals to perform daily activities. Therefore, there is an increasing attention in the development of wearable robots to assist the elderly and patients with disabilities for motion assistance and rehabilitation. In military and industrial sectors, wearable robots can increase the productivity of workers and soldiers. It is important for the wearable robots to maintain smooth interaction with the user while evolving in complex environments with minimum effort from the user. Therefore, the recognition of the user's activities such as walking or jogging in real time becomes essential to provide appropriate assistance based on the activity. This dissertation proposes two real-time human activity recognition algorithms intelligent fuzzy inference (IFI) algorithm and Amplitude omega ($A \omega$) algorithm to identify the human activities, i.e., stationary and locomotion activities. The IFI algorithm uses knee angle and ground contact forces (GCFs) measurements from four inertial measurement units (IMUs) and a pair of smart shoes. Whereas, the $A \omega$ algorithm is based on thigh angle measurements from a single IMU. This dissertation also attempts to address the problem of online tuning of virtual impedance for an assistive robot based on real-time gait and activity measurement data to personalize the assistance for different users. An automatic impedance tuning (AIT) approach is presented for a knee assistive device (KAD) in which the IFI algorithm is used for real-time activity measurements. This dissertation also proposes an adaptive oscillator method known as amplitude omega adaptive oscillator ($A\omega AO$) method for HeSA (hip exoskeleton for superior augmentation) to provide bilateral hip assistance during human locomotion activities. The $A \omega$ algorithm is integrated into the adaptive oscillator method to make the approach robust for different locomotion activities. Experiments are performed on healthy subjects to validate the efficacy of the human activities recognition algorithms and control strategies proposed in this dissertation. Both the activity recognition algorithms exhibited higher classification accuracy with less update time. The results of AIT demonstrated that the KAD assistive torque was smoother and EMG signal of Vastus Medialis is reduced, compared to constant impedance and finite state machine approaches. The $A\omega AO$ method showed real-time learning of the locomotion activities signals for three healthy subjects while wearing HeSA. To understand the influence of the assistive devices on the inherent dynamic gait stability of the human, stability analysis is performed. For this, the stability metrics derived from dynamical systems theory are used to evaluate unilateral knee assistance applied to the healthy participants.Dissertation/ThesisDoctoral Dissertation Aerospace Engineering 201

Low-Cost Sensors and Biological Signals

Many sensors are currently available at prices lower than USD 100 and cover a wide range of biological signals: motion, muscle activity, heart rate, etc. Such low-cost sensors have metrological features allowing them to be used in everyday life and clinical applications, where gold-standard material is both too expensive and time-consuming to be used. The selected papers present current applications of low-cost sensors in domains such as physiotherapy, rehabilitation, and affective technologies. The results cover various aspects of low-cost sensor technology from hardware design to software optimization