Human-activity-centered measurement system:challenges from laboratory to the real environment in assistive gait wearable robotics

Assistive gait wearable robots (AGWR) have shown a great advancement in developing intelligent devices to assist human in their activities of daily living (ADLs). The rapid technological advancement in sensory technology, actuators, materials and computational intelligence has sped up this development process towards more practical and smart AGWR. However, most assistive gait wearable robots are still confined to be controlled, assessed indoor and within laboratory environments, limiting any potential to provide a real assistance and rehabilitation required to humans in the real environments. The gait assessment parameters play an important role not only in evaluating the patient progress and assistive device performance but also in controlling smart self-adaptable AGWR in real-time. The self-adaptable wearable robots must interactively conform to the changing environments and between users to provide optimal functionality and comfort. This paper discusses the performance parameters, such as comfortability, safety, adaptability, and energy consumption, which are required for the development of an intelligent AGWR for outdoor environments. The challenges to measuring the parameters using current systems for data collection and analysis using vision capture and wearable sensors are presented and discussed

Human Motion Analysis with Wearable Inertial Sensors

High-resolution, quantitative data obtained by a human motion capture system can be used to better understand the cause of many diseases for effective treatments. Talking about the daily care of the aging population, two issues are critical. One is to continuously track motions and position of aging people when they are at home, inside a building or in the unknown environment; the other is to monitor their health status in real time when they are in the free-living environment. Continuous monitoring of human movement in their natural living environment potentially provide more valuable feedback than these in laboratory settings. However, it has been extremely challenging to go beyond laboratory and obtain accurate measurements of human physical activity in free-living environments. Commercial motion capture systems produce excellent in-studio capture and reconstructions, but offer no comparable solution for acquisition in everyday environments. Therefore in this dissertation, a wearable human motion analysis system is developed for continuously tracking human motions, monitoring health status, positioning human location and recording the itinerary. In this dissertation, two systems are developed for seeking aforementioned two goals: tracking human body motions and positioning a human. Firstly, an inertial-based human body motion tracking system with our developed inertial measurement unit (IMU) is introduced. By arbitrarily attaching a wearable IMU to each segment, segment motions can be measured and translated into inertial data by IMUs. A human model can be reconstructed in real time based on the inertial data by applying high efficient twists and exponential maps techniques. Secondly, for validating the feasibility of developed tracking system in the practical application, model-based quantification approaches for resting tremor and lower extremity bradykinesia in Parkinson’s disease are proposed. By estimating all involved joint angles in PD symptoms based on reconstructed human model, angle characteristics with corresponding medical ratings are employed for training a HMM classifier for quantification. Besides, a pedestrian positioning system is developed for tracking user’s itinerary and positioning in the global frame. Corresponding tests have been carried out to assess the performance of each system

Inertial measurement units: a brief state of the art on gait analysis

Gait analysis systems are monitoring systems that establish a symbiosis relationship with Ambient Assisted Living (AAL) environments. Human locomotion analysis has a very important role always aiming at improving the quality of life both for individuals needing treatment or rehabilitation, as well as for healthy and elderly people. In fact, a deep and detailed knowledge about gait characteristics at a given time, and not least, monitoring and evaluating over time, will allow early diagnosis of diseases and their complications, and contribute to the decision of the treatment that should be chosen. There are several techniques used for gait measuring such as: Image Processing, Floor Sensors, and Wearable Sensors. Among the wearable sensors, has emerged an electronic device that combines multiple sensors designated by Inertial Measurement Unit (IMU). This device measures angular rate, body's specific force, and in some cases the magnetic field, and this information may be used to monitor human gait. In this article, the aim is: i) to verify the sensors that build up the IMUs, and the resulting designations that the device may have depending on the sensors it contains; ii) to list the applications of the IMUs on gait analysis; iii) to be aware of the devices available on the market and the associated commercial brands; and iv) to list the advantages and disadvantages associated with the device compared to other gait analysis systems. Concerning the literature in the scientific community, although there are some studies that focus on gait analysis or IMUs, none of them aggregates the purposes that will be addressed in this article.This work is supported by the FCT - Fundação para a Ciência e Tecnologia - with the scholarship reference SFRH/BD/108309/2015, with the reference project UID/EEA/04436/2013, by FEDER funds through the COMPETE 2020 - Programa Operacional Competitividade e Internacionalização (POCI) - with the reference project POCI-01-0145- FEDER-006941

Wearable inertial sensor system towards daily human kinematic gait analysis: benchmarking analysis to MVN BIOMECH

This paper presents a cost- and time-effective wearable inertial sensor system, the InertialLAB. It includes gyroscopes and accelerometers for the real-time monitoring of 3D-angular velocity and 3D-acceleration of up to six lower limbs and trunk segment and sagittal joint angle up to six joints. InertialLAB followed an open architecture with a low computational load to be executed by wearable processing units up to 200 Hz for fostering kinematic gait data to third-party systems, advancing similar commercial systems. For joint angle estimation, we developed a trigonometric method based on the segments’ orientation previously computed by fusion-based methods. The validation covered healthy gait patterns in varying speed and terrain (flat, ramp, and stairs) and including turns, extending the experiments approached in the literature. The benchmarking analysis to MVN BIOMECH reported that InertialLAB provides more reliable measures in stairs than in flat terrain and ramp. The joint angle time-series of InertialLAB showed good waveform similarity (>0.898) with MVN BIOMECH, resulting in high reliability and excellent validity. User-independent neural network regression models successfully minimized the drift errors observed in InertialLAB’s joint angles (NRMSE < 0.092). Further, users ranked InertialLAB as good in terms of usability. InertialLAB shows promise for daily kinematic gait analysis and real-time kinematic feedback for wearable third-party systems.This work has been supported in part by the Fundação para a Ciência e Tecnologia (FCT) with the Reference Scholarship under Grant SFRH/BD/108309/2015 and SFRH/BD/147878/2019, by the FEDER Funds through the Programa Operacional Regional do Norte and national funds from FCT with the project SmartOs under Grant NORTE-01-0145-FEDER-030386, and through the COMPETE 2020—Programa Operacional Competitividade e Internacionalização (POCI)—with the Reference Project under Grant POCI-01-0145-FEDER-006941

Custom IMU-Based Wearable System for Robust 2.4 GHz Wireless Human Body Parts Orientation Tracking and 3D Movement Visualization on an Avatar

Recent studies confirm the applicability of Inertial Measurement Unit (IMU)-based systems for human motion analysis. Notwithstanding, high-end IMU-based commercial solutions are yet too expensive and complex to democratize their use among a wide range of potential users. Less featured entry-level commercial solutions are being introduced in the market, trying to fill this gap, but still present some limitations that need to be overcome. At the same time, there is a growing number of scientific papers using not commercial, but custom do-it-yourself IMU-based systems in medical and sports applications. Even though these solutions can help to popularize the use of this technology, they have more limited features and the description on how to design and build them from scratch is yet too scarce in the literature. The aim of this work is two-fold: (1) Proving the feasibility of building an affordable custom solution aimed at simultaneous multiple body parts orientation tracking; while providing a detailed bottom-up description of the required hardware, tools, and mathematical operations to estimate and represent 3D movement in real-time. (2) Showing how the introduction of a custom 2.4 GHz communication protocol including a channel hopping strategy can address some of the current communication limitations of entry-level commercial solutions. The proposed system can be used for wireless real-time human body parts orientation tracking with up to 10 custom sensors, at least at 50 Hz. In addition, it provides a more reliable motion data acquisition in Bluetooth and Wi-Fi crowded environments, where the use of entry-level commercial solutions might be unfeasible. This system can be used as a groundwork for developing affordable human motion analysis solutions that do not require an accurate kinematic analysis.This research has been partially funded by a research contract with IVECO Spain SL and by the Department of Employment and Industry of Castilla y León (Spain), under research project ErgoTwyn (INVESTUN/21/VA/0003)

Reliable and robust detection of freezing of gait episodes with wearable electronic devices

A wearable wireless sensing system for assisting patients affected by Parkinson's disease is proposed. It uses integrated micro-electro-mechanical inertial sensors able to recognize the episodes of involuntary gait freezing. The system operates in real time and is designed for outdoor and indoor applications. Standard tests were performed on a noticeable number of patients and healthy persons and the algorithm demonstrated its reliability and robustness respect to individual specific gait and postural behaviors. The overall performances of the system are excellent with a specificity higher than 97%

Centre of pressure estimation during walking using only inertial-measurement units and end-to-end statistical modelling

Estimation of the centre of pressure (COP) is an important part of the gait analysis, for example, when evaluating the functional capacity of individuals affected by motor impairment. Inertial measurement units (IMUs) and force sensors are commonly used to measure gait characteristic of healthy and impaired subjects. We present a methodology for estimating the COP solely from raw gyroscope, accelerometer, and magnetometer data from IMUs using statistical modelling. We demonstrate the viability of the method using an example of two models: a linear model and a non-linear Long-Short-Term Memory (LSTM) neural network model. Models were trained on the COP ground truth data measured using an instrumented treadmill and achieved the average intra-subject root mean square (RMS) error between estimated and ground truth COP of 12.3mm and the average inter-subject RMS error of 23.7mm which is comparable or better than similar studies so far. We show that the calibration procedure in the instrumented treadmill can be as short as a couple of minutes without the decrease in our model performance. We also show that the magnetic component of the recorded IMU signal, which is most sensitive to environmental changes, can be safely dropped without a significant decrease in model performance. Finally, we show that the number of IMUs can be reduced to five without deterioration in the model performance.Comment: 21 page

Fusion of non-visual and visual sensors for human tracking

Human tracking is an extensively researched yet still challenging area in the Computer Vision field, with a wide range of applications such as surveillance and healthcare. People may not be successfully tracked with merely the visual information in challenging cases such as long-term occlusion. Thus, we propose to combine information from other sensors with the surveillance cameras to persistently localize and track humans, which is becoming more promising with the pervasiveness of mobile devices such as cellphones, smart watches and smart glasses embedded with all kinds of sensors including accelerometers, gyroscopes, magnetometers, GPS, WiFi modules and so on. In this thesis, we firstly investigate the application of Inertial Measurement Unit (IMU) from mobile devices to human activity recognition and human tracking, we then develop novel persistent human tracking and indoor localization algorithms by the fusion of non-visual sensors and visual sensors, which not only overcomes the occlusion challenge in visual tracking, but also alleviates the calibration and drift problems in IMU tracking --Abstract, page iii

IMUs: validation, gait analysis and system’s implementation

Dissertação de mestrado integrado em Engenharia Biomédica (área de especialização em Eletrónica Médica)Falls are a prevalent problem in actual society. The number of falls has been increasing greatly in the last fifteen years. Some falls result in injuries and the cost associated with their treatment is high. However, this is a complex problem that requires several steps in order to be tackled. Namely, it is crucial to develop strategies that recognize the mode of locomotion, indicating the state of the subject in various situations, namely normal gait, step before fall (pre-fall) and fall situation. Thus, this thesis aims to develop a strategy capable of identifying these situations based on a wearable system that collects information and analyses the human gait. The strategy consists, essentially, in the construction and use of Associative Skill Memories (ASMs) as tools for recognizing the locomotion modes. Consequently, at an early stage, the capabilities of the ASMs for the different modes of locomotion were studied. Then, a classifier was developed based on a set of ASMs. Posteriorly, a neural network classifier based on deep learning was used to classify, in a similar way, the same modes of locomotion. Deep learning is a technique actually widely used in data classification. These classifiers were implemented and compared, providing for a tool with a good accuracy in recognizing the modes of locomotion. In order to implement this strategy, it was previously necessary to carry out extremely important support work. An inertial measurement units’ (IMUs) system was chosen due to its extreme potential to monitor outpatient activities in the home environment. This system, which combines inertial and magnetic sensors and is able to perform the monitoring of gait parameters in real time, was validated and calibrated. Posteriorly, this system was used to collect data from healthy subjects that mimicked Fs. Results have shown that the accuracy of the classifiers was quite acceptable, and the neural networks based classifier presented the best results with 92.71% of accuracy. As future work, it is proposed to apply these strategies in real time in order to avoid the occurrence of falls.As quedas são um problema predominante na sociedade atual. O número de quedas tem aumentado bastante nos últimos quinze anos. Algumas quedas resultam em lesões e o custo associado ao seu tratamento é alto. No entanto, trata-se de um problema complexo que requer várias etapas a serem abordadas. Ou seja, é crucial desenvolver estratégias que reconheçam o modo de locomoção, indicando o estado do sujeito em várias situações, nomeadamente, marcha normal, passo antes da queda (pré-queda) e situação de queda. Assim, esta tese tem como objetivo desenvolver uma estratégia capaz de identificar essas situações com base num sistema wearable que colete informações e analise a marcha humana. A estratégia consiste, essencialmente, na construção e utilização de Associative Skill Memories (ASMs) como ferramenta para reconhecimento dos modos de locomoção. Consequentemente, numa fase inicial, foram estudadas as capacidades das ASMs para os diferentes modos de locomoção. Depois, foi desenvolvido um classificador baseado em ASMs. Posteriormente, um classificador de redes neuronais baseado em deep learning foi utilizado para classificar, de forma semelhante, os mesmos modos de locomoção. Deep learning é uma técnica bastante utilizada em classificação de dados. Estes classificadores foram implementados e comparados, fornecendo a uma ferramenta com uma boa precisão no reconhecimento dos modos de locomoção. Para implementar esta estratégia, era necessário realizar previamente um trabalho de suporte extremamente importante. Um sistema de unidades de medição inercial (IMUs), foi escolhido devido ao seu potencial extremo para monitorizar as atividades ambulatórias no ambiente domiciliar. Este sistema que combina sensores inerciais e magnéticos e é capaz de efetuar a monitorização de parâmetros da marcha em tempo real, foi validado e calibrado. Posteriormente, este Sistema foi usado para adquirir dados da marcha de indivíduos saudáveis que imitiram quedas. Os resultados mostraram que a precisão dos classificadores foi bastante aceitável e o classificador baseado em redes neuronais apresentou os melhores resultados com 92.71% de precisão. Como trabalho futuro, propõe-se a aplicação destas estratégias em tempo real de forma a evitar a ocorrência de quedas

Survey of Motion Tracking Methods Based on Inertial Sensors: A Focus on Upper Limb Human Motion

Motion tracking based on commercial inertial measurements units (IMUs) has been widely studied in the latter years as it is a cost-effective enabling technology for those applications in which motion tracking based on optical technologies is unsuitable. This measurement method has a high impact in human performance assessment and human-robot interaction. IMU motion tracking systems are indeed self-contained and wearable, allowing for long-lasting tracking of the user motion in situated environments. After a survey on IMU-based human tracking, five techniques for motion reconstruction were selected and compared to reconstruct a human arm motion. IMU based estimation was matched against motion tracking based on the Vicon marker-based motion tracking system considered as ground truth. Results show that all but one of the selected models perform similarly (about 35 mm average position estimation error)