Robust localization with wearable sensors

Measuring physical movements of humans and understanding human behaviour is useful in a variety of areas and disciplines. Human inertial tracking is a method that can be leveraged for monitoring complex actions that emerge from interactions between human actors and their environment. An accurate estimation of motion trajectories can support new approaches to pedestrian navigation, emergency rescue, athlete management, and medicine. However, tracking with wearable inertial sensors has several problems that need to be overcome, such as the low accuracy of consumer-grade inertial measurement units (IMUs), the error accumulation problem in long-term tracking, and the artefacts generated by movements that are less common. This thesis focusses on measuring human movements with wearable head-mounted sensors to accurately estimate the physical location of a person over time. The research consisted of (i) providing an overview of the current state of research for inertial tracking with wearable sensors, (ii) investigating the performance of new tracking algorithms that combine sensor fusion and data-driven machine learning, (iii) eliminating the effect of random head motion during tracking, (iv) creating robust long-term tracking systems with a Bayesian neural network and sequential Monte Carlo method, and (v) verifying that the system can be applied with changing modes of behaviour, defined as natural transitions from walking to running and vice versa. This research introduces a new system for inertial tracking with head-mounted sensors (which can be placed in, e.g. helmets, caps, or glasses). This technology can be used for long-term positional tracking to explore complex behaviours

Human Motion Analysis with Wearable Inertial Sensors

High-resolution, quantitative data obtained by a human motion capture system can be used to better understand the cause of many diseases for effective treatments. Talking about the daily care of the aging population, two issues are critical. One is to continuously track motions and position of aging people when they are at home, inside a building or in the unknown environment; the other is to monitor their health status in real time when they are in the free-living environment. Continuous monitoring of human movement in their natural living environment potentially provide more valuable feedback than these in laboratory settings. However, it has been extremely challenging to go beyond laboratory and obtain accurate measurements of human physical activity in free-living environments. Commercial motion capture systems produce excellent in-studio capture and reconstructions, but offer no comparable solution for acquisition in everyday environments. Therefore in this dissertation, a wearable human motion analysis system is developed for continuously tracking human motions, monitoring health status, positioning human location and recording the itinerary. In this dissertation, two systems are developed for seeking aforementioned two goals: tracking human body motions and positioning a human. Firstly, an inertial-based human body motion tracking system with our developed inertial measurement unit (IMU) is introduced. By arbitrarily attaching a wearable IMU to each segment, segment motions can be measured and translated into inertial data by IMUs. A human model can be reconstructed in real time based on the inertial data by applying high efficient twists and exponential maps techniques. Secondly, for validating the feasibility of developed tracking system in the practical application, model-based quantification approaches for resting tremor and lower extremity bradykinesia in Parkinson’s disease are proposed. By estimating all involved joint angles in PD symptoms based on reconstructed human model, angle characteristics with corresponding medical ratings are employed for training a HMM classifier for quantification. Besides, a pedestrian positioning system is developed for tracking user’s itinerary and positioning in the global frame. Corresponding tests have been carried out to assess the performance of each system

Motion-based remote control device for interaction with multimedia content

This dissertation describes the development and implementation of techniques to enhance the accuracy of low-complexity lters, making them suitable for remote control devices in consumer electronics. The evolution veri ed in the last years, on multimedia contents, available for consumers in Smart TVs and set-top-boxes, is not raising the expected interest from users, and one of the pointed reasons for this nding is the user interface. Although most current pointing devices rely on relative rotation increments, absolute orientation allows for a more intuitive use and interaction. This possibility is explored in this work as well as the interaction with multimedia contents through gestures. Classical accurate fusion algorithms are computationally intensive, therefore their implementation in low-energy consumption devices is a challenging task. To tackle this problem, a performance study was carried, comparing a relevant set of professional commercial of-the-shelf units, with the developed low-complexity lters in state-of-the-art Magnetic, Angular Rate, Gravity (MARG) sensors. Part of the performance evaluation tests are carried out under harsh conditions to observe the algorithms response in a nontrivial environment. The results demonstrate that the implementation of low-complexity lters using low-cost sensors, can provide an acceptable accuracy in comparison with the more complex units/ lters. These results pave the way for faster adoption of absolute orientation-based pointing devices in interactive multimedia applications, which includes hand-held, battery-operated devices

Accuracy of the Orientation Estimate Obtained Using Four Sensor Fusion Filters Applied to Recordings of Magneto-Inertial Sensors Moving at Three Rotation Rates

6Magneto-Inertial technology is a well-established alternative to optical motion capture for human motion analysis applications since it allows prolonged monitoring in free-living conditions. Magneto and Inertial Measurement Units (MIMUs) integrate a triaxial accelerometer, a triaxial gyroscope and a triaxial magnetometer in a single and lightweight device. The orientation of the body to which a MIMU is attached can be obtained by combining its sensor readings within a sensor fusion framework. Despite several sensor fusion implementations have been proposed, no well-established conclusion about the accuracy level achievable with MIMUs has been reached yet. The aim of this preliminary study was to perform a direct comparison among four popular sensor fusion algorithms applied to the recordings of MIMUs rotating at three different rotation rates, with the orientation provided by a stereophotogrammetric system used as a reference. A procedure for suboptimal determination of the parameter filter values was also proposed. The findings highlighted that all filters exhibited reasonable accuracy (rms errors < 6.4°). Moreover, in accordance with previous studies, every algorithm's accuracy worsened as the rotation rate increased. At the highest rotation rate, the algorithm from Sabatini (2011) showed the best performance with errors smaller than 4.1° rms.partially_openopenCaruso M.; Sabatini A.M.; Knaflitz M.; Gazzoni M.; Della Croce U.; Cereatti A.Caruso, M.; Sabatini, A. M.; Knaflitz, M.; Gazzoni, M.; Della Croce, U.; Cereatti, A

Survey of Motion Tracking Methods Based on Inertial Sensors: A Focus on Upper Limb Human Motion

Motion tracking based on commercial inertial measurements units (IMUs) has been widely studied in the latter years as it is a cost-effective enabling technology for those applications in which motion tracking based on optical technologies is unsuitable. This measurement method has a high impact in human performance assessment and human-robot interaction. IMU motion tracking systems are indeed self-contained and wearable, allowing for long-lasting tracking of the user motion in situated environments. After a survey on IMU-based human tracking, five techniques for motion reconstruction were selected and compared to reconstruct a human arm motion. IMU based estimation was matched against motion tracking based on the Vicon marker-based motion tracking system considered as ground truth. Results show that all but one of the selected models perform similarly (about 35 mm average position estimation error)

Body sensor network for in-home personal healthcare

A body sensor network solution for personal healthcare under an indoor environment is developed. The system is capable of logging the physiological signals of human beings, tracking the orientations of human body, and monitoring the environmental attributes, which covers all necessary information for the personal healthcare in an indoor environment. The major three chapters of this dissertation contain three subsystems in this work, each corresponding to one subsystem: BioLogger, PAMS and CosNet. Each chapter covers the background and motivation of the subsystem, the related theory, the hardware/software design, and the evaluation of the prototype’s performance

Inertial Sensors for Human Motion Analysis: A Comprehensive Review

Inertial motion analysis is having a growing interest during the last decades due to its advantages over classical optical systems. The technological solution based on inertial measurement units allows the measurement of movements in daily living environments, such as in everyday life, which is key for a realistic assessment and understanding of movements. This is why research in this field is still developing and different approaches are proposed. This presents a systematic review of the different proposals for inertial motion analysis found in the literature. The search strategy has been carried out on eight different platforms, including journal articles and conference proceedings, which are written in English and published until August 2022. The results are analyzed in terms of the publishers, the sensors used, the applications, the monitored units, the algorithms of use, the participants of the studies, and the validation systems employed. In addition, we delve deeply into the machine learning techniques proposed in recent years and in the approaches to reduce the estimation error. In this way, we show an overview of the research carried out in this field, going into more detail in recent years, and providing some research directions for future wor

Ergowear: desenvolvimento de um vestuário inteligente para monitorização postural e biofeedback

Dissertação de mestrado em Engenharia Biomédica (especialização em Eletrónica Médica)Atualmente, as Lesões Musculoesqueléticas Relacionadas com o Trabalho (LMERT) são considera das o ”problema relacionado com o trabalho mais prevalente”na União Europeia, levando a um custo estimado de cerca de 240 biliões de euros. Em casos mais severos, estes distúrbios podem causar danos vitalícios à saúde do trabalhador, reduzindo a sua qualidade de vida. De facto, LMERTs são con sideradas a principal causa da reforma precoce dos trabalhadores. Foi reportado que os segmentos da parte superior do corpo são mais suceptíveis ao desenvolvimento de LMERTs. Para mitigar a prevalência de LMERTs, ergonomistas maioritariamente aplicam métodos de avaliação observacionais, que são alta mente dependentes da experiência do analista, e apresentam baixa objetividade e repetibilidade. Desta maneira, esforços têm sido feitos para desenvolver ferramentas de avaliação ergonómica baseadas na instrumentação, para compensar essas limitações. Além disso, com a ascensão do conceito da indústria 5.0, o trabalhador humano volta a ser o foco principal na indústria, juntamente com o robô colaborativo. No entanto, para alcançar uma relação verdadeiramente colaborativa e simbiótica entre o trabalhador e o robô, este último precisa de reconhecer as intenções do trabalhador. Para superar este obstáculo, sis temas de captura de movimento podem ser integrados nesta estrutura, fornecendo dados de movimento ao robô colaborativo. Esta dissertação visa a melhoria de um sistema de captura de movimento autónomo, da parte supe rior do corpo, de abordagem inercial que servirá, não apenas para monitorizar a postura do trabalhador, mas também avaliar a ergonomia do usuário e fornecer consciencialização postural ao usuário, por meio de motores de biofeedback. Além disso, o sistema foi já idealizado tendo em mente a sua integração numa estrutura colaborativa humano-robô. Para atingir estes objetivos, foi aplicada uma metodologia de design centrado no utilizador, começando pela análise do Estado da Arte, a avaliação das limitações do sistema anterior, a definição dos requisitos do sistema, o desenvolvimento da peça de vestuário, arquite tura do hardware e arquitetura do software do sistema. Por fim, o sistema foi validado para verificar se estava em conformidade com os requisitos especificados. O sistema é composto por 9 Unidades de Medição Inercial (UMI), posicionados na parte inferior e superior das costas, cabeça, braços, antebraços e mãos. Também foi integrado um sistema de atuação, para biofeedback postural, composto por 6 motores vibrotáteis, localizados na região lombar e próximo do pescoço, cotovelos e pulsos. O sistema é alimentado por uma powerbank e todos os dados adquiridos são enviados para uma estação de processamento, via WiFi (User Datagram Protocol (UDP)), garantindo autonomia. O sistema tem integrado um filtro de fusão Complementar Extendido e uma sequência de calibração Sensor-para-Segmento estática, de maneira a aumentar a precisão da estimativa dos ângulos das articulações. Além disso, o sistema é capaz de amostrar os dados angulares a 240 Hz, enquanto que o sistema anterior era capaz de amostrar no máximo a 100 Hz, melhorando a resolução da aquisição dos dados. O sistema foi validado em termos de hardware e usabilidade. Os testes de hardware abordaram a caracterização da autonomia, frequência de amostragem, robustez mecânica e desempenho da comuni cação sem fio do sistema, em diversos contextos, e também para verificar se estes estão em conformidade com os requisitos técnicos previamente definidos, que foi o caso. Adicionalmente, as especificações da nova versão do sistema foram comparadas com a anterior, onde se observou uma melhoria direta signifi cativa, como por exemplo, maior frequência de amostragem, menor perda de pacote, menor consumo de corrente, entre outras, e com sistemas comerciais de referência (XSens Link). Testes de usabilidade foram realizados com 9 participantes que realizaram vários movimentos uniarticulares e complexos. Após os testes, os usuários responderam a um questionário baseado na Escala de Usabilidade do Sistema (EUS). O sistema foi bem aceite pelos os usuários, em termos de estética e conforto, em geral, comprovando um elevado nível de vestibilidade.Nowadays, Work-Related Musculoskeletal Disorders (WRMSDs) are considered the ”most prevalent work-related problem” in the European Union (EU), leading to an estimated cost of about 240 billion EUR. In more severe cases, these disorders can cause life-long impairments to the workers’ health, reducing their quality of life. In fact, WRMSDs are the main cause for the workers’ early retirement. It was reported that the upper body segments of the worker are more susceptible to the development of WRMSDs. To mitigate the prevalence of WRMSD, ergonomists mostly apply observational assessment methods, which are highly dependant on the analyst’s expertise, have low objectivity and repeatability. Therefore, efforts have been made to develop instrumented-based ergonomic assessment tools, to compensate for these limitations. Moreover, with the rise of the 5.0 industry concept, the human worker is once again the main focus in the industry, along with the Collaborative Robot (cobot). However, to achieve a truly collaborative relation between the worker and the cobot, the latter needs to know the worker’s intentions. To surpass this obstacle, Motion Capture (MoCap) systems can be integrated in this framework, providing motion data to the cobot. This dissertation aims at the improvement of a stand-alone, upper-body, inertial, MoCap system, that will serve to not only monitor the worker’s posture, but also to assess the user’s ergonomics and provide posture awareness to the user, through biofeedback motors. Furthermore, it was also designed to integrate a human-robot collaborative framework. To achieve this, a user-centred design methodology was applied, starting with analyzing the State of Art (SOA), assessing the limitations of the previous system, defining the system’s requirements, developing the garment, hardware architecture and software architecture of the system. Lastly, the system was validated to ascertain if it is in conformity with the specified requirements. The developed system is composed of 9 Inertial Measurement Units (IMUs), placed on the lower and upper back, head, upper arms, forearms and hands. An actuation system was also integrated, for postural biofeedback, and it is comprised of 6 vibrotactile motors, located in the lower back, and in close proximity to the neck, elbows and wrists. The system is powered by a powerbank and all of the acquired data is sent to a main station, via WiFi (UDP), granting a standalone characteristic. The system integrates an Extended Complementary Filter (ECF) and a static Sensor-to-Segment (STS) calibration sequence to increase the joint angle estimation accuracy. Furthermore, the system is able to sample the angular data at 240 Hz, while the previous system was able to sample it at a maximum 100 Hz, improving the resolution of the data acquisition. The system was validated in terms of hardware and usability. The hardware tests addressed the char acterization of the system’s autonomy, sampling frequency, mechanical robustness and wireless commu nication performance in different contexts, and ascertain if they comply with the technical requirements, which was the case. Moreover, the specifications of the new version were compared with the previous one, where a significant direct improvement was observed, such as, higher sampling frequency, lower packet loss, lower current consumption, among others, and with a commercial system of reference (XSens Link). Usability tests were carried out with 9 participants who performed several uni-joint and complex motions. After testing, users answered a questionnaire based on the System Usability Scale (SUS). The system was very well accepted by the participants, regarding aesthetics and overall comfort, proving to have a high level of wearability

Orientation Estimation Through Magneto-Inertial Sensor Fusion: A Heuristic Approach for Suboptimal Parameters Tuning

Magneto-Inertial Measurement Units (MIMUs) are a valid alternative tool to optical stereophotogrammetry in human motion analysis. The orientation of a MIMU may be estimated by using sensor fusion algorithms. Such algorithms require input parameters that are usually set using a trial-and-error (or grid-search ) approach to find the optimal values. However, using trial-and-error requires a known reference orientation, a circumstance rarely occurring in real-life applications. In this article, we present a way to suboptimally set input parameters, by exploiting the assumption that two MIMUs rigidly connected are expected to show no orientation difference during motion. This approach was validated by applying it to the popular complementary filter by Madgwick et al. and tested on 18 experimental conditions including three commercial products, three angular rates, and two dimensionality motion conditions. Two main findings were observed: i) the selection of the optimal parameter value strongly depends on the specific experimental conditions considered, ii) in 15 out of 18 conditions the errors obtained using the proposed approach and the trial-and-error were coincident, while in the other cases the maximum discrepancy amounted to 2.5 deg and less than 1.5 deg on average

IMUs: validation, gait analysis and system’s implementation

Dissertação de mestrado integrado em Engenharia Biomédica (área de especialização em Eletrónica Médica)Falls are a prevalent problem in actual society. The number of falls has been increasing greatly in the last fifteen years. Some falls result in injuries and the cost associated with their treatment is high. However, this is a complex problem that requires several steps in order to be tackled. Namely, it is crucial to develop strategies that recognize the mode of locomotion, indicating the state of the subject in various situations, namely normal gait, step before fall (pre-fall) and fall situation. Thus, this thesis aims to develop a strategy capable of identifying these situations based on a wearable system that collects information and analyses the human gait. The strategy consists, essentially, in the construction and use of Associative Skill Memories (ASMs) as tools for recognizing the locomotion modes. Consequently, at an early stage, the capabilities of the ASMs for the different modes of locomotion were studied. Then, a classifier was developed based on a set of ASMs. Posteriorly, a neural network classifier based on deep learning was used to classify, in a similar way, the same modes of locomotion. Deep learning is a technique actually widely used in data classification. These classifiers were implemented and compared, providing for a tool with a good accuracy in recognizing the modes of locomotion. In order to implement this strategy, it was previously necessary to carry out extremely important support work. An inertial measurement units’ (IMUs) system was chosen due to its extreme potential to monitor outpatient activities in the home environment. This system, which combines inertial and magnetic sensors and is able to perform the monitoring of gait parameters in real time, was validated and calibrated. Posteriorly, this system was used to collect data from healthy subjects that mimicked Fs. Results have shown that the accuracy of the classifiers was quite acceptable, and the neural networks based classifier presented the best results with 92.71% of accuracy. As future work, it is proposed to apply these strategies in real time in order to avoid the occurrence of falls.As quedas são um problema predominante na sociedade atual. O número de quedas tem aumentado bastante nos últimos quinze anos. Algumas quedas resultam em lesões e o custo associado ao seu tratamento é alto. No entanto, trata-se de um problema complexo que requer várias etapas a serem abordadas. Ou seja, é crucial desenvolver estratégias que reconheçam o modo de locomoção, indicando o estado do sujeito em várias situações, nomeadamente, marcha normal, passo antes da queda (pré-queda) e situação de queda. Assim, esta tese tem como objetivo desenvolver uma estratégia capaz de identificar essas situações com base num sistema wearable que colete informações e analise a marcha humana. A estratégia consiste, essencialmente, na construção e utilização de Associative Skill Memories (ASMs) como ferramenta para reconhecimento dos modos de locomoção. Consequentemente, numa fase inicial, foram estudadas as capacidades das ASMs para os diferentes modos de locomoção. Depois, foi desenvolvido um classificador baseado em ASMs. Posteriormente, um classificador de redes neuronais baseado em deep learning foi utilizado para classificar, de forma semelhante, os mesmos modos de locomoção. Deep learning é uma técnica bastante utilizada em classificação de dados. Estes classificadores foram implementados e comparados, fornecendo a uma ferramenta com uma boa precisão no reconhecimento dos modos de locomoção. Para implementar esta estratégia, era necessário realizar previamente um trabalho de suporte extremamente importante. Um sistema de unidades de medição inercial (IMUs), foi escolhido devido ao seu potencial extremo para monitorizar as atividades ambulatórias no ambiente domiciliar. Este sistema que combina sensores inerciais e magnéticos e é capaz de efetuar a monitorização de parâmetros da marcha em tempo real, foi validado e calibrado. Posteriormente, este Sistema foi usado para adquirir dados da marcha de indivíduos saudáveis que imitiram quedas. Os resultados mostraram que a precisão dos classificadores foi bastante aceitável e o classificador baseado em redes neuronais apresentou os melhores resultados com 92.71% de precisão. Como trabalho futuro, propõe-se a aplicação destas estratégias em tempo real de forma a evitar a ocorrência de quedas