Smart Footwear Insole for Recognition of Foot Pronation and Supination Using Neural Networks

Abnormal foot postures during gait are common sources of pain and pathologies of the lower limbs. Measurements of foot plantar pressures in both dynamic and static conditions can detect these abnormal foot postures and prevent possible pathologies. In this work, a plantar pressure measurement system is developed to identify areas with higher or lower pressure load. This system is composed of an embedded system placed in the insole and a user application. The instrumented insole consists of a low-power microcontroller, seven pressure sensors and a low-energy bluetooth module. The user application receives and shows the insole pressure information in real-time and, finally, provides information about the foot posture. In order to identify the different pressure states and obtain the final information of the study with greater accuracy, a Deep Learning neural network system has been integrated into the user application. The neural network can be trained using a stored dataset in order to obtain the classification results in real-time. Results prove that this system provides an accuracy over 90% using a training dataset of 3000+ steps from 6 different users.Ministerio de Economía y Competitividad TEC2016-77785-

Master of Science

thesisComputing and data acquisition have become an integral part of everyday life. From reading emails on cell phones to kids playing with motion sensing game consoles, we are surrounded with sensors and mobile computing devices. As the availability of powerful computing devices increases, applications in previously limited environments become possible. Training devices in rehabilitation are becoming increasingly common and more mobile. Community based rehabilitative devices are emerging that embrace these mobile advances. To further the flexibility of devices used in rehabilitation, research has explored the use of smartphones as a means to process data and provide feedback to the user. In combination with sensor embedded insoles, smartphones provide a powerful tool for the clinician in gathering data and as a standalone training tool in rehabilitation. This thesis presents the continuing research of sensor based insoles, feedback systems and increasing the capabilities of the Adaptive Real-Time Instrumentation System for Tread Imbalance Correction, or ARTISTIC, with the introduction of ARTISTIC 2.0. To increase the capabilities of the ARTISTIC an Inertial Measurement Unit (IMU) was added, which gave the system the ability to quantify the motion of the gait cycle and, more specifically, determine stride length. The number of sensors in the insole was increased from two to ten, as well as placing the microprocessor and a vibratory motor in the insole. The transmission box weight was reduced by over 50 percent and the volume by over 60 percent. Stride length was validated against a motion capture system and found the average stride length to be within 2.7 ± 6.9 percent. To continue the improvement of the ARTISTIC 2.0, future work will include implementing real-time stride length feedback

Optimal locations and computational frameworks of FSR and IMU sensors for measuring gait abnormalities

Neuromuscular diseases cause abnormal joint movements and drastically alter gait patterns in patients. The analysis of abnormal gait patterns can provide clinicians with an in-depth insight into implementing appropriate rehabilitation therapies. Wearable sensors are used to measure the gait patterns of neuromuscular patients due to their non-invasive and cost-efficient characteristics. FSR and IMU sensors are the most popular and efficient options. When assessing abnormal gait patterns, it is important to determine the optimal locations of FSRs and IMUs on the human body, along with their computational framework. The gait abnormalities of different types and the gait analysis systems based on IMUs and FSRs have therefore been investigated. After studying a variety of research articles, the optimal locations of the FSR and IMU sensors were determined by analysing the main pressure points under the feet and prime anatomical locations on the human body. A total of seven locations (the big toe, heel, first, third, and fifth metatarsals, as well as two close to the medial arch) can be used to measure gate cycles for normal and flat feet. It has been found that IMU sensors can be placed in four standard anatomical locations (the feet, shank, thigh, and pelvis). A section on computational analysis is included to illustrate how data from the FSR and IMU sensors are processed. Sensor data is typically sampled at 100 Hz, and wireless systems use a range of microcontrollers to capture and transmit the signals. The findings reported in this article are expected to help develop efficient and cost-effective gait analysis systems by using an optimal number of FSRs and IMUs

Master of Science

thesisComputing and data acquisition have become an integral part of everyday life. From reading emails on a cell phone, to kids playing with motion sensing game consoles, we are surrounded with sensors and mobile devices. As the availability of powerful mobile computing devices expands, the road is paved for applications in previously limited environments. Rehabilitative devices are emerging that embrace these mobile advances. Research has explored the use of smartphones in rehabilitation as a means to process data and provide feedback in conjunction with established rehabilitative methods. Smartphones, combined with sensor embedded insoles, provide a powerful tool for the clinician in gathering data and may act as a standalone training technique. This thesis presents continuing research of a sensor integrated insole system that provides real-time feedback through a mobile platform, the Adaptive Real-Time Instrumentation System for Tread Imbalance Correction (ARTISTIC). The system interfaces a wireless instrumented insole with an Android smartphone application to receive gait data and provide sensory feedback to modify gait patterns. Revisions to the system hardware, software, and feedback modes brought about the introduction of the ARTISTIC 2.0. The number of sensors in the insole was increased from two to 10. The microprocessor and a vibrotactile motor were embedded in the insole and the communications box was reduced in size and weight by more than 50%. Stance time iv measurements were validated against force plate equipment and found to be within 13.5 ± 3.3% error of force plate time measurements. Human subjects were tested using each of the feedback modes to alter gait symmetry. Results from the testing showed that more than one mode of feedback caused a statistically significant change in gait symmetry ratios (p < 0.05). Preference of feedback modes varied among subjects, with the majority agreeing that several feedback modes made a difference in their gait. Further improvements will prepare the ARTISTIC 2.0 for testing in a home environment for extended periods of time and improve data capture techniques, such as including a database in the smartphone application

Doctor of Philosophy

dissertationThis thesis analyzed biped stability through a qualitative likelihood of falling and quantitative Potential to Fall (PF) analysis. Both analyses were applied to walking and skiing to better understand behaviors across a wider spectrum of bipedal gaits. For both walking and skiing, two types of locomotion were analyzed. Walking studies compared normal locomotion (gait) to an unexpected slip. Skiing studies compared wedge style locomotion (more common to beginning and intermediate skiers) to parallel style locomotion (more common to advanced and expert skiers). Two mediums of data collection were used. A motion capture laboratory with stereographic cameras and force plates were used for walking studies, and instrumented insoles, capable of force and inertial measurement, were used for skiing studies. Both kinematics and kinetics were used to evaluate the likelihood of falling. The PF metric, based on root mean squared error, was used to quantify the likelihood of falling for multiple subjects both in walking and skiing. PF was based on foot kinematics for walking and skiing studies. PF also included center of pressure for skiing studies. The PF was lower for normal gaits in walking studies and wedge style locomotion for skiing studies

Performance of the Intrac Wireless Activity Tracking System for the Afari Assistive Device

Afari is a mobility device that was designed to be more recreational, aesthetic, and functional outside than the typical mobility devices commonly used today such as walkers, crutches, and rollators. The Afari transfers weight from a user through the arm rests and enforces an upright posture while walking with correct adjustments to the arm rest height. In addition to assisting with walking or running, a sensor system fitted to the Afari device has been designed to analyze different aspects of activity tracking such as the dynamic loading applied to the arm rests, spatial-temporal gait parameters, speed, and distance. This includes various sensors, namely, load cells for each arm rest, an inertial measurement unit, and a speed and distance sensor that wirelessly transmit data via Bluetooth Low Energy (BLE) to either a smartphone or computer. The total distance, pitch angle, right and left loading on each armrest can be viewed in real time by the user. An algorithm was created in MATLAB to process all the raw data and compute cadence, stride length, average toe-off and heel strike angle, swing and stance time, and speed over the duration of active use. An Afari user can monitor these different aspects of their activity and adjust accordingly to potentially improve their balance or gait

Powered knee orthosis for human gait rehabilitation: first advances

This paper presents a new system for a powered knee orthosis, that was designed to assist and improve the gait function of patients with gait pathologies. The system contains the orthotic device (embedded with sensors for angle and user-orthosis interaction torque measurements, and an electric actuator) and wearable sensors (inertial measurement unit, force sensitive resistors, and electromyography sensors), which allows the generation of smart rehabilitation tools and several motion assistive techniques. The main goal is to present a conceptual overview and functional description of the system and use scenarios of each component. The attachment mechanism of the orthosis to the limb is also highlighted, being composed of a straps system fixed in the mechanical links of the joint. It was noticed that users with distinct lower-limb morphologies can presents difficulties wearing the orthosis, since the device needs constant adjust to align the mechanical and human joints. The system was validated in ground-level walking on healthy subjects, with emphasis on the impact of the device in the user. The subjects reported that the orthosis is comfortable to use, easy to wear, and no issues were raised regarding the aesthetics of the device. Only the weight was assimilated as a possible hindrance (compensated in the future). Future challenges involve the inclusion of an ankle joint in the system and the use of the proposed tool in rehabilitation.This work is supported by the FCT - Fundacao para a Ciencia e Tecnologia - with the reference scholarship SFRH/BD/108309/2015, with the reference project UID/EEA/04436/2013, and by FEDER funds through the COMPETE 2020 - Programa Operacional Competitividade e Internacionalizacao (POCI) - with the reference project POCI-01-0145-FEDER-006941, and partially supported with grant RYC-2014-16613 by Spanish Ministry of Economy and Competitiveness

Modeling the gait using an IMU and FSR

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Engenharia Clínica e Instrumentação Médica), Universidade de Lisboa, Faculdade de Ciências, 2020Síndrome de drop-foot é uma condição que dificulta ou impossibilita a dorsiflexão do pé. Consiste numa deficiência de comunicação com o musculo tibialis anterior, que é feita através do nervo profundo peronial (DPN), que resulta na não ativação do músculo necessário para tal movimento. Esta condição é geralmente fruto de um acidente neurológico ao nível do cérebro (um acidente vascular cerebral, por exemplo) ou algum tipo de lesão no nervo que ativa o músculo, o que resulta numa ativação deficiente do mesmo. Estas lesões e as suas consequências podem ter diversos sintomas e efeitos, que, dependendo da severidade do caso, podem ou não ser parcialmente ou totalmente reversíveis. Sendo assim, é de extrema importância um correto diagnóstico e escolha de terapia, de forma a proporcionar a pacientes com esta condição o melhor plano de tratamento. De forma a diagnosticar esta e outras condições, são utilizados modelos da passada, de forma a comparar a passada de um paciente ao esperado e determinar a gravidade da lesão. Este projeto tem como objetivo testar um novo método de modelar a passada, usando sensores inerciais (IMUs) ligados a um computador através de um microcontrolador. O dispositivo utilizado consiste num microcontrolador, uma placa lógica e os sensores. O dispositivo é ligado ao computador através de um cabo USB. O microcontrolador já continha todo o código necessário para o seu funcionamento e emparelhava com o interface instalado no computador. Existem 2 tipos de sensores ligados ao dispositivo: sensores de pressão (FSRs) e sensores inerciais (IMUs). Para este projeto, foram utilizados 5 FSRs e 1 IMU. Os sensores FSR são posicionados na planta do pé direito, de forma a detetar quando o pé toca no chão. Os dados adquiridos destes sensores irão determinar o início e fim de um passo. O sensor IMU deteta velocidade angular e aceleração linear. Estes dados irão ser exportados, através de um ficheiro .csv, para um script no MATLAB que aplica os integrais necessários de forma a obter posição e angulo do pé (em relação ao solo). O sensor obtém estes dados calculando 3 frames principais: o frame intrínseco sensor, que se mantem estático em relação ao sensor e, consequentemente, ao pé, o frame da posição inicial, que é calculado a partir do vetor de gravidade detetado pelo sensor, e o frame do passo, que, como o nome indica, é formado quando é detetado o inicio de um passo. Este último é calculado em relação ao frame definido pela gravidade. De notar que, enquanto os dois primeiros são calculados intricadamente no microcontrolador de forma a obter os dados necessários à análise, este último tipo de frame é formado durante os cálculos feitos após o registo dos dados. Este cálculo, feito num script de MATLAB, é feito usando um modelo já previsto. O interface é responsável pela recolha e registo dos dados. Quando ativado, ele irá registar os dados num ficheiro .csv, que poderá posteriormente ser lido pelo script MATLAB que irá calcular todos os valores necessários para análise. Para a análise, foram calculados e usados para comparação a posição do pé no eixo Z (eixo anti paralelo ao vetor gravidade) e o angulo do pé em relação ao solo. Como o sensor IMU apenas deteta aceleração linear e velocidade angular, são necessários integrais para calcular os dados necessários à analise. Estes cálculos foram feitos num script MATLAB modificados para os fins deste projeto. Para os cálculos, o script usa a informação do sensor FSR (informação essa é processada anteriormente. O resultado desse processamento é um vetor de zeros de tamanho igual ao dos dados adquiridos, que, quando é detetado um passo, altera o zero correspondente a esse instante para um) para dividir os dados em frames. Por cada novo frame (equivalente a um novo passo), é inicializado um novo integral com novas constantes iniciais. Estas constantes iniciais são assumidas pelo sistema. Por fim, todos os dados resultantes do cálculo integral são guardados numa matriz, que é utilizada para todas as representações gráficas e alterações necessárias para analise, como por exemplo, o alinhamento dos paços de modo a poder calcular uma curva de valores médios para a posição e angulo do pé ao longo da passada. Os sujeitos (todos indivíduos com passada saudável) foram instruídos a andar a passo regular, lento e rápido, com o dispositivo instalado na perna direita. De notar que o conceito de “passo regular”, “lento” e “rápido” foi deixado ao critério de cada sujeito, não tendo existido um compasso para os mesmos seguirem. Consequentemente, pode haver comparação matemática (i.e., comparação e analise matemática, feita de modo quantitativo, dentro de uma função do MATLAB criada com esse propósito) entre ensaios do mesmo sujeito, para a mesma velocidade, mas o mesmo não pode acontecer para diferentes sujeitos, ou diferentes velocidades. Ao longo do processo, tanto o script como a posição dos sensores foram otimizados de modo a conseguir os resultados mais confiáveis, realistas e reproduzíveis. Os resultados dos ensaios, após o tratamento de dados, foram comparados com os modelos esperados na literatura. De um modo geral, os resultados foram razoavelmente semelhantes aos esperados, não se conseguido distinguir nenhuma diferença sistemática entre os valores referentes às diferentes velocidades da passada. No entanto, dois erros sistemáticos devem ser mencionados. O primeiro corresponde às constantes iniciais referidas acima. Porque o integral aplicado à aceleração linear é um integral duplo (de forma a obter a posição), o sistema assume duas constantes como sendo 0: a posição inicial (correspondente á origem do frame criado quando se inicia uma nova passada) e a velocidade linear inicial. No entanto, esta velocidade, na prática, nunca é 0 absoluto. Devido a este facto, existe um declive entre a posição inicial e a final onde os valores das mesmas deveriam ser ambos iguais a zero. Especificamente, no eixo Z (eixo perpendicular ao chão, com direção ascendente). Após análise, foi concluído que este erro não tem qualquer padrão, tanto entre sujeitos como quando comparando ensaios do mesmo sujeito. O segundo erro corresponde a diferença entre as coordenadas do fim de um paço e as do início do próximo, no frame da posição inicial. Este erro deve-se não só, mas também ao método de deteção do passo. Idealmente, estas coordenadas, no frame da posição inicial, seriam matematicamente iguais (i.e., o instante que finaliza um passo inicia o próximo). No entanto, devido ao método utilizado no projeto (ambos de iniciar um novo integral por passo e da deteção pelo sensor FSR), pelo menos um instante de tempo não irá ser calculado (porque a frequência de aquisição foi de 80Hz, este instante corresponde a 1/80 de segundo). Sendo assim, e porque novas constantes iniciais são calculadas a cada novo passo, existe uma discrepância entre estes dois instantes, em particular no eixo Z. Tal como no erro anterior, não foi encontrado qualquer tipo de padrão, tanto intra-sujeito como inter-sujeito. Apesar das limitações do método, existe potencial em utilizar este método para modelar a passada com resultados repetíveis e confiáveis. Para melhorar o seu funcionamento, o script onde é calculado os integrais necessita de otimização, e seria necessário um método mais robusto e standard de aplicar o IMU ao pé de futuros sujeitos. Ainda assim, este provou ser um bom método para criar futuros modelos da passada com o propósito de desenvolver melhores técnicas de diagnóstico.Drop-foot syndrome is a condition that consists of an inability or difficulty of pulling the foot upwards at the ankle joint. It usually has a neurological cause, where the deep peroneal nerve is not properly activated, meaning that the tibialis anterior, responsible for the dorsiflexion of the foot, is not activated, which functionally results in “drop” of the foot while walking. To properly diagnose this condition, and other conditions related to the gait, it’s important that a base gait model is established, for comparison. Throughout the years, different techniques have been used for this purpose, from imaging to using sensors to track the foot while in movement. As technology advances, new, cheaper and more accurate ways to track and model the gait have emerged. In this project, the tracking of the foot was made by an IMU (Inertial measurement unit), while also using FSR sensors (Force sensitive resistors) to distinguish one step and the next. All the sensors were connected to a microcontroller that rested on the right leg (the leg under analysis) and were itself connected to a computer that recorded the data, via a prebuilt interface. Then, said data, recorded in a .csv file, were analyzed by a MATLAB script, that calculated the position and foot angle. This method proved effective, but incomplete to create said model. Although still on an initial stage, this method can be improved to be a reliable and useful method to build gait models in the future

Augmented Human Inspired Phase Variable Using a Canonical Dynamical System

Accurately parameterizing human gait is highly important in the continued development of assistive robotics, including but not limited to lower limb prostheses and exoskeletons. Previous studies introduce the idea of time-invariant real-time gait parameterization via human-inspired phase variables. The phase represents the location or percent of the gait cycle the user has progressed through. This thesis proposes an alternative approach for determining the gait phase leveraging previous methods and a canonical dynamical system. Human subject experiments demonstrate the ability to accurately produce a phase variable corresponding to the human gait progression for various walking configurations. Configurations include changes in incline and speed. Results show an augmented real-time approach capable of adapting to different walking conditions

A review on locomotion mode recognition and prediction when using active orthoses and exoskeletons

Understanding how to seamlessly adapt the assistance of lower-limb wearable assistive devices (active orthosis (AOs) and exoskeletons) to human locomotion modes (LMs) is challenging. Several algorithms and sensors have been explored to recognize and predict the users’ LMs. Nevertheless, it is not yet clear which are the most used and effective sensor and classifier configurations in AOs/exoskeletons and how these devices’ control is adapted according to the decoded LMs. To explore these aspects, we performed a systematic review by electronic search in Scopus and Web of Science databases, including published studies from 1 January 2010 to 31 August 2022. Sixteen studies were included and scored with 84.7 ± 8.7% quality. Decoding focused on level-ground walking along with ascent/descent stairs tasks performed by healthy subjects. Time-domain raw data from inertial measurement unit sensors were the most used data. Different classifiers were employed considering the LMs to decode (accuracy above 90% for all tasks). Five studies have adapted the assistance of AOs/exoskeletons attending to the decoded LM, in which only one study predicted the new LM before its occurrence. Future research is encouraged to develop decoding tools considering data from people with lower-limb impairments walking at self-selected speeds while performing daily LMs with AOs/exoskeletons.This work was funded in part by the Fundação para a Ciência e Tecnologia (FCT) with the Reference Scholarship under grant 2020.05711.BD, under the Stimulus of Scientific Employment with the grant 2020.03393.CEECIND, and in part by the FEDER Funds through the COMPETE 2020— Programa Operacional Competitividade e Internacionalização (POCI) and P2020 with the Reference Project SmartOs Grant POCI-01-0247-FEDER-039868, and by FCT national funds, under the national support to R&D units grant, through the reference project UIDB/04436/2020 and UIDP/04436/2020