Motion synthesis for sports using unobtrusive lightweight body-worn and environment sensing

The ability to accurately achieve performance capture of athlete motion during competitive play in near real-time promises to revolutionise not only broadcast sports graphics visualisation and commentary, but also potentially performance analysis, sports medicine, fantasy sports and wagering. In this paper, we present a highly portable, non-intrusive approach for synthesising human athlete motion in competitive game-play with lightweight instru- mentation of both the athlete and field of play. Our data-driven puppetry technique relies on a pre-captured database of short segments of motion capture data to construct a motion graph augmented with interpolated mo- tions and speed variations. An athlete’s performed motion is synthesised by finding a related action sequence through the motion graph using a sparse set of measurements from the performance, acquired from both worn inertial and global location sensors. We demonstrate the efficacy of our approach in a challenging application scenario, with a high-performance tennis athlete wearing one or more lightweight body-worn accelerometers and a single overhead camera providing the athlete’s global position and orientation data. However, the approach is flexible in both the number and variety of input sensor data used. The technique can also be adopted for searching a motion graph efficiently in linear time in alternative applications

Survey of Motion Tracking Methods Based on Inertial Sensors: A Focus on Upper Limb Human Motion

Motion tracking based on commercial inertial measurements units (IMUs) has been widely studied in the latter years as it is a cost-effective enabling technology for those applications in which motion tracking based on optical technologies is unsuitable. This measurement method has a high impact in human performance assessment and human-robot interaction. IMU motion tracking systems are indeed self-contained and wearable, allowing for long-lasting tracking of the user motion in situated environments. After a survey on IMU-based human tracking, five techniques for motion reconstruction were selected and compared to reconstruct a human arm motion. IMU based estimation was matched against motion tracking based on the Vicon marker-based motion tracking system considered as ground truth. Results show that all but one of the selected models perform similarly (about 35 mm average position estimation error)

Explaining the Ergonomic Assessment of Human Movement in Industrial Contexts

Manufacturing processes are based on human labour and the symbiosis between human operators and machines. The operators are required to follow predefined sequences of movements. The operations carried out at assembly lines are repetitive, being identified as a risk factor for the onset of musculoskeletal disorders. Ergonomics plays a big role in preventing occupational diseases. Ergonomic risk scores measure the overall risk exposure of operators however these methods still present challenges: the scores are often associated to a given workstation, being agnostic to the variability among operators. Observation methods are most often employed yet require a significant amount of effort, preventing an accurate and continuous ergonomic evaluation to the entire population of operators. Finally, the risk’s results are rendered as index scores, hindering a more comprehensive interpretation by occupational physicians. This dissertation developed a solution for automatic operator risk exposure in assembly lines. Three main contributions were presented: (1) an upper limb and torso motion tracking algorithm which relies on inertial sensors to estimate the orientation of anatomical joints; (2) an adjusted ergonomic risk score; (3) an ergonomic risk explanation approach based on the analysis of the angular risk factors. Throughout the research, two experimental assessments were conducted: laboratory validation and field evaluation. The laboratory tests enabled the creation of a movements’ dataset and used an optical motion capture system as reference. The field evaluation dataset was acquired on an automotive assembly line and serve as the basis for an ergonomic risk evaluation study. The experimental results revealed that the proposed solution has the potential to be applied in a real environment. Through direct measures, the ergonomic feedback is fastened, and consequently, the evaluation can be extended to more operators, ultimately preventing, in long-term, work-related injuries

Drift Reduction for Inertial Sensor Based Orientation and Position Estimation in the Presence of High Dynamic Variability During Competitive Skiing and Daily-Life Walking

Nowadays inertial sensors are extensively used for gait analysis. They can be used to perform temporal event detection (i.e. step detection) and to estimate the orientation of the feet and other body segments to determine walking speed and distance. Usually, orientation is estimated from integration of the measured angular velocity. Prior to integration of measured acceleration to obtain speed, the gravity component has to be estimated and removed. During each integration small measurement errors accumulate and result in so-called drift. Since the first uses of inertial sensors for gait analysis methods have been presented to model, estimate and remove the drift. The proposed methods worked well for relatively slow movements and movements taking place in the sagittal plane. Many methods also relied on periodically occurring static phases such as the stance phase during walking to correct the drift. Inertial sensors could also be used to track higher dynamic movements, for example in sports. Potential applications focus on two aspects: performance analysis and injury prevention. To better explain and predict performance, in-field measurements to assess the coordination, kinematics, and dynamics are key. While traditional movement analysis (e.g. video analysis) can answer most of the questions related to both performance and injury, they are cumbersome and complex to use in-field. Inertial sensors, however, are perfectly suited since they allow to measure the movement in any environment and are not restricted to certain capture volumes. Nevertheless, most sports have very high movement dynamics (e.g. fast direction changes, high speeds) and are therefore challenging for computing reliable estimates of orientation, speed and position. The inertial measurements are compromised by noise and movements oftentimes don't provide static or slow phases used in gait analysis for drift correction. Therefore, the present thesis aimed to propose and validate new methods to model, estimate and remove drift in sports and for movements taking place outdoors in uncontrolled environments. Three different strategies were proposed to measure the movement of classical cross-country skiing and ski mountaineering, alpine ski racing, and outdoor walking over several kilometres. For each activity specific biomechanical constraints and movement dynamics were exploited. The proposed methods rely only on inertial sensors and magnetometers and are able to provide orientation, speed, and position information with an accuracy and precision close to existing gold standards. The most complete system was designed in alpine ski racing, probably one of the most challenging sports for movement analysis. Extreme vibrations, high speeds of over 120 km/h and a timing resolution below 0.01 seconds require maximum accuracy and precision. The athlete's posture and the kinematics of his centre of mass both in a relative athlete-centred frame and in a global Earth-fixed frame could be obtained with high accuracy and precision. Where 3D video analysis requires a very complex experimental setup and takes several hours of post processing to analyse a single turn of a skier, the proposed system allows to measure multiple athletes and complete runs within minutes. Thus, new experimental designs to assess performance and injury risk in alpine ski racing became feasible, greatly helping to gain further knowledge about this highly complex and risky sport

Low-Cost Sensors and Biological Signals

Many sensors are currently available at prices lower than USD 100 and cover a wide range of biological signals: motion, muscle activity, heart rate, etc. Such low-cost sensors have metrological features allowing them to be used in everyday life and clinical applications, where gold-standard material is both too expensive and time-consuming to be used. The selected papers present current applications of low-cost sensors in domains such as physiotherapy, rehabilitation, and affective technologies. The results cover various aspects of low-cost sensor technology from hardware design to software optimization

Functional Rotation Axis Based Approach for Estimating Hip Joint Angles Using Wearable Inertial Sensors: Comparison to Existing Methods

Wearable sensors are at the heart of the digital health revolution. Integral to the use of these sensors for monitoring conditions impacting balance and mobility are accurate estimates of joint angles. To this end a simple and novel method of estimating hip joint angles from small wearable magnetic and inertial sensors is proposed and its performance is established relative to optical motion capture in a sample of human subjects. Improving upon previous work, this approach does not require precise sensor placement or specific calibration motions, thereby easing deployment outside of the research laboratory. Specific innovations include the determination of sensor to segment rotations based on functionally determined joint centers, and the development of a novel filtering algorithm for estimating the relative orientation of adjacent body segments. Hip joint angles and range of motion determined from the proposed approach and an existing method are compared to those from an optical motion capture system during walking at a variety of speeds and tasks designed to exercise the hip through its full range of motion. Results show that the proposed approach estimates flexion/extension angle more accurately (RMSE from 7.08 to 7.29 deg) than the existing method (RMSE from 11.64 deg to 14.33 deg), with similar performance for the other anatomical axes. Agreement of each method with optical motion capture is further characterized by considering correlation and regression analyses. Mean ranges of motion for the proposed method are not largely different from those reported by motion capture, and showed similar values to the existing method. Results indicate that this algorithm provides a promising approach for estimating hip joint angles using wearable inertial sensors, and would allow for use outside of constrained research laboratories

Quantifying the ergonomic risk and biomechanical exposure in automotive assembly lines

Tese de Mestrado Integrado, Engenharia Biomédica e Biofísica (Biofísica Médica e Fisiologia de Sistemas), 2021, Universidade de Lisboa, Faculdade de CiênciasAs Lesões Músculo-esqueléticas Relacionadas com o Trabalho (LMERTs) representam 15% do número total de anos de vida perdidos por danos físicos ou doenças com a sua génese no trabalho. De entre os fatores de risco para as LMERTs, no presente estudo, destacam-se as posturas corporais relacionadas com o trabalho. A exposição biomecânica a posturas consideradas prejudiciais tem um impacto negativo na saúde dos trabalhadores, na economia das empresas e na sociedade. A fim de aperceber a prática recorrente de posturas prejudiciais no local de trabalho, têm sido invocados métodos de autoavaliação ergonómica, nos quais o risco é percecionado pelo próprio trabalhador; observacionais, conduzidos por peritos em ergonomia; e de medição direta, que recorrem ao emprego de soluções tecnológicas para a recolha e monitorização objetiva de variáveis pertinentes para a avaliação ergonómica. Porém, frequentemente e em contexto industrial, são apenas aplicados métodos de autoavaliação e observacionais, apesar da medição direta constituir uma solução mais notável. O advento da Internet das Coisas vem revelar a oportunidade da utilização de wearables para uma recolha de dados omnipresente, amplificando a quantidade de dados disponível com o fim de uma avaliação ergonómica mais individual e imparcial. Deste modo, estudos relativos à avaliação ergonómica no local de trabalho têm primado pelo uso de wearables com vista a monitorização do movimento humano. A presente dissertação respeita ao desenvolvimento de uma abordagem automática para a avaliação ergonómica em contexto industrial. As contribuições principais são o desenvolvimento de (1) uma rotina de captura de movimento, através da utilização de um sistema wearable com sensores inerciais; (2) uma framework computacional para a monitorização do movimento da parte superior do corpo humano, em termos dos ângulos relativos às articulações entre os segmentos anatómicos, estimados com recurso à cinemática inversa; e (3) implementações computacionais de especificações estabelecidas e relativas aos fatores de risco de postura para a quantificação da exposição biomecânica e consequente risco ergonómico em âmbito ocupacional. Subsequentemente, as implementações das especificações foram aplicadas por forma a prover constatações acerca de um caso de estudo das linhas de montagem de automóveis da Volkswagen Autoeuropa. O estudo delineado foi dividido em dois cenários: validação e avaliação. A validação consistiu em comparar os dados provisionados por um sistema inercial de referência e determinados através dos métodos desenvolvidos. Para tal, usaram-se dados de sensores inerciais recolhidos em laboratório (N = 8 participantes) e nas linhas de montagem de automóveis (N = 9 participantes). A avaliação consistiu em quantificar a exposição biomecânica e consequente risco ergonómico respeitantes ao caso de estudo, empregando as estimativas angulares calculadas pela framework desenvolvida, e a partir dos dados recolhidos com o nosso sistema nas linhas de montagem de automóveis. Os resultados revelaram que a framework proposta tem o potencial para ser aplicada na monitorização de tarefas industriais. A avaliação ergonómica é mais lata através da medição direta, desvendando diferenças de exposição biomecânica e consequente risco ergonómico entre operadores.Work-related musculoskeletal disorders (WRMSDs) represent 15% of the total number of life-years lost due to work-related injuries and illness. Among WRMSDs’ risk factors, work-related postures are underlined in this research. Biomechanical exposure to hazardous postures negatively impacts workers’ health, enterprises’ economy, and society. Toward the apperception about the recurrent practice of hazardous postures in the workplace, self-reported, observational, and directly measured ergonomic assessment methods have been established. However, only self-reported and observational approaches are enforced on a more frequent basis, besides directly measured is a more compelling choice. The advent of the Internet of Things poses the opportunity of using wearables in the direction of ubiquitous data collection, increasing the amount of available data for a more personal and non-biased ergonomic evaluation. As follows, over workplace ergonomics research, wearables have been used to monitor human motion. The dissertation developed an automatic approach to ergonomic evaluation in industrial contexts. Its main contributions are the development of (1) a motion capture routine using inertial sensors; (2) a computational framework to monitor human upper body motion, in terms of joints’ angles, through inverse kinematics; and (3) computational implementations of posture risk factors specifications to quantify the biomechanical exposure and consequent ergonomic risk in occupational settings. Subsequently, specifications implementations were applied to provide insights in consideration of a case study from Volkswagen Autoeuropa automotive assembly lines. The research was divided into two scenarios: validation and evaluation. Validation consisted of comparing data provided by a ground truth inertial motion capture system and computed throughout the developed methods. Hence, inertial sensors’ data, collected in the laboratory (N = 8 participants) and automotive assembly lines (N = 9 participants) settings, were used. The evaluation consisted of quantifying the biomechanical exposure and consequent ergonomic risk concerning the case study, using angular estimates computed through the developed framework and about data collected in automotive assembly lines. The results revealed that the proposed framework has the potential to be applied to monitor industrial tasks. The ergonomic evaluation is more comprehensive through direct measures, uncovering differences about biomechanical exposure and consequent ergonomic risk among operators

Orientation Estimation Through Magneto-Inertial Sensor Fusion: A Heuristic Approach for Suboptimal Parameters Tuning

Magneto-Inertial Measurement Units (MIMUs) are a valid alternative tool to optical stereophotogrammetry in human motion analysis. The orientation of a MIMU may be estimated by using sensor fusion algorithms. Such algorithms require input parameters that are usually set using a trial-and-error (or grid-search ) approach to find the optimal values. However, using trial-and-error requires a known reference orientation, a circumstance rarely occurring in real-life applications. In this article, we present a way to suboptimally set input parameters, by exploiting the assumption that two MIMUs rigidly connected are expected to show no orientation difference during motion. This approach was validated by applying it to the popular complementary filter by Madgwick et al. and tested on 18 experimental conditions including three commercial products, three angular rates, and two dimensionality motion conditions. Two main findings were observed: i) the selection of the optimal parameter value strongly depends on the specific experimental conditions considered, ii) in 15 out of 18 conditions the errors obtained using the proposed approach and the trial-and-error were coincident, while in the other cases the maximum discrepancy amounted to 2.5 deg and less than 1.5 deg on average

The Use of Wearable Inertial Motion Sensors in Human Lower Limb Biomechanics Studies: A Systematic Review

Wearable motion sensors consisting of accelerometers, gyroscopes and magnetic sensors are readily available nowadays. The small size and low production costs of motion sensors make them a very good tool for human motions analysis. However, data processing and accuracy of the collected data are important issues for research purposes. In this paper, we aim to review the literature related to usage of inertial sensors in human lower limb biomechanics studies. A systematic search was done in the following search engines: ISI Web of Knowledge, Medline, SportDiscus and IEEE Xplore. Thirty nine full papers and conference abstracts with related topics were included in this review. The type of sensor involved, data collection methods, study design, validation methods and its applications were reviewed

IMUs: validation, gait analysis and system’s implementation

Dissertação de mestrado integrado em Engenharia Biomédica (área de especialização em Eletrónica Médica)Falls are a prevalent problem in actual society. The number of falls has been increasing greatly in the last fifteen years. Some falls result in injuries and the cost associated with their treatment is high. However, this is a complex problem that requires several steps in order to be tackled. Namely, it is crucial to develop strategies that recognize the mode of locomotion, indicating the state of the subject in various situations, namely normal gait, step before fall (pre-fall) and fall situation. Thus, this thesis aims to develop a strategy capable of identifying these situations based on a wearable system that collects information and analyses the human gait. The strategy consists, essentially, in the construction and use of Associative Skill Memories (ASMs) as tools for recognizing the locomotion modes. Consequently, at an early stage, the capabilities of the ASMs for the different modes of locomotion were studied. Then, a classifier was developed based on a set of ASMs. Posteriorly, a neural network classifier based on deep learning was used to classify, in a similar way, the same modes of locomotion. Deep learning is a technique actually widely used in data classification. These classifiers were implemented and compared, providing for a tool with a good accuracy in recognizing the modes of locomotion. In order to implement this strategy, it was previously necessary to carry out extremely important support work. An inertial measurement units’ (IMUs) system was chosen due to its extreme potential to monitor outpatient activities in the home environment. This system, which combines inertial and magnetic sensors and is able to perform the monitoring of gait parameters in real time, was validated and calibrated. Posteriorly, this system was used to collect data from healthy subjects that mimicked Fs. Results have shown that the accuracy of the classifiers was quite acceptable, and the neural networks based classifier presented the best results with 92.71% of accuracy. As future work, it is proposed to apply these strategies in real time in order to avoid the occurrence of falls.As quedas são um problema predominante na sociedade atual. O número de quedas tem aumentado bastante nos últimos quinze anos. Algumas quedas resultam em lesões e o custo associado ao seu tratamento é alto. No entanto, trata-se de um problema complexo que requer várias etapas a serem abordadas. Ou seja, é crucial desenvolver estratégias que reconheçam o modo de locomoção, indicando o estado do sujeito em várias situações, nomeadamente, marcha normal, passo antes da queda (pré-queda) e situação de queda. Assim, esta tese tem como objetivo desenvolver uma estratégia capaz de identificar essas situações com base num sistema wearable que colete informações e analise a marcha humana. A estratégia consiste, essencialmente, na construção e utilização de Associative Skill Memories (ASMs) como ferramenta para reconhecimento dos modos de locomoção. Consequentemente, numa fase inicial, foram estudadas as capacidades das ASMs para os diferentes modos de locomoção. Depois, foi desenvolvido um classificador baseado em ASMs. Posteriormente, um classificador de redes neuronais baseado em deep learning foi utilizado para classificar, de forma semelhante, os mesmos modos de locomoção. Deep learning é uma técnica bastante utilizada em classificação de dados. Estes classificadores foram implementados e comparados, fornecendo a uma ferramenta com uma boa precisão no reconhecimento dos modos de locomoção. Para implementar esta estratégia, era necessário realizar previamente um trabalho de suporte extremamente importante. Um sistema de unidades de medição inercial (IMUs), foi escolhido devido ao seu potencial extremo para monitorizar as atividades ambulatórias no ambiente domiciliar. Este sistema que combina sensores inerciais e magnéticos e é capaz de efetuar a monitorização de parâmetros da marcha em tempo real, foi validado e calibrado. Posteriormente, este Sistema foi usado para adquirir dados da marcha de indivíduos saudáveis que imitiram quedas. Os resultados mostraram que a precisão dos classificadores foi bastante aceitável e o classificador baseado em redes neuronais apresentou os melhores resultados com 92.71% de precisão. Como trabalho futuro, propõe-se a aplicação destas estratégias em tempo real de forma a evitar a ocorrência de quedas