DeepMoCap: Deep Optical Motion Capture Using Multiple Depth Sensors and Retro-Reflectors

In this paper, a marker-based, single-person optical motion capture method (DeepMoCap) is proposed using multiple spatio-temporally aligned infrared-depth sensors and retro-reflective straps and patches (reflectors). DeepMoCap explores motion capture by automatically localizing and labeling reflectors on depth images and, subsequently, on 3D space. Introducing a non-parametric representation to encode the temporal correlation among pairs of colorized depthmaps and 3D optical flow frames, a multi-stage Fully Convolutional Network (FCN) architecture is proposed to jointly learn reflector locations and their temporal dependency among sequential frames. The extracted reflector 2D locations are spatially mapped in 3D space, resulting in robust 3D optical data extraction. The subject’s motion is efficiently captured by applying a template-based fitting technique on the extracted optical data. Two datasets have been created and made publicly available for evaluation purposes; one comprising multi-view depth and 3D optical flow annotated images (DMC2.5D), and a second, consisting of spatio-temporally aligned multi-view depth images along with skeleton, inertial and ground truth MoCap data (DMC3D). The FCN model outperforms its competitors on the DMC2.5D dataset using 2D Percentage of Correct Keypoints (PCK) metric, while the motion capture outcome is evaluated against RGB-D and inertial data fusion approaches on DMC3D, outperforming the next best method by 4.5% in total 3D PCK accuracy

Effect on the development of computational thinking through a school technological activity that promotes a Mocap system

El pensamiento computacional (PC) representa una oportunidad de aprendizaje que involucra un conjunto de habilidades enfocadas en las prácticas de resolución de problemas. Por lo tanto, la implementación de actividades tecnológicas escolares (ATE) puede apoyar el desarrollo de elementos que caracterizan el PC, permitiendo a los estudiantes acercarse al conocimiento y al uso de tecnologías. Este estudio propone el diseño una ATE fundamentada en el aprendizaje a través de la construcción donde se plantean problemas complejos abordando estrategias exploradas en prácticas relacionadas con el PC, como: la progresión UMC (Usar-Modificar-Crear), las actividades desenchufadas y las actividades conectadas, con la intención de promover la simulación y la construcción de un dispositivo que incorpora un conjunto de sensores inerciales que registran el seguimiento de una extremidad del cuerpo humano, los datos obtenidos se envían a un entorno de visualización para animar un modelo digital en 3D. Todo el proceso es basado en los sistemas Mocap (Captura de Movimiento, o Motion Capture, en inglés). Estas acciones intencionadas buscan conocer el efecto en el desarrollo del pensamiento computacional desde el proceso cognitivo que comprende las habilidades: descomposición, generalización, abstracción, pensamiento algorítmico y evaluación.Computational Thinking (CT) represents a learning opportunity that involves a set of skills focused on problem-solving practices. Therefore, the implementation of school technology activities (ATE) can support the development of elements that characterize CT, allowing students to approach knowledge and the use of technologies. This study proposes the design of an ATE based on learning through construction where complex problems are posed by addressing strategies explored in practices related to PC, such as: the UMC progression (Use-Modify-Create), unplugged activities and activities connected, with the intention of promoting the simulation and construction of a device that incorporates a set of inertial sensors that record the tracking of an extremity of the human body, the data obtained is sent to a visualization environment to animate a 3D digital model . The entire process is based on Mocap systems (Motion Capture, or Motion Capture, in English). These intentional actions seek to know the effect on the development of computational thinking from the cognitive process that includes the skills: decomposition, generalization, abstraction, algorithmic thinking and evaluation

Ergowear: desenvolvimento de um vestuário inteligente para monitorização postural e biofeedback

Dissertação de mestrado em Engenharia Biomédica (especialização em Eletrónica Médica)Atualmente, as Lesões Musculoesqueléticas Relacionadas com o Trabalho (LMERT) são considera das o ”problema relacionado com o trabalho mais prevalente”na União Europeia, levando a um custo estimado de cerca de 240 biliões de euros. Em casos mais severos, estes distúrbios podem causar danos vitalícios à saúde do trabalhador, reduzindo a sua qualidade de vida. De facto, LMERTs são con sideradas a principal causa da reforma precoce dos trabalhadores. Foi reportado que os segmentos da parte superior do corpo são mais suceptíveis ao desenvolvimento de LMERTs. Para mitigar a prevalência de LMERTs, ergonomistas maioritariamente aplicam métodos de avaliação observacionais, que são alta mente dependentes da experiência do analista, e apresentam baixa objetividade e repetibilidade. Desta maneira, esforços têm sido feitos para desenvolver ferramentas de avaliação ergonómica baseadas na instrumentação, para compensar essas limitações. Além disso, com a ascensão do conceito da indústria 5.0, o trabalhador humano volta a ser o foco principal na indústria, juntamente com o robô colaborativo. No entanto, para alcançar uma relação verdadeiramente colaborativa e simbiótica entre o trabalhador e o robô, este último precisa de reconhecer as intenções do trabalhador. Para superar este obstáculo, sis temas de captura de movimento podem ser integrados nesta estrutura, fornecendo dados de movimento ao robô colaborativo. Esta dissertação visa a melhoria de um sistema de captura de movimento autónomo, da parte supe rior do corpo, de abordagem inercial que servirá, não apenas para monitorizar a postura do trabalhador, mas também avaliar a ergonomia do usuário e fornecer consciencialização postural ao usuário, por meio de motores de biofeedback. Além disso, o sistema foi já idealizado tendo em mente a sua integração numa estrutura colaborativa humano-robô. Para atingir estes objetivos, foi aplicada uma metodologia de design centrado no utilizador, começando pela análise do Estado da Arte, a avaliação das limitações do sistema anterior, a definição dos requisitos do sistema, o desenvolvimento da peça de vestuário, arquite tura do hardware e arquitetura do software do sistema. Por fim, o sistema foi validado para verificar se estava em conformidade com os requisitos especificados. O sistema é composto por 9 Unidades de Medição Inercial (UMI), posicionados na parte inferior e superior das costas, cabeça, braços, antebraços e mãos. Também foi integrado um sistema de atuação, para biofeedback postural, composto por 6 motores vibrotáteis, localizados na região lombar e próximo do pescoço, cotovelos e pulsos. O sistema é alimentado por uma powerbank e todos os dados adquiridos são enviados para uma estação de processamento, via WiFi (User Datagram Protocol (UDP)), garantindo autonomia. O sistema tem integrado um filtro de fusão Complementar Extendido e uma sequência de calibração Sensor-para-Segmento estática, de maneira a aumentar a precisão da estimativa dos ângulos das articulações. Além disso, o sistema é capaz de amostrar os dados angulares a 240 Hz, enquanto que o sistema anterior era capaz de amostrar no máximo a 100 Hz, melhorando a resolução da aquisição dos dados. O sistema foi validado em termos de hardware e usabilidade. Os testes de hardware abordaram a caracterização da autonomia, frequência de amostragem, robustez mecânica e desempenho da comuni cação sem fio do sistema, em diversos contextos, e também para verificar se estes estão em conformidade com os requisitos técnicos previamente definidos, que foi o caso. Adicionalmente, as especificações da nova versão do sistema foram comparadas com a anterior, onde se observou uma melhoria direta signifi cativa, como por exemplo, maior frequência de amostragem, menor perda de pacote, menor consumo de corrente, entre outras, e com sistemas comerciais de referência (XSens Link). Testes de usabilidade foram realizados com 9 participantes que realizaram vários movimentos uniarticulares e complexos. Após os testes, os usuários responderam a um questionário baseado na Escala de Usabilidade do Sistema (EUS). O sistema foi bem aceite pelos os usuários, em termos de estética e conforto, em geral, comprovando um elevado nível de vestibilidade.Nowadays, Work-Related Musculoskeletal Disorders (WRMSDs) are considered the ”most prevalent work-related problem” in the European Union (EU), leading to an estimated cost of about 240 billion EUR. In more severe cases, these disorders can cause life-long impairments to the workers’ health, reducing their quality of life. In fact, WRMSDs are the main cause for the workers’ early retirement. It was reported that the upper body segments of the worker are more susceptible to the development of WRMSDs. To mitigate the prevalence of WRMSD, ergonomists mostly apply observational assessment methods, which are highly dependant on the analyst’s expertise, have low objectivity and repeatability. Therefore, efforts have been made to develop instrumented-based ergonomic assessment tools, to compensate for these limitations. Moreover, with the rise of the 5.0 industry concept, the human worker is once again the main focus in the industry, along with the Collaborative Robot (cobot). However, to achieve a truly collaborative relation between the worker and the cobot, the latter needs to know the worker’s intentions. To surpass this obstacle, Motion Capture (MoCap) systems can be integrated in this framework, providing motion data to the cobot. This dissertation aims at the improvement of a stand-alone, upper-body, inertial, MoCap system, that will serve to not only monitor the worker’s posture, but also to assess the user’s ergonomics and provide posture awareness to the user, through biofeedback motors. Furthermore, it was also designed to integrate a human-robot collaborative framework. To achieve this, a user-centred design methodology was applied, starting with analyzing the State of Art (SOA), assessing the limitations of the previous system, defining the system’s requirements, developing the garment, hardware architecture and software architecture of the system. Lastly, the system was validated to ascertain if it is in conformity with the specified requirements. The developed system is composed of 9 Inertial Measurement Units (IMUs), placed on the lower and upper back, head, upper arms, forearms and hands. An actuation system was also integrated, for postural biofeedback, and it is comprised of 6 vibrotactile motors, located in the lower back, and in close proximity to the neck, elbows and wrists. The system is powered by a powerbank and all of the acquired data is sent to a main station, via WiFi (UDP), granting a standalone characteristic. The system integrates an Extended Complementary Filter (ECF) and a static Sensor-to-Segment (STS) calibration sequence to increase the joint angle estimation accuracy. Furthermore, the system is able to sample the angular data at 240 Hz, while the previous system was able to sample it at a maximum 100 Hz, improving the resolution of the data acquisition. The system was validated in terms of hardware and usability. The hardware tests addressed the char acterization of the system’s autonomy, sampling frequency, mechanical robustness and wireless commu nication performance in different contexts, and ascertain if they comply with the technical requirements, which was the case. Moreover, the specifications of the new version were compared with the previous one, where a significant direct improvement was observed, such as, higher sampling frequency, lower packet loss, lower current consumption, among others, and with a commercial system of reference (XSens Link). Usability tests were carried out with 9 participants who performed several uni-joint and complex motions. After testing, users answered a questionnaire based on the System Usability Scale (SUS). The system was very well accepted by the participants, regarding aesthetics and overall comfort, proving to have a high level of wearability

Комп’ютерно-інтегрована інерціальна система захоплення руху людини

Актуальність на сьогоднішній день актуальною лишається проблема використання точних систем захоплення руху людини (СЗР), що не залежать від умов проведення експе-рименту. Використання інерціальних систем захоплення руху дозволяє розширити діапазон використання таких систем. Важливою проблемою є розробка завадостійкого алгоритму оці-нки орієнтації та методи підвищення точності. Магістерська дисертація виконана відповідно до основних напрямків наукових дослі-джень кафедри. Мета магістерської дисертації є підвищення завадостійкості та точності інерціальної системи захоплення руху. Завдання: 1. Огляд систем захоплення руху, опис основних принципів, коротка характерис-тика. 2. Аналіз алгоритмів орієнтації, що використовують інерціальні датчики. 3. Вибір базового алгоритму орієнтації, опис його мат. моделі. 4. Розробка алгоритму підвищення завадостійкості системи на основі сигналів гі-роскопів, акселерометрів, магнітометрів. 5. Калібрування датчиків, експериментальна перевірка розроблених макетів. 6. Розробка власного макету інерціальної системи захоплення руху. Об’єкт: процес захоплення руху людини та система захоплення руху людини Предмет: Підвищення точності та завадостійкості алгоритмів оцінки орієнтації об’єктів та системи захоплення руху людини, калібрування та врахування особливостей ма-тематичної моделі датчиків інерціального вимірювального блоку. Методи дослідження: Методи оптимальної обробки сигналів, методи чисельної ма-тематики, методи теоретичної механіки, методи теорії оптимального оцінювання, алгоритми фільтрації Наукова новизна: 1. Вперше запропоновано метод розділення каналів корекції кватерніонного до-повняльного фільтру оцінки орієнтації, у алгоритмі якого мінімізовано кількість обчислюва-льних операцій. 5 2. Запропоновано використання слідкуючого П-регулятора для обмеження гли-бини корекції. Практичне значення: розроблено алгоритмічне забезпечення із підвищеною завадо-стійкістю для системи захоплення руху, яке також може використовуватися для систем ста-білізації та орієнтації; Створено платформу для тестування нового алгоритмічного забезпе-чення. Апробація результатів дисертації (виступи на науково технічних конференціях): 1. ХХ Міжнародна молодіжна науково-практична конференція «Людина і кос-мос», Дніпро, 2018 2. Міжнародна науково-технічна XI конференція молодих вчених «Електроніка - 2018», Київ, 2018 3. Науково-практична конференція студентів та аспірантів «Погляд у майбутнє приладобудування», Київ, 2018The urgency to date is the problem of the use of precise human motion capture system (MoCap), which do not depend on the conditions of the experiment. The use of inertial motion cap-ture systems allows to extend the range of use of such systems. An important problem is the devel-opment of a perturbation-tolerant algorithm and methods of improving accuracy for estimating ori-entation. The master's thesis is executed in accordance with the main directions of scientific research of the department. The purpose of the master's thesis is to increase the perturbation immunity and accuracy of the inertial motion capture system. Task: 1. Overview of the motion capture systems, describe basic principles, describe their charac-teristics. 2. Analysis of orientation algorithms using inertial sensors. 3. Select the basic orientation algorithm, description of its mat. model. 4. Development of the algorithm for increasing noise immunity of the system based on the signals of gyroscopes, accelerometers, magnetometers. 5. Instrument calibration, experimental check of the developed models. 6. Development of own motion capture system. Object: the process of capturing the movement of a person and MoCap system. Subject: Improvement of precision and algorithms perturbation immunity for estimating the orientation of objects and for MoCap system, calibration and taking into account the features of the mathematical model of sensors of an inertial measuring unit. Methods of research: Methods of optimal signal processing, numerical methods, methods of theoretical mechanics, methods of theory of optimal evaluation, filtration algorithms Scientific novelty: 1. For the first time, the method of separating channels of correction of a quaternion-to-full-blown filter of orientation evaluation is proposed. 2. The use of a follower of the P-regulator for limiting globin correction is proposed. Practical significance: algorithmic support with increased noise immunity for the motion capture system, which can also be used for stabilization and orientation systems, is developed; A platform for testing new algorithms is created

Low-Cost Sensors and Biological Signals

Many sensors are currently available at prices lower than USD 100 and cover a wide range of biological signals: motion, muscle activity, heart rate, etc. Such low-cost sensors have metrological features allowing them to be used in everyday life and clinical applications, where gold-standard material is both too expensive and time-consuming to be used. The selected papers present current applications of low-cost sensors in domains such as physiotherapy, rehabilitation, and affective technologies. The results cover various aspects of low-cost sensor technology from hardware design to software optimization

Inertial Motion Capture Costume Design Study

The paper describes a scalable, wearable multi-sensor system for motion capture based on inertial measurement units (IMUs). Such a unit is composed of accelerometer, gyroscope and magnetometer. The final quality of an obtained motion arises from all the individual parts of the described system. The proposed system is a sequence of the following stages: sensor data acquisition, sensor orientation estimation, system calibration, pose estimation and data visualisation. The construction of the system’s architecture with the dataflow programming paradigm makes it easy to add, remove and replace the data processing steps. The modular architecture of the system allows an effortless introduction of a new sensor orientation estimation algorithms. The original contribution of the paper is the design study of the individual components used in the motion capture system. The two key steps of the system design are explored in this paper: the evaluation of sensors and algorithms for the orientation estimation. The three chosen algorithms have been implemented and investigated as part of the experiment. Due to the fact that the selection of the sensor has a significant impact on the final result, the sensor evaluation process is also explained and tested. The experimental results confirmed that the choice of sensor and orientation estimation algorithm affect the quality of the final results