인간공학적 자세 평가를 위한 비디오 기반의 작업 자세 입력 시스템 개발

학위논문(석사) -- 서울대학교대학원 : 공과대학 산업공학과, 2022. 8. 윤명환.Work-related musculoskeletal disorders are a crucial problem for the worker’s safety and productivity of the workplace. The purpose of this study is to propose and develop a video-based work pose entry system for ergonomic postural assessment methods, Rapid Upper Limb Assessment(RULA) and Rapid Entire Body Assessment(REBA). This study developed a work pose entry system using the YOLOv3 algorithm for human tracking and the SPIN approach for 3D human pose estimation. The work pose entry system takes in a 2D video and scores of few evaluation items as input and outputs a final RULA or REBA score and the corresponding action level. An experiment for validation was conducted to 20 evaluators which were classified into two groups, experienced and novice, based on their level of knowledge or experience on ergonomics and musculoskeletal disorders. Participants were asked to manually evaluate working postures of 20 working videos taken at an automobile assembly plant, recording their scores on an Excel worksheet. Scores were generated by the work pose entry system based on individual items that need to be inputted, and the results of manual evaluation and results from the work pose entry system were compared. Descriptive statistics and Mann-Whitney U test showed that using the proposed work pose entry system decreased the difference and standard deviation between the groups. Also, findings showed that experienced evaluators tend to score higher than novice evaluators. Fisher’s exact test was also conducted on evaluation items that are inputted into the work pose entry system, and results have shown that some items that may seem apparent can be perceived differently between groups as well. The work pose entry system developed in this study can contribute to increasing consistency of ergonomic risk assessment and reducing time and effort of ergonomic practitioners during the process. Directions for future research on developing work pose entry systems for ergonomic posture assessment using computer vision are also suggested in the current study.작업 관련 근골격계 질환은 근로자의 안전과 작업장의 생산성 향상에 중요한 문제다. 본 연구의 목적은 인간공학적 자세 분석에 사용되는 대표적인 방법인 Rapid Upper Limb Assessment(RULA) 및 Rapid Entire Body Assessment(REBA)를 위한 비디오 기반의 작업 자세 입력 시스템을 제안하는 것이다. 본 연구는 영상 내 사람 탐지 및 추적을 위한 YOLOv3 알고리즘과 3차원 사람 자세 추정을 위한 SPIN 접근법을 사용하는 시스템을 개발했다. 해당 작업 자세 입력 시스템은 2차원 영상과 몇 개의 평가 항목 점수를 입력으로 받아 최종 RULA 또는 REBA 점수와 해당 조치수준(Action level)을 출력한다. 본 연구에서 제안하는 작업 자세 입력 시스템이 일관적인 결과를 산출하는지 알아보기 위해 인간공학 및 근골격계 질환에 대한 지식이나 경험을 기준으로 숙련된 평가자와 초보 평가자의 두 그룹으로 분류된 평가자 20명을 대상으로 검증 실험을 진행했다. 참가자들은 국내 자동차 조립 공장에서 찍은 20개의 작업 영상의 작업 자세를 수동으로 평가하여 Excel 워크시트에 점수를 기록하였다. 시스템 사용 시 입력해야 하는 개별 항목을 기준으로 시스템을 통한 점수를 생성하고 기존의 전통적인 방법으로 평가한 결과와 시스템에서 얻은 결과를 비교하였으며, 기술 통계와 Mann-Whitney U test는 제안된 시스템을 사용하면 그룹 간의 차이와 표준편차가 감소한다는 것을 보여주었다. 또한, 경험이 많은 평가자들이 초보 평가자들보다 더 높은 점수를 받는 경향이 있다는 것을 보여주었다. 시스템에 입력되는 평가 항목과 경험 정도와의 관계를 확인하기 위해 Fisher’s exact test를 수행하였으며, 결과는 명백해 보일 수 있는 일부 항목도 그룹 간에 다르게 인식될 수 있음을 보여주었다. 이 도구에서 개발된 작업 자세 입력 시스템은 인간공학적 자세 평가의 일관성을 높이고 평가 과정 중 중에 인간공학적 평가자의 시간과 노력을 줄이는 데 기여할 수 있다. 또한 컴퓨터 비전을 활용한 인간공학적 자세 평가를 위한 작업 자세 입력 시스템 개발에 대한 향후 연구 방향도 이번 연구에서 제시된다.Chapter 1 Introduction 1 1.1 Background 1 1.2 Research Objectives 4 1.3 Organization of the Thesis 5 Chapter 2 Literature Review 6 2.1 Overview 6 2.2 Work-related Musculoskeletal Disorders 6 2.3 Ergonomic Posture Analysis 7 2.3.1 Self-reports 7 2.3.2 Observational Methods 7 2.3.3 Direct Methods 15 2.3.4 Vision-based Methods 17 2.4 3D Human Pose Estimation 19 2.4.1 Model-free Approaches 20 2.4.2 Model-based Approaches 21 Chapter 3 Proposed System Design 23 3.1 Overview 23 3.2 Human Tracking 24 3.3 3D Human Pose Estimation 24 3.4 Score Calculation 26 3.4.1 Posture Score Calculation 26 3.4.2 Output of the Proposed System 31 Chapter 4 Validation Experiment 32 4.1 Hypotheses 32 4.2 Methods 32 4.2.1 Participants 32 4.2.2 Apparatus 33 4.2.3 Procedure 33 4.2.4 Data Analysis 37 4.3 Results 38 4.3.1 RULA 38 4.3.2 REBA 46 4.3.3 Evaluation Items for Manual Input 54 Chapter 5 Discussion 56 5.1 Group Difference 56 5.1.1 RULA 57 5.1.2 REBA 57 5.2 Evaluation Items for Manual Input 58 5.3 Proposed Work Pose Entry System 59 Chapter 6 Conclusion 62 6.1 Conclusion 62 6.2 Limitation, Contribution, and Future Direction 62 Bibliography 65 국문초록 77석

The development of fully automated RULA assessment system based on Computer Vision

The purpose of this study was to develop an automated, RULA-based posture assessment system using a deep learning algorithm to estimate RULA scores, including scores for wrist posture, based on images of workplace postures. The proposed posture estimation system reported a mean absolute error (MAE) of 2.86 on the validation dataset obtained by randomly splitting 20% of the original training dataset before data augmentation. The results of the proposed system were compared with those of two experts’ manual evaluation by computing the Intraclass correlation coefficient (ICC), which yielded index values greater than 0.75, thereby confirming good agreement between manual raters and the proposed system. This system will reduce the time required for postural evaluation while producing highly reliable RULA scores that are consistent with those generated by manual approach. Thus, we expect that this study will aid ergonomic experts in conducting RULA-based surveys of occupational postures in workplace conditions

UPPER LIMB JOINTS AND MOTIONS SAMPLING SYSTEM USING KINECT CAMERA

The needs of research on human posture and its joint-motion relationships are important. Providing a real-time postural measurement tool has attracted the attention of many human postural-related researchers. This study has developed and performed a validation analysis on a new innovative system for sampling and finding the angles of motions of each posture with its related joints using Kinect camera. The validation investigated the static and dynamic accuracy analyses by comparing to a Jamar goniometer and ErgoFellow system. The results showed that Mean Absolute Errors of Kinect in static and dynamic motions are 15.12% and 45.33% respectively. It is concluded that the postural measurement system developed by this study requires further improvements

New deep learning approaches to domain adaptation and their applications in 3D hand pose estimation

This study investigates several methods for using artificial intelligence to give machines the ability to see. It introduced several methods for image recognition that are more accurate and efficient compared to the existing approaches

A Review on the Use of Microsoft Kinect for Gait Abnormality and Postural Disorder Assessment

Review ArticleCopyright © 2021 Anthony Bawa et al. Gait and posture studies have gained much prominence among researchers and have attracted the interest of clinicians. The ability to detect gait abnormality and posture disorder plays a crucial role in the diagnosis and treatment of some diseases. Microsoft Kinect is presented as a noninvasive sensor essential for medical diagnostic and therapeutic purposes. There are currently no relevant studies that attempt to summarise the existing literature on gait and posture abnormalities using Kinect technology. The purpose of this study is to critically evaluate the existing research on gait and posture abnormalities using the Kinect sensor as the main diagnostic tool. Our studies search identified 458 for gait abnormality, 283 for posture disorder of which 26 studies were included for gait abnormality, and 13 for posture. The results indicate that Kinect sensor is a useful tool for the assessment of kinematic features. In conclusion, Microsoft Kinect sensor is presented as a useful tool for gait abnormality, postural disorder analysis, and physiotherapy. It can also help track the progress of patients who are undergoing rehabilitation

A Control and Posture Recognition Strategy for Upper-Limb Rehabilitation of Stroke Patients

At present, the study of upper-limb posture recognition is still in the primary stage; due to the diversity of the objective environment and the complexity of the human body posture, the upper-limb posture has no public dataset. In this paper, an upper extremity data acquisition system is designed, with a three-channel data acquisition mode, collect acceleration signal, and gyroscope signal as sample data. The datasets were preprocessed with deweighting, interpolation, and feature extraction. With the goal of recognizing human posture, experiments with KNN, logistic regression, and random gradient descent algorithms were conducted. In order to verify the superiority of each algorithm, the data window was adjusted to compare the recognition speed, computation time, and accuracy of each classifier. For the problem of improving the accuracy of human posture recognition, a neural network model based on full connectivity is developed. In addition, this paper proposes a finite state machine- (FSM-) based FES control model for controlling the upper limb to perform a range of functional tasks. In the process of constructing the network model, the effects of different hidden layers, activation functions, and optimizers on the recognition rate were experimental for the comparative analysis; the softplus activation function with better recognition performance and the adagrad optimizer are selected. Finally, by comparing the comprehensive recognition accuracy and time efficiency with other classification models, the fully connected neural network is verified in the human posture superiority in identification

Estimation de posture 3D à partir de données imprécises et incomplètes : application à l'analyse d'activité d'opérateurs humains dans un centre de tri

Dans un contexte d’étude de la pénibilité et de l’ergonomie au travail pour la prévention des troubles musculo-squelettiques, la société Ebhys cherche à développer un outil d’analyse de l’activité des opérateurs humains dans un centre de tri, par l’évaluation d’indicateurs ergonomiques. Pour faire face à l’environnement non contrôlé du centre de tri et pour faciliter l’acceptabilité du dispositif, ces indicateurs sont mesurés à partir d’images de profondeur. Une étude ergonomique nous permet de définir les indicateurs à mesurer. Ces indicateurs sont les zones d’évolution des mains de l’opérateur et d’angulations de certaines articulations du haut du corps. Ce sont donc des indicateurs obtenables à partir d’une analyse de la posture 3D de l’opérateur. Le dispositif de calcul des indicateurs sera donc composé de trois parties : une première partie sépare l’opérateur du reste de la scène pour faciliter l’estimation de posture 3D, une seconde partie calcule la posture 3D de l’opérateur, et la troisième utilise la posture 3D de l’opérateur pour calculer les indicateurs ergonomiques. Tout d’abord, nous proposons un algorithme qui permet d’extraire l’opérateur du reste de l’image de profondeur. Pour ce faire, nous utilisons une première segmentation automatique basée sur la suppression du fond statique et la sélection d’un objet dynamique à l’aide de sa position et de sa taille. Cette première segmentation sert à entraîner un algorithme d’apprentissage qui améliore les résultats obtenus. Cet algorithme d’apprentissage est entraîné à l’aide des segmentations calculées précédemment, dont on sélectionne automatiquement les échantillons de meilleure qualité au cours de l’entraînement. Ensuite, nous construisons un modèle de réseau de neurones pour l’estimation de la posture 3D de l’opérateur. Nous proposons une étude qui permet de trouver un modèle léger et optimal pour l’estimation de posture 3D sur des images de profondeur de synthèse, que nous générons numériquement. Finalement, comme ce modèle n’est pas directement applicable sur les images de profondeur acquises dans les centres de tri, nous construisons un module qui permet de transformer les images de profondeur de synthèse en images de profondeur plus réalistes. Ces images de profondeur plus réalistes sont utilisées pour réentrainer l’algorithme d’estimation de posture 3D, pour finalement obtenir une estimation de posture 3D convaincante sur les images de profondeur acquises en conditions réelles, permettant ainsi de calculer les indicateurs ergonomique

Towards a multisensor station for automated biodiversity monitoring

Rapid changes of the biosphere observed in recent years are caused by both small and large scale drivers, like shifts in temperature, transformations in land-use, or changes in the energy budget of systems. While the latter processes are easily quantifiable, documentation of the loss of biodiversity and community structure is more difficult. Changes in organismal abundance and diversity are barely documented. Censuses of species are usually fragmentary and inferred by often spatially, temporally and ecologically unsatisfactory simple species lists for individual study sites. Thus, detrimental global processes and their drivers often remain unrevealed. A major impediment to monitoring species diversity is the lack of human taxonomic expertise that is implicitly required for large-scale and fine-grained assessments. Another is the large amount of personnel and associated costs needed to cover large scales, or the inaccessibility of remote but nonetheless affected areas. To overcome these limitations we propose a network of Automated Multisensor stations for Monitoring of species Diversity (AMMODs) to pave the way for a new generation of biodiversity assessment centers. This network combines cutting-edge technologies with biodiversity informatics and expert systems that conserve expert knowledge. Each AMMOD station combines autonomous samplers for insects, pollen and spores, audio recorders for vocalizing animals, sensors for volatile organic compounds emitted by plants (pVOCs) and camera traps for mammals and small invertebrates. AMMODs are largely self-containing and have the ability to pre-process data (e.g. for noise filtering) prior to transmission to receiver stations for storage, integration and analyses. Installation on sites that are difficult to access require a sophisticated and challenging system design with optimum balance between power requirements, bandwidth for data transmission, required service, and operation under all environmental conditions for years. An important prerequisite for automated species identification are databases of DNA barcodes, animal sounds, for pVOCs, and images used as training data for automated species identification. AMMOD stations thus become a key component to advance the field of biodiversity monitoring for research and policy by delivering biodiversity data at an unprecedented spatial and temporal resolution. (C) 2022 Published by Elsevier GmbH on behalf of Gesellschaft fur Okologie