Human Motion Analysis with Wearable Inertial Sensors

High-resolution, quantitative data obtained by a human motion capture system can be used to better understand the cause of many diseases for effective treatments. Talking about the daily care of the aging population, two issues are critical. One is to continuously track motions and position of aging people when they are at home, inside a building or in the unknown environment; the other is to monitor their health status in real time when they are in the free-living environment. Continuous monitoring of human movement in their natural living environment potentially provide more valuable feedback than these in laboratory settings. However, it has been extremely challenging to go beyond laboratory and obtain accurate measurements of human physical activity in free-living environments. Commercial motion capture systems produce excellent in-studio capture and reconstructions, but offer no comparable solution for acquisition in everyday environments. Therefore in this dissertation, a wearable human motion analysis system is developed for continuously tracking human motions, monitoring health status, positioning human location and recording the itinerary. In this dissertation, two systems are developed for seeking aforementioned two goals: tracking human body motions and positioning a human. Firstly, an inertial-based human body motion tracking system with our developed inertial measurement unit (IMU) is introduced. By arbitrarily attaching a wearable IMU to each segment, segment motions can be measured and translated into inertial data by IMUs. A human model can be reconstructed in real time based on the inertial data by applying high efficient twists and exponential maps techniques. Secondly, for validating the feasibility of developed tracking system in the practical application, model-based quantification approaches for resting tremor and lower extremity bradykinesia in Parkinson’s disease are proposed. By estimating all involved joint angles in PD symptoms based on reconstructed human model, angle characteristics with corresponding medical ratings are employed for training a HMM classifier for quantification. Besides, a pedestrian positioning system is developed for tracking user’s itinerary and positioning in the global frame. Corresponding tests have been carried out to assess the performance of each system

Upper Limb Position Tracking with a Single Inertial Sensor Using Dead Reckoning Method with Drift Correction Techniques

Inertial sensors are widely used in human motion monitoring. Orientation and position are the two most widely used measurements for motion monitoring. Tracking with the use of multiple inertial sensors is based on kinematic modelling which achieves a good level of accuracy when biomechanical constraints are applied. More recently, there is growing interest in tracking motion with a single inertial sensor to simplify the measurement system. The dead reckoning method is commonly used for estimating position from inertial sensors. However, significant errors are generated after applying the dead reckoning method because of the presence of sensor offsets and drift. These errors limit the feasibility of monitoring upper limb motion via a single inertial sensing system. In this paper, error correction methods are evaluated to investigate the feasibility of using a single sensor to track the movement of one upper limb segment. These include zero velocity update, wavelet analysis and high-pass filtering. The experiments were carried out using the nine-hole peg test. The results show that zero velocity update is the most effective method to correct the drift from the dead reckoning-based position tracking. If this method is used, then the use of a single inertial sensor to track the movement of a single limb segment is feasible

Enhanced Indoor Localization System based on Inertial Navigation

An algorithm for indoor localization of pedestrians using an improved Inertial Navigation system is presented for smartphone based applications. When using standard inertial navigation algorithm, errors in sensors due to random noise and bias result in a large drift from the actual location with time. Novel corrections are introduced for the basic system to increase the accuracy by counteracting the accumulation of this drift error, which are applied using a Kalman filter framework. A generalized velocity model was applied to correct the walking velocity and the accuracy of the algorithm was investigated with three different velocity models which were derived from the actual velocity measured at the hip of walking person. Spatial constraints based on knowledge of indoor environment were applied to correct the walking direction. Analysis of absolute heading corrections from magnetic direction was performed . Results show that the proposed method with Gaussian velocity model achieves competitive accuracy with a 30\% less variance over Step and Heading approach proving the accuracy and robustness of proposed method. We also investigated the frequency of applying corrections and found that a 4\% corrections per step is required for improved accuracy. The proposed method is applicable in indoor localization and tracking applications based on smart phone where traditional approaches such as GNSS suffers from many issues

Mixed Reality Browsers and Pedestrian Navigation in Augmented Cities

International audienceIn this paper, We use a declarative format for positional audio with synchronization between audio chunks using SMIL. This format has been specifically designed for the type of audio used in AR applications. The audio engine associated to this format is running on mobile platforms (iOS, Android). Our MRB browser called IXE use a format based on volunteered geographic information (OpenStreetMap) and OSM documents for IXE can be fully authored in side OSM editors like JOSM. This is in contrast with the other AR browsers like Layar, Juniao, Wikitude, which use a Point of Interest (POI) based format having no notion of ways. This introduces a fundamental difference and in some senses a duality relation between IXE and the other AR browsers. In IXE, Augmented Virtuality (AV) navigation along a route (composed of ways) is central and AR interaction with objects is delegated to associate 3D activities. In AR browsers, navigation along a route is delegated to associated map activities and AR interaction with objects is central. IXE supports multiple tracking technologies and therefore allows both indoor navigation in buildings and outdoor navigation at the level of sidewalks. A first android version of the IXE browser will be released at the end of 2013. Being based on volunteered geographic it will allow building accessible pedestrian networks in augmented cities

적분 및 매개변수 기법 융합을 이용한 스마트폰 다중 동작에서 보행 항법

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 기계항공공학부, 2020. 8. 박찬국.In this dissertation, an IA-PA fusion-based PDR (Pedestrian Dead Reckoning) using low-cost inertial sensors is proposed to improve the indoor position estimation. Specifically, an IA (Integration Approach)-based PDR algorithm combined with measurements from PA (Parametric Approach) is constructed so that the algorithm is operated even in various poses that occur when a pedestrian moves with a smartphone indoors. In addition, I propose an algorithm that estimates the device attitude robustly in a disturbing situation by an ellipsoidal method. In addition, by using the machine learning-based pose recognition, it is possible to improve the position estimation performance by varying the measurement update according to the poses. First, I propose an adaptive attitude estimation based on ellipsoid technique to accurately estimate the direction of movement of a smartphone device. The AHRS (Attitude and Heading Reference System) uses an accelerometer and a magnetometer as measurements to calculate the attitude based on the gyro and to compensate for drift caused by gyro sensor errors. In general, the attitude estimation performance is poor in acceleration and geomagnetic disturbance situations, but in order to effectively improve the estimation performance, this dissertation proposes an ellipsoid-based adaptive attitude estimation technique. When a measurement disturbance comes in, it is possible to update the measurement more accurately than the adaptive estimation technique without considering the direction by adjusting the measurement covariance with the ellipsoid method considering the direction of the disturbance. In particular, when the disturbance only comes in one axis, the proposed algorithm can use the measurement partly by updating the other two axes considering the direction. The proposed algorithm shows its effectiveness in attitude estimation under disturbances through the rate table and motion capture equipment. Next, I propose a PDR algorithm that integrates IA and PA that can be operated in various poses. When moving indoors using a smartphone, there are many degrees of freedom, so various poses such as making a phone call, texting, and putting a pants pocket are possible. In the existing smartphone-based positioning algorithms, the position is estimated based on the PA, which can be used only when the pedestrian's walking direction and the device's direction coincide, and if it does not, the position error due to the mismatch in angle is large. In order to solve this problem, this dissertation proposes an algorithm that constructs state variables based on the IA and uses the position vector from the PA as a measurement. If the walking direction and the device heading do not match based on the pose recognized through machine learning technique, the position is updated in consideration of the direction calculated using PCA (Principal Component Analysis) and the step length obtained through the PA. It can be operated robustly even in various poses that occur. Through experiments considering various operating conditions and paths, it is confirmed that the proposed method stably estimates the position and improves performance even in various indoor environments.본 논문에서는 저가형 관성센서를 이용한 보행항법시스템 (PDR: Pedestrian Dead Reckoning)의 성능 향상 알고리즘을 제안한다. 구체적으로 보행자가 실내에서 스마트폰을 들고 이동할 때 발생하는 다양한 동작 상황에서도 운용될 수 있도록, 매개변수 기반 측정치를 사용하는 적분 기반의 보행자 항법 알고리즘을 구성한다. 또한 타원체 기반 자세 추정 알고리즘을 구성하여 외란 상황에서도 강인하게 자세를 추정하는 알고리즘을 제안한다. 추가적으로 기계학습 기반의 동작 인식 정보를 이용, 동작에 따른 측정치 업데이트를 달리함으로써 위치 추정 성능을 향상시킨다. 먼저 스마트폰 기기의 이동 방향을 정확하게 추정하기 위해 타원체 기법 기반 적응 자세 추정을 제안한다. 자세 추정 기법 (AHRS: Attitude and Heading Reference System)은 자이로를 기반으로 자세를 계산하고 자이로 센서오차에 의해 발생하는 드리프트를 보정하기 위해 측정치로 가속도계와 지자계를 사용한다. 일반적으로 가속 및 지자계 외란 상황에서는 자세 추정 성능이 떨어지는데, 추정 성능을 효과적으로 향상시키기 위해 본 논문에서는 타원체 기반 적응 자세 추정 기법을 제안한다. 측정치 외란이 들어오는 경우, 외란의 방향을 고려하여 타원체 기법으로 측정치 공분산을 조정해줌으로써 방향을 고려하지 않은 적응 추정 기법보다 정확하게 측정치 업데이트를 할 수 있다. 특히 외란이 한 축으로만 들어오는 경우, 제안한 알고리즘은 방향을 고려해 나머지 두 축에 대해서는 업데이트 해줌으로써 측정치를 부분적으로 사용할 수 있다. 레이트 테이블, 모션 캡쳐 장비를 통해 제안한 알고리즘의 자세 성능이 향상됨을 확인하였다. 다음으로 다양한 동작에서도 운용 가능한 적분 및 매개변수 기법을 융합하는 보행항법 알고리즘을 제안한다. 스마트폰을 이용해 실내를 이동할 때에는 자유도가 크기 때문에 전화 걸기, 문자, 바지 주머니 넣기 등 다양한 동작이 발생 가능하다. 기존의 스마트폰 기반 보행 항법에서는 매개변수 기법을 기반으로 위치를 추정하는데, 이는 보행자의 진행 방향과 기기의 방향이 일치하는 경우에만 사용 가능하며 일치하지 않는 경우 자세 오차로 인한 위치 오차가 크게 발생한다. 이러한 문제를 해결하기 위해 본 논문에서는 적분 기반 기법을 기반으로 상태변수를 구성하고 매개변수 기법을 통해 나오는 위치 벡터를 측정치로 사용하는 알고리즘을 제안한다. 만약 기계학습을 통해 인식한 동작을 바탕으로 진행 방향과 기기 방향이 일치하지 않는 경우, 주성분 분석을 통해 계산한 진행방향을 이용해 진행 방향을, 매개변수 기법을 통해 얻은 보폭으로 거리를 업데이트해 줌으로써 보행 중 발생하는 여러 동작에서도 강인하게 운용할 수 있다. 다양한 동작 상황 및 경로를 고려한 실험을 통해 위에서 제안한 방법이 다양한 실내 환경에서도 안정적으로 위치를 추정하고 성능이 향상됨을 확인하였다.Chapter 1 Introduction 1 1.1 Motivation and Background 1 1.2 Objectives and Contribution 5 1.3 Organization of the Dissertation 6 Chapter 2 Pedestrian Dead Reckoning System 8 2.1 Overview of Pedestrian Dead Reckoning 8 2.2 Parametric Approach 9 2.2.1 Step detection algorithm 11 2.2.2 Step length estimation algorithm 13 2.2.3 Heading estimation 14 2.3 Integration Approach 15 2.3.1 Extended Kalman filter 16 2.3.2 INS-EKF-ZUPT 19 2.4 Activity Recognition using Machine Learning 21 2.4.1 Challenges in HAR 21 2.4.2 Activity recognition chain 22 Chapter 3 Attitude Estimation in Smartphone 26 3.1 Adaptive Attitude Estimation in Smartphone 26 3.1.1 Indirect Kalman filter-based attitude estimation 26 3.1.2 Conventional attitude estimation algorithms 29 3.1.3 Adaptive attitude estimation using ellipsoidal methods 30 3.2 Experimental Results 36 3.2.1 Simulation 36 3.2.2 Rate table experiment 44 3.2.3 Handheld rotation experiment 46 3.2.4 Magnetic disturbance experiment 49 3.3 Summary 53 Chapter 4 Pedestrian Dead Reckoning in Multiple Poses of a Smartphone 54 4.1 System Overview 55 4.2 Machine Learning-based Pose Classification 56 4.2.1 Training dataset 57 4.2.2 Feature extraction and selection 58 4.2.3 Pose classification result using supervised learning in PDR 62 4.3 Fusion of the Integration and Parametric Approaches in PDR 65 4.3.1 System model 67 4.3.2 Measurement model 67 4.3.3 Mode selection 74 4.3.4 Observability analysis 76 4.4 Experimental Results 82 4.4.1 AHRS results 82 4.4.2 PCA results 84 4.4.3 IA-PA results 88 4.5 Summary 100 Chapter 5 Conclusions 103 5.1 Summary of the Contributions 103 5.2 Future Works 105 국문초록 125 Acknowledgements 127Docto

A framework for experimental determination of localised vertical pedestrian forces on full-scale structures using wireless attitude and heading reference systems

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this record.A major weakness among loading models for pedestrians walking on flexible structures proposed in recent years is the various uncorroborated assumptions made in their development. This applies to spatio- temporal characteristics of pedestrian loading and the nature of multi-object interactions. To alleviate this problem, a framework for the determination of localised pedestrian forces on full-scale structures is presented using a wireless attitude and heading reference systems (AHRS). An AHRS comprises a triad of tri-axial accelerometers, gyroscopes and magnetometers managed by a dedicated data processing unit, allowing motion in three-dimensional space to be reconstructed. A pedestrian loading model based on a single point inertial measurement from an AHRS is derived and shown to perform well against benchmark data collected on an instrumented treadmill. Unlike other models, the current model does not take any predefined form nor does it require any extrapolations as to the timing and amplitude of pedestrian loading. In order to assess correctly the influence of the moving pedestrian on behaviour of a structure, an algorithm for tracking the point of application of pedestrian force is developed based on data from a single AHRS attached to a foot. A set of controlled walking tests with a single pedestrian is conducted on a real footbridge for validation purposes. A remarkably good match between the measured and simulated bridge response is found, indeed confirming applicability of the proposed framework.The research presented here was funded by EPSRC (grant EP/I029567/2). Authors thank Devon County Council for permitting the experimental campaign to be conducted on Baker Bridge in Exeter, UK, and Dr Erfan Shahabpour (supported by EPSRC grant EP/K03877X/1) for providing access to and assisting with measurements on the ADAL-3D treadmill at the University of Sheffield (funded by EPSRC grant EP/E018734/1)

Mixed Reality Browsers and Pedestrian Navigation in Augmented Cities

International audienceIn this paper, We use a declarative format for positional audio with synchronization between audio chunks using SMIL. This format has been specifically designed for the type of audio used in AR applications. The audio engine associated to this format is running on mobile platforms (iOS, Android). Our MRB browser called IXE use a format based on volunteered geographic information (OpenStreetMap) and OSM documents for IXE can be fully authored in side OSM editors like JOSM. This is in contrast with the other AR browsers like Layar, Juniao, Wikitude, which use a Point of Interest (POI) based format having no notion of ways. This introduces a fundamental difference and in some senses a duality relation between IXE and the other AR browsers. In IXE, Augmented Virtuality (AV) navigation along a route (composed of ways) is central and AR interaction with objects is delegated to associate 3D activities. In AR browsers, navigation along a route is delegated to associated map activities and AR interaction with objects is central. IXE supports multiple tracking technologies and therefore allows both indoor navigation in buildings and outdoor navigation at the level of sidewalks. A first android version of the IXE browser will be released at the end of 2013. Being based on volunteered geographic it will allow building accessible pedestrian networks in augmented cities

A framework for experimental determination of localised vertical pedestrian forces on full-scale structures using wireless attitude and heading reference systems

A major weakness among loading models for pedestrians walking on flexible structures proposed in recent years is the various uncorroborated assumptions made in their development. This applies to spatio-temporal characteristics of pedestrian loading and the nature of multi-object interactions. To alleviate this problem, a framework for the determination of localised pedestrian forces on full-scale structures is presented using a wireless attitude and heading reference systems (AHRS). An AHRS comprises a triad of tri-axial accelerometers, gyroscopes and magnetometers managed by a dedicated data processing unit, allowing motion in three-dimensional space to be reconstructed. A pedestrian loading model based on a single point inertial measurement from an AHRS is derived and shown to perform well against benchmark data collected on an instrumented treadmill. Unlike other models, the current model does not take any predefined form nor does it require any extrapolations as to the timing and amplitude of pedestrian loading. In order to assess correctly the influence of the moving pedestrian on behaviour of a structure, an algorithm for tracking the point of application of pedestrian force is developed based on data from a single AHRS attached to a foot. A set of controlled walking tests with a single pedestrian is conducted on a real footbridge for validation purposes. A remarkably good match between the measured and simulated bridge response is found, indeed confirming applicability of the proposed framework

Planetary Rover Inertial Navigation Applications: Pseudo Measurements and Wheel Terrain Interactions

Accurate localization is a critical component of any robotic system. During planetary missions, these systems are often limited by energy sources and slow spacecraft computers. Using proprioceptive localization (e.g., using an inertial measurement unit and wheel encoders) without external aiding is insufficient for accurate localization. This is mainly due to the integrated and unbounded errors of the inertial navigation solutions and the drifted position information from wheel encoders caused by wheel slippage. For this reason, planetary rovers often utilize exteroceptive (e.g., vision-based) sensors. On the one hand, localization with proprioceptive sensors is straightforward, computationally efficient, and continuous. On the other hand, using exteroceptive sensors for localization slows rover driving speed, reduces rover traversal rate, and these sensors are sensitive to the terrain features. Given the advantages and disadvantages of both methods, this thesis focuses on two objectives. First, improving the proprioceptive localization performance without significant changes to the rover operations. Second, enabling adaptive traversability rate based on the wheel-terrain interactions while keeping the localization reliable. To achieve the first objective, we utilized the zero-velocity, zero-angular rate updates, and non-holonomicity of a rover to improve rover localization performance even with the limited available sensor usage in a computationally efficient way. Pseudo-measurements generated from proprioceptive sensors when the rover is stationary conditions and the non-holonomic constraints while traversing can be utilized to improve the localization performance without any significant changes to the rover operations. Through this work, it is observed that a substantial improvement in localization performance, without the aid of additional exteroceptive sensor information. To achieve the second objective, the relationship between the estimation of localization uncertainty and wheel-terrain interactions through slip-ratio was investigated. This relationship was exposed with a Gaussian process with time series implementation by using the slippage estimation while the rover is moving. Then, it is predicted when to change from moving to stationary conditions by mapping the predicted slippage into localization uncertainty prediction. Instead of a periodic stopping framework, the method introduced in this work is a slip-aware localization method that enables the rover to stop more frequently in high-slip terrains whereas stops rover less frequently for low-slip terrains while keeping the proprioceptive localization reliable

Upper Limb Position Tracking with a Single Inertial Sensor Using Dead Reckoning Method with Drift Correction Techniques

Inertial sensors are widely used in human motion monitoring. Orientation and position are the two most widely used measurements for motion monitoring. Tracking with the use of multiple inertial sensors is based on kinematic modelling which achieves a good level of accuracy when biomechanical constraints are applied. More recently, there is growing interest in tracking motion with a single inertial sensor to simplify the measurement system. The dead reckoning method is commonly used for estimating position from inertial sensors. However, significant errors are generated after applying the dead reckoning method because of the presence of sensor offsets and drift. These errors limit the feasibility of monitoring upper limb motion via a single inertial sensing system. In this paper, error correction methods are evaluated to investigate the feasibility of using a single sensor to track the movement of one upper limb segment. These include zero velocity update, wavelet analysis and high-pass filtering. The experiments were carried out using the nine-hole peg test. The results show that zero velocity update is the most effective method to correct the drift from the dead reckoning-based position tracking. If this method is used, then the use of a single inertial sensor to track the movement of a single limb segment is feasible