적분 및 매개변수 기법 융합을 이용한 스마트폰 다중 동작에서 보행 항법

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 기계항공공학부, 2020. 8. 박찬국.In this dissertation, an IA-PA fusion-based PDR (Pedestrian Dead Reckoning) using low-cost inertial sensors is proposed to improve the indoor position estimation. Specifically, an IA (Integration Approach)-based PDR algorithm combined with measurements from PA (Parametric Approach) is constructed so that the algorithm is operated even in various poses that occur when a pedestrian moves with a smartphone indoors. In addition, I propose an algorithm that estimates the device attitude robustly in a disturbing situation by an ellipsoidal method. In addition, by using the machine learning-based pose recognition, it is possible to improve the position estimation performance by varying the measurement update according to the poses. First, I propose an adaptive attitude estimation based on ellipsoid technique to accurately estimate the direction of movement of a smartphone device. The AHRS (Attitude and Heading Reference System) uses an accelerometer and a magnetometer as measurements to calculate the attitude based on the gyro and to compensate for drift caused by gyro sensor errors. In general, the attitude estimation performance is poor in acceleration and geomagnetic disturbance situations, but in order to effectively improve the estimation performance, this dissertation proposes an ellipsoid-based adaptive attitude estimation technique. When a measurement disturbance comes in, it is possible to update the measurement more accurately than the adaptive estimation technique without considering the direction by adjusting the measurement covariance with the ellipsoid method considering the direction of the disturbance. In particular, when the disturbance only comes in one axis, the proposed algorithm can use the measurement partly by updating the other two axes considering the direction. The proposed algorithm shows its effectiveness in attitude estimation under disturbances through the rate table and motion capture equipment. Next, I propose a PDR algorithm that integrates IA and PA that can be operated in various poses. When moving indoors using a smartphone, there are many degrees of freedom, so various poses such as making a phone call, texting, and putting a pants pocket are possible. In the existing smartphone-based positioning algorithms, the position is estimated based on the PA, which can be used only when the pedestrian's walking direction and the device's direction coincide, and if it does not, the position error due to the mismatch in angle is large. In order to solve this problem, this dissertation proposes an algorithm that constructs state variables based on the IA and uses the position vector from the PA as a measurement. If the walking direction and the device heading do not match based on the pose recognized through machine learning technique, the position is updated in consideration of the direction calculated using PCA (Principal Component Analysis) and the step length obtained through the PA. It can be operated robustly even in various poses that occur. Through experiments considering various operating conditions and paths, it is confirmed that the proposed method stably estimates the position and improves performance even in various indoor environments.본 논문에서는 저가형 관성센서를 이용한 보행항법시스템 (PDR: Pedestrian Dead Reckoning)의 성능 향상 알고리즘을 제안한다. 구체적으로 보행자가 실내에서 스마트폰을 들고 이동할 때 발생하는 다양한 동작 상황에서도 운용될 수 있도록, 매개변수 기반 측정치를 사용하는 적분 기반의 보행자 항법 알고리즘을 구성한다. 또한 타원체 기반 자세 추정 알고리즘을 구성하여 외란 상황에서도 강인하게 자세를 추정하는 알고리즘을 제안한다. 추가적으로 기계학습 기반의 동작 인식 정보를 이용, 동작에 따른 측정치 업데이트를 달리함으로써 위치 추정 성능을 향상시킨다. 먼저 스마트폰 기기의 이동 방향을 정확하게 추정하기 위해 타원체 기법 기반 적응 자세 추정을 제안한다. 자세 추정 기법 (AHRS: Attitude and Heading Reference System)은 자이로를 기반으로 자세를 계산하고 자이로 센서오차에 의해 발생하는 드리프트를 보정하기 위해 측정치로 가속도계와 지자계를 사용한다. 일반적으로 가속 및 지자계 외란 상황에서는 자세 추정 성능이 떨어지는데, 추정 성능을 효과적으로 향상시키기 위해 본 논문에서는 타원체 기반 적응 자세 추정 기법을 제안한다. 측정치 외란이 들어오는 경우, 외란의 방향을 고려하여 타원체 기법으로 측정치 공분산을 조정해줌으로써 방향을 고려하지 않은 적응 추정 기법보다 정확하게 측정치 업데이트를 할 수 있다. 특히 외란이 한 축으로만 들어오는 경우, 제안한 알고리즘은 방향을 고려해 나머지 두 축에 대해서는 업데이트 해줌으로써 측정치를 부분적으로 사용할 수 있다. 레이트 테이블, 모션 캡쳐 장비를 통해 제안한 알고리즘의 자세 성능이 향상됨을 확인하였다. 다음으로 다양한 동작에서도 운용 가능한 적분 및 매개변수 기법을 융합하는 보행항법 알고리즘을 제안한다. 스마트폰을 이용해 실내를 이동할 때에는 자유도가 크기 때문에 전화 걸기, 문자, 바지 주머니 넣기 등 다양한 동작이 발생 가능하다. 기존의 스마트폰 기반 보행 항법에서는 매개변수 기법을 기반으로 위치를 추정하는데, 이는 보행자의 진행 방향과 기기의 방향이 일치하는 경우에만 사용 가능하며 일치하지 않는 경우 자세 오차로 인한 위치 오차가 크게 발생한다. 이러한 문제를 해결하기 위해 본 논문에서는 적분 기반 기법을 기반으로 상태변수를 구성하고 매개변수 기법을 통해 나오는 위치 벡터를 측정치로 사용하는 알고리즘을 제안한다. 만약 기계학습을 통해 인식한 동작을 바탕으로 진행 방향과 기기 방향이 일치하지 않는 경우, 주성분 분석을 통해 계산한 진행방향을 이용해 진행 방향을, 매개변수 기법을 통해 얻은 보폭으로 거리를 업데이트해 줌으로써 보행 중 발생하는 여러 동작에서도 강인하게 운용할 수 있다. 다양한 동작 상황 및 경로를 고려한 실험을 통해 위에서 제안한 방법이 다양한 실내 환경에서도 안정적으로 위치를 추정하고 성능이 향상됨을 확인하였다.Chapter 1 Introduction 1 1.1 Motivation and Background 1 1.2 Objectives and Contribution 5 1.3 Organization of the Dissertation 6 Chapter 2 Pedestrian Dead Reckoning System 8 2.1 Overview of Pedestrian Dead Reckoning 8 2.2 Parametric Approach 9 2.2.1 Step detection algorithm 11 2.2.2 Step length estimation algorithm 13 2.2.3 Heading estimation 14 2.3 Integration Approach 15 2.3.1 Extended Kalman filter 16 2.3.2 INS-EKF-ZUPT 19 2.4 Activity Recognition using Machine Learning 21 2.4.1 Challenges in HAR 21 2.4.2 Activity recognition chain 22 Chapter 3 Attitude Estimation in Smartphone 26 3.1 Adaptive Attitude Estimation in Smartphone 26 3.1.1 Indirect Kalman filter-based attitude estimation 26 3.1.2 Conventional attitude estimation algorithms 29 3.1.3 Adaptive attitude estimation using ellipsoidal methods 30 3.2 Experimental Results 36 3.2.1 Simulation 36 3.2.2 Rate table experiment 44 3.2.3 Handheld rotation experiment 46 3.2.4 Magnetic disturbance experiment 49 3.3 Summary 53 Chapter 4 Pedestrian Dead Reckoning in Multiple Poses of a Smartphone 54 4.1 System Overview 55 4.2 Machine Learning-based Pose Classification 56 4.2.1 Training dataset 57 4.2.2 Feature extraction and selection 58 4.2.3 Pose classification result using supervised learning in PDR 62 4.3 Fusion of the Integration and Parametric Approaches in PDR 65 4.3.1 System model 67 4.3.2 Measurement model 67 4.3.3 Mode selection 74 4.3.4 Observability analysis 76 4.4 Experimental Results 82 4.4.1 AHRS results 82 4.4.2 PCA results 84 4.4.3 IA-PA results 88 4.5 Summary 100 Chapter 5 Conclusions 103 5.1 Summary of the Contributions 103 5.2 Future Works 105 국문초록 125 Acknowledgements 127Docto

Evaluation of Pedometer Performance Across Multiple Gait Types Using Video for Ground Truth

This dissertation is motivated by improving healthcare through the development of wearable sensors. This work seeks improvement in the evaluation and development of pedometer algorithms, and is composed of two chapters describing the collection of the dataset and describing the im-plementation and evaluation of three previously developed pedometer algorithms on the dataset collected. Our goal is to analyze pedometer algorithms under more natural conditions that occur during daily living where gaits are frequently changing or remain regular for only brief periods of time. We video recorded 30 participants performing 3 activities: walking around a track, walking through a building, and moving around a room. The ground truth time of each step was manu-ally marked in the accelerometer signals through video observation. Collectively 60,853 steps were recorded and annotated. A subclass of steps called shifts were identified as those occurring at the beginning and end of regular strides, during gait changes, and during pivots changing the direction of motion. While shifts comprised only .03% of steps in the regular stride activity, they comprised 10-25% of steps in the semi-regular and unstructured activities. We believe these motions should be identified separately, as they provide different accelerometer signals, and likely result in different amounts of energy expenditure. This dataset will be the first to specifically allow for pedometer algorithms to be evaluated on unstructured gaits that more closely model natural activities. In order to provide pilot evaluation data, a commercial pedometer, the Fitbit Charge 2, and three prior step detection algorithms were analyzed. The Fitbit consistently underestimated the total number of steps taken across each gait type. Because the Fitbit algorithm is proprietary, it could not be reimplemented and examined beyond a raw step count comparison. Three previously published step detection algorithms, however, were implemented and examined in detail on the dataset. The three algorithms are based on three different methods of step detection; peak detection, zero crossing (threshold based), and autocorrelation. The evaluation of these algorithms was performed across 5 dimensions, including algorithm, parameter set, gait type, sensor position, and evaluation metric, which yielded 108 individual measures of accuracy. Accuracy across each of the 5 dimensions were examined individually in order to determine trends. In general, training parameters to this dataset caused a significant accuracy improvement. The most accurate algorithm was dependent on gait type, sensor position, and evaluation metric, indicating no clear “best approach” to step detection. In general, algorithms were most accurate for regular gait and least accurate for unstructured gait. In general, accuracy was higher for hip and ankle worn sensors than it was for wrist worn sensors. Finally, evaluation across running count accuracy (RCA) and step detection accuracy (SDA) revealed similar trends across gait type and sensor position, but each metric indicated a different algorithm with the highest overall accuracy. A classifier was developed to identify gait type in an effort to use this information to improve pedometer accuracy. The classifier’s features are based on the Fast Fourier Transform (FFT) applied to the accelerometer data gathered from each sensor throughout each activity. A peak detector was developed to identify the maximum value of the FFT, the width of the peak yielding the maximum value, and the number of peaks in each FFT. These features were then applied to a Naive Bayes classifier, which correctly identified the gait (regular, semi-regular, or unstructured) with 84% accuracy. A varying algorithm pedometer was then developed which switched between the peak detection, threshold crossing, and autocorrelation based algorithms depending on which algorithm performed best for the sensor location and detected gait type. This process yielded a step detection accuracy of 84%. This was a 3% improvement when compared to the greatest accuracy achieved by the best performing algorithm, the peak detection algorithm. It was also identified that in order to provide quicker real-time transitions between algorithms, the data should be examined in smaller windows. Window sizes of 3, 5, 8, 10, 15, 20, and 30 seconds were tested, and the highest overall accuracy was found for a window size of 5 seconds. These smaller windows of time included behaviors which do not correspond directly with the regular, semi-regular, and unstructured gait activities. Instead, three stride types were identified: steady stride, irregular stride, and idle. These stride types were identified with 82% accuracy. This experiment showed that at an activity level, gait detection can improve pedometer accuracy and indicated that applying the same principles to a smaller window size could allow for more responsive real-time algorithm selection

Highly accurate step counting at variouswalking states using low-cost inertial measurement unit support indoor positioning system

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. Accurate step counting is essential for indoor positioning, health monitoring systems, and other indoor positioning services. There are several publications and commercial applications in step counting. Nevertheless, over-counting, under-counting, and false walking problems are still encountered in these methods. In this paper, we propose to develop a highly accurate step counting method to solve these limitations by proposing four features: Minimal peak distance, minimal peak prominence, dynamic thresholding, and vibration elimination, and these features are adaptive with the user’s states. Our proposed features are combined with periodicity and similarity features to solve false walking problem. The proposed method shows a significant improvement of 99.42% and 96.47% of the average of accuracy in free walking and false walking problems, respectively, on our datasets. Furthermore, our proposed method also achieves the average accuracy of 97.04% on public datasets and better accuracy in comparison with three commercial step counting applications: Pedometer and Weight Loss Coach installed on Lenovo P780, Health apps in iPhone 5s (iOS 10.3.3), and S-health in Samsung Galaxy S5 (Android 6.01)

Situation inference and context recognition for intelligent mobile sensing applications

The usage of smart devices is an integral element in our daily life. With the richness of data streaming from sensors embedded in these smart devices, the applications of ubiquitous computing are limitless for future intelligent systems. Situation inference is a non-trivial issue in the domain of ubiquitous computing research due to the challenges of mobile sensing in unrestricted environments. There are various advantages to having robust and intelligent situation inference from data streamed by mobile sensors. For instance, we would be able to gain a deeper understanding of human behaviours in certain situations via a mobile sensing paradigm. It can then be used to recommend resources or actions for enhanced cognitive augmentation, such as improved productivity and better human decision making. Sensor data can be streamed continuously from heterogeneous sources with different frequencies in a pervasive sensing environment (e.g., smart home). It is difficult and time-consuming to build a model that is capable of recognising multiple activities. These activities can be performed simultaneously with different granularities. We investigate the separability aspect of multiple activities in time-series data and develop OPTWIN as a technique to determine the optimal time window size to be used in a segmentation process. As a result, this novel technique reduces need for sensitivity analysis, which is an inherently time consuming task. To achieve an effective outcome, OPTWIN leverages multi-objective optimisation by minimising the impurity (the number of overlapped windows of human activity labels on one label space over time series data) while maximising class separability. The next issue is to effectively model and recognise multiple activities based on the user's contexts. Hence, an intelligent system should address the problem of multi-activity and context recognition prior to the situation inference process in mobile sensing applications. The performance of simultaneous recognition of human activities and contexts can be easily affected by the choices of modelling approaches to build an intelligent model. We investigate the associations of these activities and contexts at multiple levels of mobile sensing perspectives to reveal the dependency property in multi-context recognition problem. We design a Mobile Context Recognition System, which incorporates a Context-based Activity Recognition (CBAR) modelling approach to produce effective outcome from both multi-stage and multi-target inference processes to recognise human activities and their contexts simultaneously. Upon our empirical evaluation on real-world datasets, the CBAR modelling approach has significantly improved the overall accuracy of simultaneous inference on transportation mode and human activity of mobile users. The accuracy of activity and context recognition can also be influenced progressively by how reliable user annotations are. Essentially, reliable user annotation is required for activity and context recognition. These annotations are usually acquired during data capture in the world. We research the needs of reducing user burden effectively during mobile sensor data collection, through experience sampling of these annotations in-the-wild. To this end, we design CoAct-nnotate --- a technique that aims to improve the sampling of human activities and contexts by providing accurate annotation prediction and facilitates interactive user feedback acquisition for ubiquitous sensing. CoAct-nnotate incorporates a novel multi-view multi-instance learning mechanism to perform more accurate annotation prediction. It also includes a progressive learning process (i.e., model retraining based on co-training and active learning) to improve its predictive performance over time. Moving beyond context recognition of mobile users, human activities can be related to essential tasks that the users perform in daily life. Conversely, the boundaries between the types of tasks are inherently difficult to establish, as they can be defined differently from the individuals' perspectives. Consequently, we investigate the implication of contextual signals for user tasks in mobile sensing applications. To define the boundary of tasks and hence recognise them, we incorporate such situation inference process (i.e., task recognition) into the proposed Intelligent Task Recognition (ITR) framework to learn users' Cyber-Physical-Social activities from their mobile sensing data. By recognising the engaged tasks accurately at a given time via mobile sensing, an intelligent system can then offer proactive supports to its user to progress and complete their tasks. Finally, for robust and effective learning of mobile sensing data from heterogeneous sources (e.g., Internet-of-Things in a mobile crowdsensing scenario), we investigate the utility of sensor data in provisioning their storage and design QDaS --- an application agnostic framework for quality-driven data summarisation. This allows an effective data summarisation by performing density-based clustering on multivariate time series data from a selected source (i.e., data provider). Thus, the source selection process is determined by the measure of data quality. Nevertheless, this framework allows intelligent systems to retain comparable predictive results by its effective learning on the compact representations of mobile sensing data, while having a higher space saving ratio. This thesis contains novel contributions in terms of the techniques that can be employed for mobile situation inference and context recognition, especially in the domain of ubiquitous computing and intelligent assistive technologies. This research implements and extends the capabilities of machine learning techniques to solve real-world problems on multi-context recognition, mobile data summarisation and situation inference from mobile sensing. We firmly believe that the contributions in this research will help the future study to move forward in building more intelligent systems and applications

Optimising the Loading Diversity of Rail Passenger Crowding using On-Board Occupancy Data

Crowded conditions on trains can lead to lower passenger satisfaction, discourage rail travel, result in negative economic impacts and are a factor in a number of health and safety hazards. In the UK there is an annual survey of rail passenger crowding, although the measures used do not reflect coach-by-coach variations, nor do they reflect variations across the peak period. In this MPhil thesis I investigated the application of weight-based automatic passenger counting data to deliver more even loadings on trains through the provision of new real-time and static solutions. In addition I investigated the potential benefits of such solutions in terms of reduced dwell times and reduced crowding. The overall concept proposed was to make the most of the existing available capacity; for example, so that no-one is standing when seats are available. Through analysing a large sample of air suspension data, I identified station-specific trends where some coaches were over capacity while others had spare capacity. I also conducted a critical review of academic research into on-train crowding and solutions that seek to optimise ‘loading diversity’. This study contributes to this emerging subject area in several ways: I propose two new metrics to describe inter-coach loading diversity that, unlike existing metrics, contain information relative to the capacity; I have revealed a link between the inter-coach loading diversity metrics and estimated boarding times, with trains classified as ‘very uneven’ on departure typically having dwell times of approximately five to ten seconds greater than services that were classified as being ‘even’ with a similar total number of passengers on board; and finally I have applied classification supervised learning techniques to predict the load factor for a given service and these predictors were an improvement over taking the historical average

Studies on Sensor Aided Positioning and Context Awareness

This thesis studies Global Navigation Satellite Systems (GNSS) in combination with sensor systems that can be used for positioning and obtaining richer context information. When a GNSS is integrated with sensors, such as accelerometers, gyroscopes and barometric altimeters, valuable information can be produced for several applications; for example availability or/and performance of the navigation system can be increased. In addition to position technologies, GNSS devices are integrated more often with different types of technologies to fulfil several needs, e.g., different types of context recognition. The most common integrated devices are accelerometer, gyroscope, and magnetometer but also other sensors could be used.More specifically, this thesis presents sensor aided positioning with two satellite signals with altitude assistance. The method uses both pseudorange and Doppler measurements. The system is required to be stationary during the process and a source of altitude information, e.g., a MEMS barometer, is needed in addition to a basic GNSS receiver. Authentic pseudorange and Doppler measurements with simulated altitude were used used to test the algorithm. Results showed that normally the accuracy of couple of kilometers is acquired. Thesis also studies on what kind of errors barometric altimeter might encounter especially in personal positioning. The results show that barometers in differential mode provide highly accurate altitude solution (within tens of centimeters), but local disturbances in pressure need to be acknowledged in the application design. For example, heating, ventilating, and air conditioning in a car can have effect of few meters. Thus this could cause problems if the barometer is used as a altimeter for under meter-level positioning or navigation.We also explore methods for sensor aided GNSS systems for context recognition. First, the activity and environment recognition from mobile phone sensor and radio receiver data is investigated. The aim is in activity (e.g., walking, running, or driving a vehicle) and environment (e.g., street, home, or restaurant) detection. The thesis introduces an algorithm for user specific adaptation of the context model parameters using the feedback from the user, which can provide a confidence measure about the correctness of a classification. A real-life data collection campaign validate the proposed method. In addition, the thesis presents a concept for automated crash detection to motorcycles. In this concept, three different inertial measurement units are attached to the motorist’s helmet, torso of the motorist, and to the rear of the motor cycle. A maximum a posteriori classifier is trained to classify the crash and normal driving. Crash dummy tests were done by throwing the dummy from different altitudes to simulate the effect of crash to the motorist and real data is collected by driving the motorcycle. Preliminary results proved the potential of the proposed method could be applicable in real situations. In all the proposed systems in this thesis, knowledge of the context can help the positioning system, but also positioning system can help in determining the context

Flexible Application-Layer Multicast in Heterogeneous Networks

This work develops a set of peer-to-peer-based protocols and extensions in order to provide Internet-wide group communication. The focus is put to the question how different access technologies can be integrated in order to face the growing traffic load problem. Thereby, protocols are developed that allow autonomous adaptation to the current network situation on the one hand and the integration of WiFi domains where applicable on the other hand