Examination of Indoor Mobile Mapping Systems in a Diversified Internal/External Test Field

Nowadays, simultaneous localization and mapping (SLAM) algorithms support several commercial sensors which have recently been introduced to the market, and, like the more common mobile mapping systems (MMSs), are designed to acquire three-dimensional and high-resolution point clouds. The new systems are said to work both in external and internal environments, and completely avoid the use of targets and control points. The possibility of increasing productivity in three-dimensional digitization projects is fascinating, but data quality needs to be carefully evaluated to define appropriate fields of application. The paper presents the analytical measurement principle of these indoor mobile mapping systems (IMMSs) and the results of some tests performed on three commercial systems. A common test field was defined in order to acquire comparable data. By taking the already available terrestrial laser scan survey as the ground truth, the datasets under examination were compared with the reference and some assessments are presented which consider both quantitative and qualitative aspects. Geometric deformation in the final models was computed using the so-called Multiscale Model to Model Cloud Comparison (M3C2) algorithm. Cross sections and cloud to mesh (C2M) distances were also employed for a more detailed analysis. The real usability assessment is based on the features of recognizability, double surface evidence, and visualization effectiveness. For these evaluations, comparative images and tables are presented. © 2018 by the authors

적분 및 매개변수 기법 융합을 이용한 스마트폰 다중 동작에서 보행 항법

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 기계항공공학부, 2020. 8. 박찬국.In this dissertation, an IA-PA fusion-based PDR (Pedestrian Dead Reckoning) using low-cost inertial sensors is proposed to improve the indoor position estimation. Specifically, an IA (Integration Approach)-based PDR algorithm combined with measurements from PA (Parametric Approach) is constructed so that the algorithm is operated even in various poses that occur when a pedestrian moves with a smartphone indoors. In addition, I propose an algorithm that estimates the device attitude robustly in a disturbing situation by an ellipsoidal method. In addition, by using the machine learning-based pose recognition, it is possible to improve the position estimation performance by varying the measurement update according to the poses. First, I propose an adaptive attitude estimation based on ellipsoid technique to accurately estimate the direction of movement of a smartphone device. The AHRS (Attitude and Heading Reference System) uses an accelerometer and a magnetometer as measurements to calculate the attitude based on the gyro and to compensate for drift caused by gyro sensor errors. In general, the attitude estimation performance is poor in acceleration and geomagnetic disturbance situations, but in order to effectively improve the estimation performance, this dissertation proposes an ellipsoid-based adaptive attitude estimation technique. When a measurement disturbance comes in, it is possible to update the measurement more accurately than the adaptive estimation technique without considering the direction by adjusting the measurement covariance with the ellipsoid method considering the direction of the disturbance. In particular, when the disturbance only comes in one axis, the proposed algorithm can use the measurement partly by updating the other two axes considering the direction. The proposed algorithm shows its effectiveness in attitude estimation under disturbances through the rate table and motion capture equipment. Next, I propose a PDR algorithm that integrates IA and PA that can be operated in various poses. When moving indoors using a smartphone, there are many degrees of freedom, so various poses such as making a phone call, texting, and putting a pants pocket are possible. In the existing smartphone-based positioning algorithms, the position is estimated based on the PA, which can be used only when the pedestrian's walking direction and the device's direction coincide, and if it does not, the position error due to the mismatch in angle is large. In order to solve this problem, this dissertation proposes an algorithm that constructs state variables based on the IA and uses the position vector from the PA as a measurement. If the walking direction and the device heading do not match based on the pose recognized through machine learning technique, the position is updated in consideration of the direction calculated using PCA (Principal Component Analysis) and the step length obtained through the PA. It can be operated robustly even in various poses that occur. Through experiments considering various operating conditions and paths, it is confirmed that the proposed method stably estimates the position and improves performance even in various indoor environments.본 논문에서는 저가형 관성센서를 이용한 보행항법시스템 (PDR: Pedestrian Dead Reckoning)의 성능 향상 알고리즘을 제안한다. 구체적으로 보행자가 실내에서 스마트폰을 들고 이동할 때 발생하는 다양한 동작 상황에서도 운용될 수 있도록, 매개변수 기반 측정치를 사용하는 적분 기반의 보행자 항법 알고리즘을 구성한다. 또한 타원체 기반 자세 추정 알고리즘을 구성하여 외란 상황에서도 강인하게 자세를 추정하는 알고리즘을 제안한다. 추가적으로 기계학습 기반의 동작 인식 정보를 이용, 동작에 따른 측정치 업데이트를 달리함으로써 위치 추정 성능을 향상시킨다. 먼저 스마트폰 기기의 이동 방향을 정확하게 추정하기 위해 타원체 기법 기반 적응 자세 추정을 제안한다. 자세 추정 기법 (AHRS: Attitude and Heading Reference System)은 자이로를 기반으로 자세를 계산하고 자이로 센서오차에 의해 발생하는 드리프트를 보정하기 위해 측정치로 가속도계와 지자계를 사용한다. 일반적으로 가속 및 지자계 외란 상황에서는 자세 추정 성능이 떨어지는데, 추정 성능을 효과적으로 향상시키기 위해 본 논문에서는 타원체 기반 적응 자세 추정 기법을 제안한다. 측정치 외란이 들어오는 경우, 외란의 방향을 고려하여 타원체 기법으로 측정치 공분산을 조정해줌으로써 방향을 고려하지 않은 적응 추정 기법보다 정확하게 측정치 업데이트를 할 수 있다. 특히 외란이 한 축으로만 들어오는 경우, 제안한 알고리즘은 방향을 고려해 나머지 두 축에 대해서는 업데이트 해줌으로써 측정치를 부분적으로 사용할 수 있다. 레이트 테이블, 모션 캡쳐 장비를 통해 제안한 알고리즘의 자세 성능이 향상됨을 확인하였다. 다음으로 다양한 동작에서도 운용 가능한 적분 및 매개변수 기법을 융합하는 보행항법 알고리즘을 제안한다. 스마트폰을 이용해 실내를 이동할 때에는 자유도가 크기 때문에 전화 걸기, 문자, 바지 주머니 넣기 등 다양한 동작이 발생 가능하다. 기존의 스마트폰 기반 보행 항법에서는 매개변수 기법을 기반으로 위치를 추정하는데, 이는 보행자의 진행 방향과 기기의 방향이 일치하는 경우에만 사용 가능하며 일치하지 않는 경우 자세 오차로 인한 위치 오차가 크게 발생한다. 이러한 문제를 해결하기 위해 본 논문에서는 적분 기반 기법을 기반으로 상태변수를 구성하고 매개변수 기법을 통해 나오는 위치 벡터를 측정치로 사용하는 알고리즘을 제안한다. 만약 기계학습을 통해 인식한 동작을 바탕으로 진행 방향과 기기 방향이 일치하지 않는 경우, 주성분 분석을 통해 계산한 진행방향을 이용해 진행 방향을, 매개변수 기법을 통해 얻은 보폭으로 거리를 업데이트해 줌으로써 보행 중 발생하는 여러 동작에서도 강인하게 운용할 수 있다. 다양한 동작 상황 및 경로를 고려한 실험을 통해 위에서 제안한 방법이 다양한 실내 환경에서도 안정적으로 위치를 추정하고 성능이 향상됨을 확인하였다.Chapter 1 Introduction 1 1.1 Motivation and Background 1 1.2 Objectives and Contribution 5 1.3 Organization of the Dissertation 6 Chapter 2 Pedestrian Dead Reckoning System 8 2.1 Overview of Pedestrian Dead Reckoning 8 2.2 Parametric Approach 9 2.2.1 Step detection algorithm 11 2.2.2 Step length estimation algorithm 13 2.2.3 Heading estimation 14 2.3 Integration Approach 15 2.3.1 Extended Kalman filter 16 2.3.2 INS-EKF-ZUPT 19 2.4 Activity Recognition using Machine Learning 21 2.4.1 Challenges in HAR 21 2.4.2 Activity recognition chain 22 Chapter 3 Attitude Estimation in Smartphone 26 3.1 Adaptive Attitude Estimation in Smartphone 26 3.1.1 Indirect Kalman filter-based attitude estimation 26 3.1.2 Conventional attitude estimation algorithms 29 3.1.3 Adaptive attitude estimation using ellipsoidal methods 30 3.2 Experimental Results 36 3.2.1 Simulation 36 3.2.2 Rate table experiment 44 3.2.3 Handheld rotation experiment 46 3.2.4 Magnetic disturbance experiment 49 3.3 Summary 53 Chapter 4 Pedestrian Dead Reckoning in Multiple Poses of a Smartphone 54 4.1 System Overview 55 4.2 Machine Learning-based Pose Classification 56 4.2.1 Training dataset 57 4.2.2 Feature extraction and selection 58 4.2.3 Pose classification result using supervised learning in PDR 62 4.3 Fusion of the Integration and Parametric Approaches in PDR 65 4.3.1 System model 67 4.3.2 Measurement model 67 4.3.3 Mode selection 74 4.3.4 Observability analysis 76 4.4 Experimental Results 82 4.4.1 AHRS results 82 4.4.2 PCA results 84 4.4.3 IA-PA results 88 4.5 Summary 100 Chapter 5 Conclusions 103 5.1 Summary of the Contributions 103 5.2 Future Works 105 국문초록 125 Acknowledgements 127Docto

Map matching and heuristic elimination of gyro drift for personal navigation systems in GPS-denied conditions

This paper introduces a method for the substantial reduction of heading errors in inertial navigation systems used under GPS-denied conditions. Presumably, the method is applicable for both vehicle-based and personal navigation systems, but experiments were performed only with a personal navigation system called 'personal dead reckoning' (PDR). In order to work under GPS-denied conditions, the PDR system uses a foot-mounted inertial measurement unit (IMU). However, gyro drift in this IMU can cause large heading errors after just a few minutes of walking. To reduce these errors, the map-matched heuristic drift elimination (MAPHDE) method was developed, which estimates gyro drift errors by comparing IMU-derived heading to the direction of the nearest street segment in a database of street maps. A heuristic component in this method provides tolerance to short deviations from walking along the street, such as when crossing streets or intersections. MAPHDE keeps heading errors almost at zero, and, as a result, position errors are dramatically reduced. In this paper, MAPHDE was used in a variety of outdoor walks, without any use of GPS. This paper explains the MAPHDE method in detail and presents experimental results.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/90785/1/0957-0233_22_2_025205.pd

MULTI-SOURCE APPROACHES FOR COMPLEX ARCHITECTURE DOCUMENTATION: THE “PALAZZO DUCALE” IN GUBBIO (PERUGIA, ITALY)

Nowadays, the employment of rapid mapping solutions for architectural survey is more and more considered, not only for the strong reduction of the primary data acquisition times, but also thanks to their adaptability to various contexts, especially in the framework of Cultural Heritage documentation where tailored solutions are required. The combined use of Mobile Mapping Systems (MMS), Unmanned Aerial Vehicles (UAV) for close range aerial image acquisition, and Terrestrial Laser Scanning (TLS) seems to be an effective solution for the architectonic scale compliant to the level of detail and accuracy of 1 : 200, and 1 : 100 scales. The present research tries to evaluate the use of a handheld MMS, the ZEB-Revo RT by Geoslam, an UAV, the DJI Mavic Pro, and a LiDAR system, the Faro Focus3D S 120 by CAM2. The complex case of the documentation metric survey of Palazzo Ducale in Gubbio, the Montefeltro’s Palace, now hosting the museum of itself, allows the comparison of the effectiveness of the used technologies

Robust localization with wearable sensors

Measuring physical movements of humans and understanding human behaviour is useful in a variety of areas and disciplines. Human inertial tracking is a method that can be leveraged for monitoring complex actions that emerge from interactions between human actors and their environment. An accurate estimation of motion trajectories can support new approaches to pedestrian navigation, emergency rescue, athlete management, and medicine. However, tracking with wearable inertial sensors has several problems that need to be overcome, such as the low accuracy of consumer-grade inertial measurement units (IMUs), the error accumulation problem in long-term tracking, and the artefacts generated by movements that are less common. This thesis focusses on measuring human movements with wearable head-mounted sensors to accurately estimate the physical location of a person over time. The research consisted of (i) providing an overview of the current state of research for inertial tracking with wearable sensors, (ii) investigating the performance of new tracking algorithms that combine sensor fusion and data-driven machine learning, (iii) eliminating the effect of random head motion during tracking, (iv) creating robust long-term tracking systems with a Bayesian neural network and sequential Monte Carlo method, and (v) verifying that the system can be applied with changing modes of behaviour, defined as natural transitions from walking to running and vice versa. This research introduces a new system for inertial tracking with head-mounted sensors (which can be placed in, e.g. helmets, caps, or glasses). This technology can be used for long-term positional tracking to explore complex behaviours

LASER-VISUAL-INERTIAL ODOMETRY BASED SOLUTION FOR 3D HERITAGE MODELING: THE SANCTUARY OF THE BLESSED VIRGIN OF TROMPONE

Abstract. The advent of new mobile mapping systems that integrate different sensors has made it easier to acquire multiple 3D information with high speed. Today, technological development has allowed the creation of portable systems particularly suitable for indoor surveys, which mainly integrating LiDAR devices, chambers and inertial platforms, make it possible to create in a fast and easy way, full 3D model of the environment. However, the performance of these instruments differs depending on the acquisition context (indoor and outdoor), the characteristics of the scene (for example lighting, the presence of objects and people, reflecting surfaces, textures) and, above all, the mapping and localization algorithms implemented in devices. The purpose of this study is to analyse the results, and their accuracy, deriving from a survey conducted with the KAARTA Stencil 2 handheld system. This instrument, composed of a 3D LiDAR Velodyne VLP-16, a MEMS inertial platform and a feature tracker camera, it is able to realize the temporal 3D map of the environment. Specifically, the acquisition tests were carried out in a context of metrical documentation of an architectural heritage, in order extract architectural detail for the future reconstruction of virtual and augmented reality environments and for Historical Building Information Modeling purposes. The achieved results were analysed and the discrepancies from some reference LiDAR data are computed for a final evaluation. The system was tested in the church and cloister of the Sanctuary of the Beata Vergine del Trompone in Moncrivello (VC) (Italy).</p

Ambient Intelligence for Next-Generation AR

Next-generation augmented reality (AR) promises a high degree of context-awareness - a detailed knowledge of the environmental, user, social and system conditions in which an AR experience takes place. This will facilitate both the closer integration of the real and virtual worlds, and the provision of context-specific content or adaptations. However, environmental awareness in particular is challenging to achieve using AR devices alone; not only are these mobile devices' view of an environment spatially and temporally limited, but the data obtained by onboard sensors is frequently inaccurate and incomplete. This, combined with the fact that many aspects of core AR functionality and user experiences are impacted by properties of the real environment, motivates the use of ambient IoT devices, wireless sensors and actuators placed in the surrounding environment, for the measurement and optimization of environment properties. In this book chapter we categorize and examine the wide variety of ways in which these IoT sensors and actuators can support or enhance AR experiences, including quantitative insights and proof-of-concept systems that will inform the development of future solutions. We outline the challenges and opportunities associated with several important research directions which must be addressed to realize the full potential of next-generation AR.Comment: This is a preprint of a book chapter which will appear in the Springer Handbook of the Metavers

LASER-VISUAL-INERTIAL ODOMETRY BASED SOLUTION for 3D HERITAGE MODELING: The SANCTUARY of the BLESSED VIRGIN of TROMPONE

The advent of new mobile mapping systems that integrate different sensors has made it easier to acquire multiple 3D information with high speed. Today, technological development has allowed the creation of portable systems particularly suitable for indoor surveys, which mainly integrating LiDAR devices, chambers and inertial platforms, make it possible to create in a fast and easy way, full 3D model of the environment. However, the performance of these instruments differs depending on the acquisition context (indoor and outdoor), the characteristics of the scene (for example lighting, the presence of objects and people, reflecting surfaces, textures) and, above all, the mapping and localization algorithms implemented in devices. The purpose of this study is to analyse the results, and their accuracy, deriving from a survey conducted with the KAARTA Stencil 2 handheld system. This instrument, composed of a 3D LiDAR Velodyne VLP-16, a MEMS inertial platform and a feature tracker camera, it is able to realize the temporal 3D map of the environment. Specifically, the acquisition tests were carried out in a context of metrical documentation of an architectural heritage, in order extract architectural detail for the future reconstruction of virtual and augmented reality environments and for Historical Building Information Modeling purposes. The achieved results were analysed and the discrepancies from some reference LiDAR data are computed for a final evaluation. The system was tested in the church and cloister of the Sanctuary of the Beata Vergine del Trompone in Moncrivello (VC) (Italy)