지자기 지문을 이용한 비용 효율적이고 실용적인 실내 측위 시스템

학위논문(박사)--서울대학교 대학원 :공과대학 전기·컴퓨터공학부,2020. 2. 권태경.지난 십수 년 동안 학계와 산업 영역을 막론하여 실내 측위 시스템이 널리 연구되어 왔다. 실내 측위 시스템을 설계할 때 WiFi, Bluetooth, 관성 센서와 같은 다양한 종류의 센서나 무선 인터페이스들을 활용할 수 있는데, 그 중에서도 지자기 센서에서 측정한 자기장의 패턴을 측위에 사용하는 시스템은 정확성과 안정성 측면에서 타 시스템에 비해 큰 장점을 지니고 있다. 철골 구조에 기반하여 건축된 현대 건축물들의 실내 공간에는 지자기장의 왜곡이 발생하며 이는 곧 실내의 개별 공간들에 고유하고 안정적인 지자기 패턴을 발생시킨다. 이를 실내 측위의 맥락에서는 지자기 지문이라고 정의하는데 이 지자기 지문은 사용자의 움직임, 문과 창문의 여닫힘 등과 같은 일상적인 환경 변화에 강건하며, 특히 WiFi 등 무선 신호와 비교하였을 때 높은 수준의 안정성을 보인다. 그러나 이와 같은 안정성에도 불구하고 지자기 지문을 이용하여 실내 측위 시스템을 설계할 때에는 몇가지 고려해야할 사항들이 있다. 먼저 지자기 지문은 센서 데이터의 낮은 차원 개수로 인하여 AP 개수에 따라 데이터의 차원이 수십, 수백개에 달하는 무선 신호에 비하여 구별성이 매우 낮다. 대부분의 기존 연구들은 많은 양의 연산을 수행하는 복잡한 알고리즘과 많은 센서를 이용하여 이를 극복하였다. 또 대상 공간의 지자기 지문을 수집하기 위한 사전 조사 비용과 주기적인 지자기 지문 재수집에 드는 관리 비용 역시 지자기 지문을 사용할 때 고려해야할 문제점이다. 따라서 본 논문은 이 두 가지 문제점을 해결하는 데에 초점을 맞추어 작성되었다. 먼저 사물 인터넷 (IoT, Internet of Things) 환경에서 실내 측위를 위해 지자기 지문을 활용하는 에너지 효율적인 경량 시스템을 연구하였다. BLE 인터페이스와 센서 2개(지자기 및 가속도계)만 탑재한 새로운 하드웨어 설계 방식을 제안하였으며, 이 기기는 코인 크기의 배터리를 사용할 경우 1년 동안 동작 가능하였다. 또한 최소한의 센서 데이터만을 이용하여 강건한 사용자 보행 모델과 효율적인 알고리즘을 도입한 파티클 필터 프레임워크를 제안하였다. 직접 설계한 사물인터넷 기기와 알고리즘을 적용한 결과, 본 시스템은 낮은 계산 복잡성과 높은 에너지 효율을 보여주면서 일반적인 사무실 공간에 대하여 평균 1.62m의 측위 정확도를 달성하였다. 다음으로는 크라우드소싱 방식을 활용하는 지자기 지문 기반의 실내 측위 시스템을 제안하였다. 실내 측위의 맥락에서 크라우드소싱은 명시적인 대상 공간의 사전 조사 과정 없이 지자기 지문 데이터베이스를 구성하는 방법이다. 지난 십수년 간 실내 측위 연구를 위해 크라우드소싱 방법론이 활발하게 연구되어 왔으나, 크라우드소싱에 기반한 기존 측위 시스템은 일반적으로 사전 조사 기반 시스템보다 위치 정확도가 낮게 측정되어왔다. 크라우드소싱 기반 시스텝의 낮은 측위 성능을 극복하기 위해, 본 연구에서는 지자기 기반 크라우드소싱 데이터를 사용한 실내 측위 시스템을 제안한다. 명시적인 수집 과정 없이 스마트폰 사용자가 일상생활에서 자연스럽게 지자기 지문 데이터베이스를 구축할 수 있도록 HMM 기반의 새로운 학습 모델을 구현하였으며, 주로 복도 위주로 구성된 실내 공간에서의 평가 결과, 제안된 시스템이 96.47\%의 학습 정확도와 0.25m의 중간값 측위 정확도를 달성하였다.Over the decades, indoor localization system has been widely studied in the academic and also in the industrial area. Many sensors or wireless signals such as WiFi, Bluetooth, and inertial sensors are available when designing an indoor localization system, but among them, the systems using the geomagnetic field has advantages concerning accuracy and stability. Every spatial point in an indoor space has its own distinct and stable fingerprint, which arises owing to the distortion of the magnetic field induced by the surrounding steel and iron structures. The magnetic fingerprint is robust to environmental changes like pedestrian activities and door/window movements, particularly compared with radio signals such as WiFi. This phenomenon makes many indoor positioning techniques rely on the magnetic field as an essential source of localization. Despite the robustness, there are some challenges when leveraging the magnetic fingerprint to design the indoor localization system. Due to lower discernibility of the magnetic fingerprint, most of the existing studies have exploited high computational algorithms and many sensors. Also, the cost of a site survey to collect the fingerprints and periodic management of target spaces is still problematic when using magnetic fingerprints. This dissertation thus focuses on these two challenges. First, we present an energy-efficient and lightweight system that utilizes the magnetic field for indoor positioning in the Internet of Things (IoT) environments. We propose a new hardware design of an IoT device that only has a BLE interface and two sensors (magnetometer and accelerometer), with the lifetime of one year when using a coin-size battery. We further propose an augmented particle filter framework that features a robust motion model and algorithmic-efficient localization heuristics with minimal sensory data. The prototype-based evaluation shows that the proposed system achieves a median accuracy of 1.62 m for an office building while exhibiting low computational complexity and high energy efficiency. Next, we propose a magnetic fingerprint-based indoor localization system leveraging a crowdsourcing approach. In the aspect of indoor localization, crowdsourcing is a method to construct the fingerprint database without the explicit site-survey process. Over the past decade, crowdsourcing has been actively studied for indoor localization. However, the existing localization systems based on crowdsourcing usually achieve lower location accuracy than the site survey based systems. To overcome the low performance of the crowdsourcing based approaches, we design an indoor positioning system using the crowdsourced data of the magnetic field. We substantiate a novel HMM-based learning model to construct a database of magnetic field fingerprints from smartphone users. Experiments in an indoor space consisting of aisles show that the proposed system achieves the learning accuracy of 96.47\% and median positioning accuracy of 0.25m.Chapter 1 Introduction 1p - Motivation 1p - Indoor Localization Overview 2p - Magnetic Field based Systems 5p - Organization of Dissertation 7p Chapter 2 An Energy-efficient and Lightweight Indoor Localization System for Internet-of-Things (IoT) Environments 8p - Introduction 8p - Related Work 11p - Issues on Using Magnetic Field 12p - Characteristics of Magnetic Field 12p - Variation Issues 13p - Sensing Rate 17p - Design of Energy-efficient Device for Localization 18p - Hardware Design 18p - Structure of the BLE Beacon Frame 21p - Processing Sensor Data 22p - System Architecture 25p - Overview 25p - Site Survey Methodology 25p - Particle Filter Framework 26p - Evaluation 36p - Implementation 37p - Experiment Setup 38p - Localization Performance 38p - Energy and Algorithmic Efficiency 44p - Discussion 46p Chapter 3 Magnetic Field based Indoor Localization System:A Crowdsourcing Approach 51p - Introduction 51p - Characteristics of Magnetic Field 54p - Robustness 54p - Distinctness 54p - Diversity issues 55p - Design of HMM for Crowdsoucing 56p - Basic Model 56p - Issues in HMM Learning 59p - Preliminary Experiments 59p - System Architecture 63p - Enhanced Learning Model 63p - Pre-processing Crowdsourced Data 65p - Allocating Initial HMM Parameters 67p - Comparing Similarity between the Magnetic Fingerprints 70p - Evaluation 71p - Experimental Settings 71p - Learning Accuracy 71p - Positioning Accuracy 73p - Algorithmic Efficiency 75p Chapter 4 Discussion and Future Work 76p - Open Space Issue 76p Chapter 5 Conclusion 82p Bibliography 84p 초록 92pDocto

Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding

Computer vision has a great potential to help our daily lives by searching for lost keys, watering flowers or reminding us to take a pill. To succeed with such tasks, computer vision methods need to be trained from real and diverse examples of our daily dynamic scenes. While most of such scenes are not particularly exciting, they typically do not appear on YouTube, in movies or TV broadcasts. So how do we collect sufficiently many diverse but boring samples representing our lives? We propose a novel Hollywood in Homes approach to collect such data. Instead of shooting videos in the lab, we ensure diversity by distributing and crowdsourcing the whole process of video creation from script writing to video recording and annotation. Following this procedure we collect a new dataset, Charades, with hundreds of people recording videos in their own homes, acting out casual everyday activities. The dataset is composed of 9,848 annotated videos with an average length of 30 seconds, showing activities of 267 people from three continents. Each video is annotated by multiple free-text descriptions, action labels, action intervals and classes of interacted objects. In total, Charades provides 27,847 video descriptions, 66,500 temporally localized intervals for 157 action classes and 41,104 labels for 46 object classes. Using this rich data, we evaluate and provide baseline results for several tasks including action recognition and automatic description generation. We believe that the realism, diversity, and casual nature of this dataset will present unique challenges and new opportunities for computer vision community

Iterative Design and Prototyping of Computer Vision Mediated Remote Sighted Assistance

Remote sighted assistance (RSA) is an emerging navigational aid for people with visual impairments (PVI). Using scenario-based design to illustrate our ideas, we developed a prototype showcasing potential applications for computer vision to support RSA interactions. We reviewed the prototype demonstrating real-world navigation scenarios with an RSA expert, and then iteratively refined the prototype based on feedback. We reviewed the refined prototype with 12 RSA professionals to evaluate the desirability and feasibility of the prototyped computer vision concepts. The RSA expert and professionals were engaged by, and reacted insightfully and constructively to the proposed design ideas. We discuss what we learned about key resources, goals, and challenges of the RSA prosthetic practice through our iterative prototype review, as well as implications for the design of RSA systems and the integration of computer vision technologies into RSA

A Self Regulating and Crowdsourced Indoor Positioning System through Wi-Fi Fingerprinting for Multi Storey Building

[EN] Unobtrusive indoor location systems must rely on methods that avoid the deployment of large hardware infrastructures or require information owned by network administrators. Fingerprinting methods can work under these circumstances by comparing the real-time received RSSI values of a smartphone coming from existing Wi-Fi access points with a previous database of stored values with known locations. Under the fingerprinting approach, conventional methods suffer from large indoor scenarios since the number of fingerprints grows with the localization area. To that aim, fingerprinting-based localization systems require fast machine learning algorithms that reduce the computational complexity when comparing real-time and stored values. In this paper, popular machine learning (ML) algorithms have been implemented for the classification of real time RSSI values to predict the user location and propose an intelligent indoor positioning system (I-IPS). The proposed I-IPS has been integrated with multi-agent framework for betterment of context-aware service (CAS). The obtained results have been analyzed and validated through established statistical measurements and superior performance achieved