User Experience Enhancement on Smartphones using Wireless Communication Technologies

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2020. 8. 박세웅.Recently, various sensors as well as wireless communication technologies such as Wi-Fi and Bluetooth Low Energy (BLE) have been equipped with smartphones. In addition, in many cases, users use a smartphone while on the move, so if a wireless communication technologies and various sensors are used for a mobile user, a better user experience can be provided. For example, when a user moves while using Wi-Fi, the user experience can be improved by providing a seamless Wi-Fi service. In addition, it is possible to provide a special service such as indoor positioning or navigation by estimating the users mobility in an indoor environment, and additional services such as location-based advertising and payment systems can also be provided. Therefore, improving the user experience by using wireless communication technology and smartphones sensors is considered to be an important research field in the future. In this dissertation, we propose three systems that can improve the user experience or convenience by usingWi-Fi, BLE, and smartphones sensors: (i) BLEND: BLE beacon-aided fast Wi-Fi handoff for smartphones, (ii) PYLON: Smartphone based Indoor Path Estimation and Localization without Human Intervention, (iii) FINISH: Fully-automated Indoor Navigation using Smartphones with Zero Human Assistance. First, we propose fast handoff scheme called BLEND exploiting BLE as secondary radio. We conduct detailed analysis of the sticky client problem on commercial smartphones with experiment and close examination of Android source code. We propose BLEND, which exploits BLE modules to provide smartphones with prior knowledge of the presence and information of APs operating at 2.4 and 5 GHz Wi-Fi channels. BLEND operating with only application requires no hardware and Android source code modification of smartphones.We prototype BLEND with commercial smartphones and evaluate the performance in real environment. Our measurement results demonstrate that BLEND significantly improves throughput and video bitrate by up to 61% and 111%, compared to a commercial Android application, respectively, with negligible energy overhead. Second, we design a path estimation and localization system, termed PYLON, which is plug-and-play on Android smartphones. PYLON includes a novel landmark correction scheme that leverages real doors of indoor environments consisting of floor plan mapping, door passing time detection and correction. It operates without any user intervention. PYLON relaxes some requirements for localization systems. It does not require any modifications to hardware or software of smartphones, and the initial location of WiFi APs, BLE beacons, and users. We implement PYLON on five Android smartphones and evaluate it on two office buildings with the help of three participants to prove applicability and scalability. PYLON achieves very high floor plan mapping accuracy with a low localization error. Finally, We design a fully-automated navigation system, termed FINISH, which addresses the problems of existing previous indoor navigation systems. FINISH generates the radio map of an indoor building based on the localization system to determine the initial location of the user. FINISH relaxes some requirements for current indoor navigation systems. It does not require any human assistance to provide navigation instructions. In addition, it is plug-and-play on Android smartphones. We implement FINISH on five Android smartphones and evaluate it on five floors of an office building with the help of multiple users to prove applicability and scalability. FINISH determines the location of the user with extremely high accuracy with in one step. In summary, we propose systems that enhance the users convenience and experience by utilizing wireless infrastructures such as Wi-Fi and BLE and various smartphones sensors such as accelerometer, gyroscope, and barometer equipped in smartphones. Systems are implemented on commercial smartphones to verify the performance through experiments. As a result, systems show the excellent performance that can enhance the users experience.1 Introduction 1 1.1 Motivation 1 1.2 Overview of Existing Approaches 3 1.2.1 Wi-Fi handoff for smartphones 3 1.2.2 Indoor path estimation and localization 4 1.2.3 Indoor navigation 5 1.3 Main Contributions 7 1.3.1 BLEND: BLE Beacon-aided Fast Handoff for Smartphones 7 1.3.2 PYLON: Smartphone Based Indoor Path Estimation and Localization with Human Intervention 8 1.3.3 FINISH: Fully-automated Indoor Navigation using Smartphones with Zero Human Assistance 9 1.4 Organization of Dissertation 10 2 BLEND: BLE Beacon-Aided FastWi-Fi Handoff for Smartphones 11 2.1 Introduction 11 2.2 Related Work 14 2.2.1 Wi-Fi-based Handoff 14 2.2.2 WPAN-aided AP Discovery 15 2.3 Background 16 2.3.1 Handoff Procedure in IEEE 802.11 16 2.3.2 BSS Load Element in IEEE 802.11 16 2.3.3 Bluetooth Low Energy 17 2.4 Sticky Client Problem 17 2.4.1 Sticky Client Problem of Commercial Smartphone 17 2.4.2 Cause of Sticky Client Problem 20 2.5 BLEND: Proposed Scheme 21 2.5.1 Advantages and Necessities of BLE as Secondary Low-Power Radio 21 2.5.2 Overall Architecture 22 2.5.3 AP Operation 23 2.5.4 Smartphone Operation 24 2.5.5 Verification of aTH estimation 28 2.6 Performance Evaluation 30 2.6.1 Implementation and Measurement Setup 30 2.6.2 Saturated Traffic Scenario 31 2.6.3 Video Streaming Scenario 35 2.7 Summary 38 3 PYLON: Smartphone based Indoor Path Estimation and Localization without Human Intervention 41 3.1 Introduction 41 3.2 Background and Related Work 44 3.2.1 Infrastructure-Based Localization 44 3.2.2 Fingerprint-Based Localization 45 3.2.3 Model-Based Localization 45 3.2.4 Dead Reckoning 46 3.2.5 Landmark-Based Localization 47 3.2.6 Simultaneous Localization and Mapping (SLAM) 47 3.3 System Overview 48 3.3.1 Notable RSSI Signature 49 3.3.2 Smartphone Operation 50 3.3.3 Server Operation 51 3.4 Path Estimation 52 3.4.1 Step Detection 52 3.4.2 Step Length Estimation 54 3.4.3 Walking Direction 54 3.4.4 Location Update 55 3.5 Landmark Correction Part 1: Virtual Room Generation 56 3.5.1 RSSI Stacking Difference 56 3.5.2 Virtual Room Generation 57 3.5.3 Virtual Graph Generation 59 3.5.4 Physical Graph Generation 60 3.6 Landmark Correction Part 2: From Floor Plan Mapping to Path Correction 60 3.6.1 Candidate Graph Generation 60 3.6.2 Backbone Node Mapping 62 3.6.3 Dead-end Node Mapping 65 3.6.4 Final Candidate Graph Selection 66 3.6.5 Door Passing Time Detection 68 3.6.6 Path Correction 70 3.7 Particle Filter 71 3.8 Performance Evaluation 73 3.8.1 Implementation and Measurement Setup 73 3.8.2 Step Detection Accuracy 77 3.8.3 Floor Plan Mapping Accuracy 77 3.8.4 Door Passing Time 78 3.8.5 Walking Direction and Localization Performance 81 3.8.6 Impact of WiFi AP and BLE Beacon Number 84 3.8.7 Impact of Walking Distance and Speed 84 3.8.8 Performance on Different Areas 87 3.9 Summary 87 4 FINISH: Fully-automated Indoor Navigation using Smartphones with Zero Human Assistance 91 4.1 Introduction 91 4.2 Related Work 92 4.2.1 Localization-based Navigation System 92 4.2.2 Peer-to-peer Navigation System 93 4.3 System Overview 93 4.3.1 System Architecture 93 4.3.2 An Example for Navigation 95 4.4 Level Change Detection and Floor Decision 96 4.4.1 Level Change Detection 96 4.5 Real-time navigation 97 4.5.1 Initial Floor and Location Decision 97 4.5.2 Orientation Adjustment 98 4.5.3 Shortest Path Estimation 99 4.6 Performance Evaluation 99 4.6.1 Initial Location Accuracy 99 4.6.2 Real-Time Navigation Accuracy 100 4.7 Summary 101 5 Conclusion 102 5.1 Research Contributions 102 5.2 Future Work 103 Abstract (In Korean) 118 감사의 글Docto

A review of smartphones based indoor positioning: challenges and applications

The continual proliferation of mobile devices has encouraged much effort in using the smartphones for indoor positioning. This article is dedicated to review the most recent and interesting smartphones based indoor navigation systems, ranging from electromagnetic to inertia to visible light ones, with an emphasis on their unique challenges and potential real-world applications. A taxonomy of smartphones sensors will be introduced, which serves as the basis to categorise different positioning systems for reviewing. A set of criteria to be used for the evaluation purpose will be devised. For each sensor category, the most recent, interesting and practical systems will be examined, with detailed discussion on the open research questions for the academics, and the practicality for the potential clients

Recent Advances in Indoor Localization Systems and Technologies

Despite the enormous technical progress seen in the past few years, the maturity of indoor localization technologies has not yet reached the level of GNSS solutions. The 23 selected papers in this book present the recent advances and new developments in indoor localization systems and technologies, propose novel or improved methods with increased performance, provide insight into various aspects of quality control, and also introduce some unorthodox positioning methods

BLE-based Indoor Localization and Contact Tracing Approaches

Internet of Things (IoT) has penetrated different aspects of modern life with smart sensors being prevalent within our surrounding indoor environments. Furthermore, dependence on IoT-based Contact Tracing (CT) models has significantly increased mainly due to the COVID-19 pandemic. There is, therefore, an urgent quest to develop/design efficient, autonomous, trustworthy, and secure indoor CT solutions leveraging accurate indoor localization/tracking approaches. In this context, the first objective of this Ph.D. thesis is to enhance accuracy of Bluetooth Low Energy (BLE)-based indoor localization. BLE-based localization is typically performed based on the Received Signal Strength Indicator (RSSI). Extreme fluctuations of the RSSI occurring due to different factors such as multi-path effects and noise, however, prevent the BLE technology to be a reliable solution with acceptable accuracy for dynamic tracking/localization in indoor environments. In this regard, first, an IoT dataset is constructed based on multiple thoroughly separated indoor environments to incorporate the effects of various interferences faced in different spaces. The constructed dataset is then used to develop a Reinforcement Learning (RL)-based information fusion strategy to form a multiple-model implementation consisting of RSSI, Pedestrian dead reckoning (PDR), and Angle-of-Arrival (AoA)-based models. In the second part of the thesis, the focus is devoted to application of multi-agent Deep Neural Networks (DNN) models for indoor tracking. DNN-based approaches are, however, prone to overfitting and high sensitivity to parameter selection, which results in sample inefficiency. Moreover, data labelling is a time-consuming and costly procedure. To address these issues, we leverage Successor Representations (SR)-based techniques, which can learn the expected discounted future state occupancy, and the immediate reward of each state. A Deep Multi-Agent Successor Representation framework is proposed that can adapt quickly to the changes in a multi-agent environment faster than the Model-Free (MF) RL methods and with a lower computational cost compared to Model-Based (MB) RL algorithms. In the third part of the thesis, the developed indoor localization techniques are utilized to design a novel indoor CT solution, referred to as the Trustworthy Blockchain-enabled system for Indoor Contact Tracing (TB-ICT) framework. The TB-ICT is a fully distributed and innovative blockchain platform exploiting the proposed dynamic Proof of Work (dPoW) approach coupled with a Randomized Hash Window (W-Hash) and dynamic Proof of Credit (dPoC) mechanisms

Map matching by using inertial sensors: literature review

This literature review aims to clarify what is known about map matching by using inertial sensors and what are the requirements for map matching, inertial sensors, placement and possible complementary position technology. The target is to develop a wearable location system that can position itself within a complex construction environment automatically with the aid of an accurate building model. The wearable location system should work on a tablet computer which is running an augmented reality (AR) solution and is capable of track and visualize 3D-CAD models in real environment. The wearable location system is needed to support the system in initialization of the accurate camera pose calculation and automatically finding the right location in the 3D-CAD model. One type of sensor which does seem applicable to people tracking is inertial measurement unit (IMU). The IMU sensors in aerospace applications, based on laser based gyroscopes, are big but provide a very accurate position estimation with a limited drift. Small and light units such as those based on Micro-Electro-Mechanical (MEMS) sensors are becoming very popular, but they have a significant bias and therefore suffer from large drifts and require method for calibration like map matching. The system requires very little fixed infrastructure, the monetary cost is proportional to the number of users, rather than to the coverage area as is the case for traditional absolute indoor location systems.Siirretty Doriast

Prioritizing Content of Interest in Multimedia Data Compression

Image and video compression techniques make data transmission and storage in digital multimedia systems more efficient and feasible for the system's limited storage and bandwidth. Many generic image and video compression techniques such as JPEG and H.264/AVC have been standardized and are now widely adopted. Despite their great success, we observe that these standard compression techniques are not the best solution for data compression in special types of multimedia systems such as microscopy videos and low-power wireless broadcast systems. In these application-specific systems where the content of interest in the multimedia data is known and well-defined, we should re-think the design of a data compression pipeline. We hypothesize that by identifying and prioritizing multimedia data's content of interest, new compression methods can be invented that are far more effective than standard techniques. In this dissertation, a set of new data compression methods based on the idea of prioritizing the content of interest has been proposed for three different kinds of multimedia systems. I will show that the key to designing efficient compression techniques in these three cases is to prioritize the content of interest in the data. The definition of the content of interest of multimedia data depends on the application. First, I show that for microscopy videos, the content of interest is defined as the spatial regions in the video frame with pixels that don't only contain noise. Keeping data in those regions with high quality and throwing out other information yields to a novel microscopy video compression technique. Second, I show that for a Bluetooth low energy beacon based system, practical multimedia data storage and transmission is possible by prioritizing content of interest. I designed custom image compression techniques that preserve edges in a binary image, or foreground regions of a color image of indoor or outdoor objects. Last, I present a new indoor Bluetooth low energy beacon based augmented reality system that integrates a 3D moving object compression method that prioritizes the content of interest.Doctor of Philosoph

Multiple Model Bayesian Estimation for BLE-based Localization and RL-based Decision Support of Autonomous Agents

With the rapid emergence of Internet of Things (IoT), we are more and more surrounded by smart connected devices (agents) with integrated sensing, processing, and communication capabilities. In particular, IoT-based positioning has become of primary importance for providing advanced Location-based Services (LBSs) in indoor environments. Several LBSs have been developed recently such as navigation assistance in hospitals, localization/tracking in smart buildings, and providing assistive services via autonomous agents collectively act as an Internet of Robotic Things (IoRT). The focus of the thesis is on the following two research topics when it comes to autonomous agents providing LBSs in indoor environments: (i) Self-Localization, which is the autonomous agent’s ability to obtain knowledge of its own location, and; (ii) Localized Decision Support System, which refers to an autonomous agent’s ability to perform optimal actions towards achieving pre-defined objectives. With regards to Item (i), the thesis develops innovative localization solutions based on Bluetooth Low Energy (BLE), referred to Bluetooth Smart. Given unavailability of Global Positioning System (GPS) in indoor environments, BLE has attracted considerable attention due to its low cost, low energy consumption, and widespread availability in smart hand-held devices. Because of multipath fading and fluctuations in the indoor environment, however, BLE-based localization approaches fail to achieve high accuracies. To address these challenges, different linear and non-linear Bayesian-based estimation frameworks are proposed in this thesis. Among which, the thesis proposes a novel Multiple-Model and BLE-based tracking framework, referred to as the STUPEFY. The proposed STUPEFY framework uses set-valued information and is designed by coupling a non-linear Bayesian-based estimation model (Box Particle Filter) with fingerprinting-based methodologies to improve the overall localization accuracy. With regards to the Item (ii), there has been an increasing surge of interest on development of advanced Reinforcement Learning (RL) systems. The objective is development of intelligent approaches to learn optimal control policies directly from smart agents’ interactions with the environment. In this regard, Deep Neural Networks (DNNs) provide an attractive modeling mechanism to approximate the value function using sample transitions. DNN-based solutions, however, suffer from high sensitivity to parameter selection, are prone to overfitting, and are not very sample efficient. As a remedy to the aforementioned problems, the thesis proposes an innovative Multiple-Model Kalman Temporal Difference (MM-KTD) framework, which adapts the parameters of the filter using the observed states and rewards. Moreover, an active learning method is proposed to enhance the sampling efficiency of the overall system. The proposed MM-KTD framework can learn the optimal policy with significantly reduced number of samples as compared to its DNN-based counterparts