CSI-fingerprinting Indoor Localization via Attention-Augmented Residual Convolutional Neural Network

Deep learning has been widely adopted for channel state information (CSI)-fingerprinting indoor localization systems. These systems usually consist of two main parts, i.e., a positioning network that learns the mapping from high-dimensional CSI to physical locations and a tracking system that utilizes historical CSI to reduce the positioning error. This paper presents a new localization system with high accuracy and generality. On the one hand, the receptive field of the existing convolutional neural network (CNN)-based positioning networks is limited, restricting their performance as useful information in CSI is not explored thoroughly. As a solution, we propose a novel attention-augmented residual CNN to utilize the local information and global context in CSI exhaustively. On the other hand, considering the generality of a tracking system, we decouple the tracking system from the CSI environments so that one tracking system for all environments becomes possible. Specifically, we remodel the tracking problem as a denoising task and solve it with deep trajectory prior. Furthermore, we investigate how the precision difference of inertial measurement units will adversely affect the tracking performance and adopt plug-and-play to solve the precision difference problem. Experiments show the superiority of our methods over existing approaches in performance and generality improvement.Comment: 32 pages, Added references in section 2,3; Added explanations for some academic terms; Corrected typos; Added experiments in section 5, previous results unchanged; is under review for possible publicatio

User Experience Enhancement on Smartphones using Wireless Communication Technologies

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2020. 8. 박세웅.Recently, various sensors as well as wireless communication technologies such as Wi-Fi and Bluetooth Low Energy (BLE) have been equipped with smartphones. In addition, in many cases, users use a smartphone while on the move, so if a wireless communication technologies and various sensors are used for a mobile user, a better user experience can be provided. For example, when a user moves while using Wi-Fi, the user experience can be improved by providing a seamless Wi-Fi service. In addition, it is possible to provide a special service such as indoor positioning or navigation by estimating the users mobility in an indoor environment, and additional services such as location-based advertising and payment systems can also be provided. Therefore, improving the user experience by using wireless communication technology and smartphones sensors is considered to be an important research field in the future. In this dissertation, we propose three systems that can improve the user experience or convenience by usingWi-Fi, BLE, and smartphones sensors: (i) BLEND: BLE beacon-aided fast Wi-Fi handoff for smartphones, (ii) PYLON: Smartphone based Indoor Path Estimation and Localization without Human Intervention, (iii) FINISH: Fully-automated Indoor Navigation using Smartphones with Zero Human Assistance. First, we propose fast handoff scheme called BLEND exploiting BLE as secondary radio. We conduct detailed analysis of the sticky client problem on commercial smartphones with experiment and close examination of Android source code. We propose BLEND, which exploits BLE modules to provide smartphones with prior knowledge of the presence and information of APs operating at 2.4 and 5 GHz Wi-Fi channels. BLEND operating with only application requires no hardware and Android source code modification of smartphones.We prototype BLEND with commercial smartphones and evaluate the performance in real environment. Our measurement results demonstrate that BLEND significantly improves throughput and video bitrate by up to 61% and 111%, compared to a commercial Android application, respectively, with negligible energy overhead. Second, we design a path estimation and localization system, termed PYLON, which is plug-and-play on Android smartphones. PYLON includes a novel landmark correction scheme that leverages real doors of indoor environments consisting of floor plan mapping, door passing time detection and correction. It operates without any user intervention. PYLON relaxes some requirements for localization systems. It does not require any modifications to hardware or software of smartphones, and the initial location of WiFi APs, BLE beacons, and users. We implement PYLON on five Android smartphones and evaluate it on two office buildings with the help of three participants to prove applicability and scalability. PYLON achieves very high floor plan mapping accuracy with a low localization error. Finally, We design a fully-automated navigation system, termed FINISH, which addresses the problems of existing previous indoor navigation systems. FINISH generates the radio map of an indoor building based on the localization system to determine the initial location of the user. FINISH relaxes some requirements for current indoor navigation systems. It does not require any human assistance to provide navigation instructions. In addition, it is plug-and-play on Android smartphones. We implement FINISH on five Android smartphones and evaluate it on five floors of an office building with the help of multiple users to prove applicability and scalability. FINISH determines the location of the user with extremely high accuracy with in one step. In summary, we propose systems that enhance the users convenience and experience by utilizing wireless infrastructures such as Wi-Fi and BLE and various smartphones sensors such as accelerometer, gyroscope, and barometer equipped in smartphones. Systems are implemented on commercial smartphones to verify the performance through experiments. As a result, systems show the excellent performance that can enhance the users experience.1 Introduction 1 1.1 Motivation 1 1.2 Overview of Existing Approaches 3 1.2.1 Wi-Fi handoff for smartphones 3 1.2.2 Indoor path estimation and localization 4 1.2.3 Indoor navigation 5 1.3 Main Contributions 7 1.3.1 BLEND: BLE Beacon-aided Fast Handoff for Smartphones 7 1.3.2 PYLON: Smartphone Based Indoor Path Estimation and Localization with Human Intervention 8 1.3.3 FINISH: Fully-automated Indoor Navigation using Smartphones with Zero Human Assistance 9 1.4 Organization of Dissertation 10 2 BLEND: BLE Beacon-Aided FastWi-Fi Handoff for Smartphones 11 2.1 Introduction 11 2.2 Related Work 14 2.2.1 Wi-Fi-based Handoff 14 2.2.2 WPAN-aided AP Discovery 15 2.3 Background 16 2.3.1 Handoff Procedure in IEEE 802.11 16 2.3.2 BSS Load Element in IEEE 802.11 16 2.3.3 Bluetooth Low Energy 17 2.4 Sticky Client Problem 17 2.4.1 Sticky Client Problem of Commercial Smartphone 17 2.4.2 Cause of Sticky Client Problem 20 2.5 BLEND: Proposed Scheme 21 2.5.1 Advantages and Necessities of BLE as Secondary Low-Power Radio 21 2.5.2 Overall Architecture 22 2.5.3 AP Operation 23 2.5.4 Smartphone Operation 24 2.5.5 Verification of aTH estimation 28 2.6 Performance Evaluation 30 2.6.1 Implementation and Measurement Setup 30 2.6.2 Saturated Traffic Scenario 31 2.6.3 Video Streaming Scenario 35 2.7 Summary 38 3 PYLON: Smartphone based Indoor Path Estimation and Localization without Human Intervention 41 3.1 Introduction 41 3.2 Background and Related Work 44 3.2.1 Infrastructure-Based Localization 44 3.2.2 Fingerprint-Based Localization 45 3.2.3 Model-Based Localization 45 3.2.4 Dead Reckoning 46 3.2.5 Landmark-Based Localization 47 3.2.6 Simultaneous Localization and Mapping (SLAM) 47 3.3 System Overview 48 3.3.1 Notable RSSI Signature 49 3.3.2 Smartphone Operation 50 3.3.3 Server Operation 51 3.4 Path Estimation 52 3.4.1 Step Detection 52 3.4.2 Step Length Estimation 54 3.4.3 Walking Direction 54 3.4.4 Location Update 55 3.5 Landmark Correction Part 1: Virtual Room Generation 56 3.5.1 RSSI Stacking Difference 56 3.5.2 Virtual Room Generation 57 3.5.3 Virtual Graph Generation 59 3.5.4 Physical Graph Generation 60 3.6 Landmark Correction Part 2: From Floor Plan Mapping to Path Correction 60 3.6.1 Candidate Graph Generation 60 3.6.2 Backbone Node Mapping 62 3.6.3 Dead-end Node Mapping 65 3.6.4 Final Candidate Graph Selection 66 3.6.5 Door Passing Time Detection 68 3.6.6 Path Correction 70 3.7 Particle Filter 71 3.8 Performance Evaluation 73 3.8.1 Implementation and Measurement Setup 73 3.8.2 Step Detection Accuracy 77 3.8.3 Floor Plan Mapping Accuracy 77 3.8.4 Door Passing Time 78 3.8.5 Walking Direction and Localization Performance 81 3.8.6 Impact of WiFi AP and BLE Beacon Number 84 3.8.7 Impact of Walking Distance and Speed 84 3.8.8 Performance on Different Areas 87 3.9 Summary 87 4 FINISH: Fully-automated Indoor Navigation using Smartphones with Zero Human Assistance 91 4.1 Introduction 91 4.2 Related Work 92 4.2.1 Localization-based Navigation System 92 4.2.2 Peer-to-peer Navigation System 93 4.3 System Overview 93 4.3.1 System Architecture 93 4.3.2 An Example for Navigation 95 4.4 Level Change Detection and Floor Decision 96 4.4.1 Level Change Detection 96 4.5 Real-time navigation 97 4.5.1 Initial Floor and Location Decision 97 4.5.2 Orientation Adjustment 98 4.5.3 Shortest Path Estimation 99 4.6 Performance Evaluation 99 4.6.1 Initial Location Accuracy 99 4.6.2 Real-Time Navigation Accuracy 100 4.7 Summary 101 5 Conclusion 102 5.1 Research Contributions 102 5.2 Future Work 103 Abstract (In Korean) 118 감사의 글Docto

Artificial Intelligence Of Things For Ubiquitous Sports Analytics

To enable mobile devices to perform in-the-wild sports analytics, particularly swing tracking, remains an open question. A crucial challenge is to develop robust methods that can operate across various sports (e.g., golf and tennis), different sensors (cameras and IMU), and diverse human users. Traditional approaches typically rely on vision-based or IMU-based methods to extract key points from subjects in order to estimate trajectory predictions. However, these methods struggle to generate accurate swing tracking, as vision-based techniques are susceptible to occlusion, and IMU sensors are notorious for accumulated errors. In this thesis, we propose several innovative solutions by leveraging AIoT, including the IoT with ubiquitous wearable devices such as smartphones and smart wristbands, and harnessing the power of AI such as deep neural networks, to achieve ubiquitous sports analytics. We make three main technical contributions: a tailored deep neural network design, network model automatic search, and model domain adaptation to address the problem of heterogeneity among devices, human subjects, and sports for ubiquitous sports analytics. In Chapter 2, we begin with the design of a prototype that combines IMU and depth sensor fusion, along with a tailored deep neural network, to address the occlusion problems faced by depth sensors during swings. To recover swing trajectories with fine-grained details, we propose a CNN-LSTM architecture that learns multi-modalities within depth and IMU sensor fusion. In Chapter 3, we develop a framework to reduce the overhead of model design for new devices, sports, and human users. By designing a regression-based stochastic NAS method, we improve swing-tracking algorithms through automatic model generation. We also extend our studies to include unseen human users, sensor devices, and sports. Leveraging a domain adaptation method, we propose a framework that eliminates the need for tedious training data collection and labeling for new users, devices, and sports via adversarial learning. In Chapter 4, we present a framework to alleviate the model parameter selection process in NAS, as introduced in Chapter 3. By employing zero-cost proxies, we search for the optimal swing tracking architecture without training, in a significantly larger candidate model pool. We demonstrate that the proposed method outperforms state-of-the-art approaches in swing tracking, as well as in adapting to different subjects, sports, and devices. Overall, this thesis develops a series of innovative machine learning algorithms to enable ubiquitous IoT wearable devices to perform accurate swing analytics (e.g., tracking, analysis, and assessment) in real-world conditions

Qualitative Action Recognition by Wireless Radio Signals in Human–Machine Systems

Human-machine systems required a deep understanding of human behaviors. Most existing research on action recognition has focused on discriminating between different actions, however, the quality of executing an action has received little attention thus far. In this paper, we study the quality assessment of driving behaviors and present WiQ, a system to assess the quality of actions based on radio signals. This system includes three key components, a deep neural network based learning engine to extract the quality information from the changes of signal strength, a gradient-based method to detect the signal boundary for an individual action, and an activity-based fusion policy to improve the recognition performance in a noisy environment. By using the quality information, WiQ can differentiate a triple body status with an accuracy of 97%, whereas for identification among 15 drivers, the average accuracy is 88%. Our results show that, via dedicated analysis of radio signals, a fine-grained action characterization can be achieved, which can facilitate a large variety of applications, such as smart driving assistants

RSS Indoor Localization Based on a Single Access Point

This research work investigates how RSS information fusion from a single, multi-antenna access point (AP) can be used to perform device localization in indoor RSS based localization systems. The proposed approach demonstrates that different RSS values can be obtained by carefully modifying each AP antenna orientation and polarization, allowing the generation of unique, low correlation fingerprints, for the area of interest. Each AP antenna can be used to generate a set of fingerprint radiomaps for different antenna orientations and/or polarization. The RSS fingerprints generated from all antennas of the single AP can be then combined to create a multi-layer fingerprint radiomap. In order to select the optimum fingerprint layers in the multilayer radiomap the proposed methodology evaluates the obtained localization accuracy, for each fingerprint radio map combination, for various well-known deterministic and probabilistic algorithms (Weighted k-Nearest-Neighbor-WKNN and Minimum Mean Square Error-MMSE). The optimum candidate multi-layer radiomap is then examined by calculating the correlation level of each fingerprint pair by using the "Tolerance Based-Normal Probability Distribution (TBNPD)" algorithm. Both steps take place during the offline phase, and it is demonstrated that this approach results in selecting the optimum multi-layer fingerprint radiomap combination. The proposed approach can be used to provide localisation services in areas served only by a single AP

Motion tracking problems in Internet of Things (IoT) and wireless networking

The dissertation focuses on inferring various motion patterns of internet-of-things (IoT) devices, by leveraging inertial sensors embedded in these objects, as well as wireless signals emitted (or reflected) from them. For instance, we use a combination of GPS signals and inertial sensors on drones to precisely track its 3D orientation over time, ultimately improving safety against failures and crashes. In another application in sports analytics, we embed sensors and radios inside baseballs and cricket balls and compute their 3D trajectory and spin patterns, even when they move at extremely high speeds. In a third application for wireless networks, we explore the possibility of physically moving wireless infrastructure like Access Points and basestations on robots and drones for enhancing the network performance. While these are diverse applications in drones, sports analytics, and wireless networks, the common theme underlying the research is in the development of the core motion-related building blocks. Specifically, we emphasize the philosophy of "fusion of multi modal sensor data with application specific model” as the design principle for building the next generation of diverse IoT applications. To this end, we draw on theoretical techniques in wireless communication, signal processing, and statistics, but translate them to completely functional systems on real-world platforms

A Meta-Review of Indoor Positioning Systems

An accurate and reliable Indoor Positioning System (IPS) applicable to most indoor scenarios has been sought for many years. The number of technologies, techniques, and approaches in general used in IPS proposals is remarkable. Such diversity, coupled with the lack of strict and verifiable evaluations, leads to difficulties for appreciating the true value of most proposals. This paper provides a meta-review that performed a comprehensive compilation of 62 survey papers in the area of indoor positioning. The paper provides the reader with an introduction to IPS and the different technologies, techniques, and some methods commonly employed. The introduction is supported by consensus found in the selected surveys and referenced using them. Thus, the meta-review allows the reader to inspect the IPS current state at a glance and serve as a guide for the reader to easily find further details on each technology used in IPS. The analyses of the meta-review contributed with insights on the abundance and academic significance of published IPS proposals using the criterion of the number of citations. Moreover, 75 works are identified as relevant works in the research topic from a selection of about 4000 works cited in the analyzed surveys

Recent Advances in Indoor Localization Systems and Technologies

Despite the enormous technical progress seen in the past few years, the maturity of indoor localization technologies has not yet reached the level of GNSS solutions. The 23 selected papers in this book present the recent advances and new developments in indoor localization systems and technologies, propose novel or improved methods with increased performance, provide insight into various aspects of quality control, and also introduce some unorthodox positioning methods