Multimodal Sensing for Robust and Energy-Efficient Context Detection with Smart Mobile Devices

Adoption of smart mobile devices (smartphones, wearables, etc.) is rapidly growing. There are already over 2 billion smartphone users worldwide [1] and the percentage of smartphone users is expected to be over 50% in the next five years [2]. These devices feature rich sensing capabilities which allow inferences about mobile device user’s surroundings and behavior. Multiple and diverse sensors common on such mobile devices facilitate observing the environment from different perspectives, which helps to increase robustness of inferences and enables more complex context detection tasks. Though a larger number of sensing modalities can be beneficial for more accurate and wider mobile context detection, integrating these sensor streams is non-trivial. This thesis presents how multimodal sensor data can be integrated to facilitate ro- bust and energy efficient mobile context detection, considering three important and challenging detection tasks: indoor localization, indoor-outdoor detection and human activity recognition. This thesis presents three methods for multimodal sensor inte- gration, each applied for a different type of context detection task considered in this thesis. These are gradually decreasing in design complexity, starting with a solution based on an engineering approach decomposing context detection to simpler tasks and integrating these with a particle filter for indoor localization. This is followed by man- ual extraction of features from different sensors and using an adaptive machine learn- ing technique called semi-supervised learning for indoor-outdoor detection. Finally, a method using deep neural networks capable of extracting non-intuitive features di- rectly from raw sensor data is used for human activity recognition; this method also provides higher degree of generalization to other context detection tasks. Energy efficiency is an important consideration in general for battery powered mo- bile devices and context detection is no exception. In the various context detection tasks and solutions presented in this thesis, particular attention is paid to this issue by relying largely on sensors that consume low energy and on lightweight computations. Overall, the solutions presented improve on the state of the art in terms of accuracy and robustness while keeping the energy consumption low, making them practical for use on mobile devices

A Non-Line-of-Sight Mitigation Method For Indoor Ultra-Wideband Localization With Multiple Walls

Ultra-wideband (UWB) ranging techniques can provide accurate distance measurement under line-of-sight (LOS) conditions. However, various walls and obstacles in indoor non-LOS (NLOS) environments, which obstruct the direct propagation of UWB signals, can generate significant ranging errors. Due to the complex through-wall UWB signal propagation, most conventional studies simplify the ranging error model by assuming that the incidence angle is zero or the relative permittivity\u27s for different walls are the same to improve the through-wall UWB localization performance. Considering walls are different in realistic settings, this article presents a through-multiple-wall NLOS mitigation method for UWB indoor positioning. First, spatial geometric equilibrium equations of UWB through-wall propagation and a numerical method are developed for the precise modeling of UWB through-wall ranging errors. Then, calculated error maps are determined numerically without field measurements. Finally, the determined error maps are combined with a gray wolf optimization algorithm for localization. The proposed method is evaluated via field experiments with four rooms, three walls, and six penetration cases. The results demonstrate that the method can strongly mitigate the multi-wall. NLOS effects on the performance of UWB positioning systems. This solution can reduce project costs and number of power supplies for UWB indoor positioning applications

Indoor Positioning for BIM

Building Informational Modeling (BIM) is very popular in the construction industry in Norway today, and Omega 365 has created a suite of tools for BIM, including a 3D visualising tool for 3D models of buildings, called a BIMViewer. This tool exists in multiple forms, and one of them is an app for mobile phones, which construction workers carry with them on construction sites. When determining one's own position in the BIMViewer, it may take time to find and select the correct position. This study aims to create a feature for the BIMViewer using new technology, IEEE802.11mc and comparing it with an old method, Wi-Fi received signal strength (RSS) with the Log Distance Path Loss model. In addition, GPS was tried in order to prove it was not usable for this use case and in order to compare it with the other two methods. The main goal is to find a method that is cheap for clients to implement in regards to equipment and installation, but is precise enough to provide a good user experience. Three experiments were conducted for this study, one using only GPS and two for the other two methods. One experiment used only a single floor and the other used two floors. Both of these experiments used only 6 access points and were conducted at NyeSUS, the new hospital in Stavanger which was an active construction zone during the experiments. The experiments showed that GPS was a bad choice for the use case and that both the other methods were usable. The round trip time (RTT) method, which used the IEEE802.11mc measurements was more precise than the RSS method, however suffered from the need for more access points than the RSS method. This study concludes that both the RTT and the RSS methods may be usable, however some improvements would be needed for a truly good user experience. The study also suggests that a mix of the two methods may be beneficial

An Efficient and Accurate Indoor Positioning System

In this thesis, an indoor localization method using on-line independent support vector machine (OISVM) classification method and under-sampling techniques is proposed. The proposed positioning method is based on the received signal strength indicator (RSSI) of Wi-Fi signals. A new under-sampling algorithm is developed to address the imbalanced data problem associated with the OISVM, and a kernel function parameter selection algorithm is introduced for the training process. The time complexity of both the training process and the prediction process are decreased. Comparative experimental results indicate that the training speed and the prediction speed are improved by at least 10 times and 5 times, respectively. Furthermore, through on-line learning, the estimation error is decreased by 0.8m. Such an improvement makes the proposed method an ideal indoor positioning solution for portable devices where the processing power and the memory capacity are limited. A new Particle Filter (PF) scheme for indoor localization using Wi-Fi received signal strength indicator (RSSI) and inertial sensor measurements has also been presented. RSSI is affected significantly by multipath fading, building structure and obstacles in indoor environments. The information provided by inertial sensors combined with the proposed particle filter are used to develop a positioning algorithm supporting a smooth and stable localization experience. To differentiate similar fingerprints, a single-hidden layer feedforward networks (SLFNs) is used to model the multiple probabilistic estimations and to improve the performance of the PF. A new initialization algorithm using Random Sample Consensus (RANSAC) has also been presented to reduce the convergence time. Experimental measurements were carried out to determine the performance of the proposed algorithm. The results indicate that the positioning error falls to less than 1.2 (m)

An Investigation of Indoor Positioning Systems and their Applications

PhDActivities of Daily Living (ADL) are important indicators of both cognitive and physical well-being in healthy and ill humans. There is a range of methods to recognise ADLs, each with its own limitations. The focus of this research was on sensing location-driven activities, in which ADLs are derived from location sensed using Radio Frequency (RF, e.g., WiFi or BLE), Magnetic Field (MF) and light (e.g., Lidar) measurements in three different environments. This research discovered that different environments can have different constraints and requirements. It investigated how to improve the positioning accuracy and hence how to improve the ADL recognition accuracy. There are several challenges that need to be addressed in order to do this. First, RF location fingerprinting is affected by the heterogeneity smartphones and their orientation with respect to transmitters, increasing the location determination error. To solve this, a novel Received Signal Strength Indication (RSSI) ranking based location fingerprinting methods that use Kendall Tau Correlation Coefficient (KTCC) and Convolutional Neural Networks (CNN) are proposed to correlate a signal position to pre-defined Reference Points (RPs) or fingerprints, more accurately, The accuracy has increased by up to 25.8% when compared to using Euclidean Distance (ED) based Weighted K-Nearest Neighbours Algorithm (WKNN). Second, the use of MF measurements as fingerprints can overcome some additional RF fingerprinting challenges, as MF measurements are far more invariant to static and dynamic physical objects that affect RF transmissions. Hence, a novel fast path matching data algorithm for an MF sensor combined with an Inertial Measurement Unit (IMU) to determine direction was researched and developed. It can achieve an average of 1.72 m positioning accuracy when the user walks far fewer (5) steps. Third, a device-free or off-body novel location-driven ADL method based upon 2D Lidar was investigated. An innovative method for recognising daily activities using a Seq2Seq model to analyse location data from a low-cost rotating 2D Lidar is proposed. It provides an accuracy of 88% when recognising 17 targeted ADLs. These proposed methods in this thesis have been validated in real environments.Chinese Scholarship Counci

Sensors and Systems for Indoor Positioning

This reprint is a reprint of the articles that appeared in Sensors' (MDPI) Special Issue on “Sensors and Systems for Indoor Positioning". The published original contributions focused on systems and technologies to enable indoor applications

Off-line evaluation of indoor positioning systems in different scenarios: the experiences from IPIN 2020 competition

Every year, for ten years now, the IPIN competition has aimed at evaluating real-world indoor localisation systems by testing them in a realistic environment, with realistic movement, using the EvAAL framework. The competition provided a unique overview of the state-of-the-art of systems, technologies, and methods for indoor positioning and navigation purposes. Through fair comparison of the performance achieved by each system, the competition was able to identify the most promising approaches and to pinpoint the most critical working conditions. In 2020, the competition included 5 diverse off-site off-site Tracks, each resembling real use cases and challenges for indoor positioning. The results in terms of participation and accuracy of the proposed systems have been encouraging. The best performing competitors obtained a third quartile of error of 1 m for the Smartphone Track and 0.5 m for the Foot-mounted IMU Track. While not running on physical systems, but only as algorithms, these results represent impressive achievements.Track 3 organizers were supported by the European Union’s Horizon 2020 Research and Innovation programme under the Marie Skłodowska Curie Grant 813278 (A-WEAR: A network for dynamic WEarable Applications with pRivacy constraints), MICROCEBUS (MICINN, ref. RTI2018-095168-B-C55, MCIU/AEI/FEDER UE), INSIGNIA (MICINN ref. PTQ2018-009981), and REPNIN+ (MICINN, ref. TEC2017-90808-REDT). We would like to thanks the UJI’s Library managers and employees for their support while collecting the required datasets for Track 3. Track 5 organizers were supported by JST-OPERA Program, Japan, under Grant JPMJOP1612. Track 7 organizers were supported by the Bavarian Ministry for Economic Affairs, Infrastructure, Transport and Technology through the Center for Analytics-Data-Applications (ADA-Center) within the framework of “BAYERN DIGITAL II. ” Team UMinho (Track 3) was supported by FCT—Fundação para a Ciência e Tecnologia within the R&D Units Project Scope under Grant UIDB/00319/2020, and the Ph.D. Fellowship under Grant PD/BD/137401/2018. Team YAI (Track 3) was supported by the Ministry of Science and Technology (MOST) of Taiwan under Grant MOST 109-2221-E-197-026. Team Indora (Track 3) was supported in part by the Slovak Grant Agency, Ministry of Education and Academy of Science, Slovakia, under Grant 1/0177/21, and in part by the Slovak Research and Development Agency under Contract APVV-15-0091. Team TJU (Track 3) was supported in part by the National Natural Science Foundation of China under Grant 61771338 and in part by the Tianjin Research Funding under Grant 18ZXRHSY00190. Team Next-Newbie Reckoners (Track 3) were supported by the Singapore Government through the Industry Alignment Fund—Industry Collaboration Projects Grant. This research was conducted at Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU), which is a collaboration between Singapore Telecommunications Limited (Singtel) and Nanyang Technological University (NTU). Team KawaguchiLab (Track 5) was supported by JSPS KAKENHI under Grant JP17H01762. Team WHU&AutoNavi (Track 6) was supported by the National Key Research and Development Program of China under Grant 2016YFB0502202. Team YAI (Tracks 6 and 7) was supported by the Ministry of Science and Technology (MOST) of Taiwan under Grant MOST 110-2634-F-155-001

Off-Line Evaluation of Indoor Positioning Systems in Different Scenarios: The Experiences From IPIN 2020 Competition

Every year, for ten years now, the IPIN competition has aimed at evaluating real-world indoor localisation systems by testing them in a realistic environment, with realistic movement, using the EvAAL framework. The competition provided a unique overview of the state-of-the-art of systems, technologies, and methods for indoor positioning and navigation purposes. Through fair comparison of the performance achieved by each system, the competition was able to identify the most promising approaches and to pinpoint the most critical working conditions. In 2020, the competition included 5 diverse off-site off-site Tracks, each resembling real use cases and challenges for indoor positioning. The results in terms of participation and accuracy of the proposed systems have been encouraging. The best performing competitors obtained a third quartile of error of 1 m for the Smartphone Track and 0.5 m for the Foot-mounted IMU Track. While not running on physical systems, but only as algorithms, these results represent impressive achievements