Accurate pedestrian indoor navigation by tightly coupling foot-mounted IMU and RFID measurements

We present a new method to accurately locate persons indoors by fusing inertial navigation system (INS) techniques with active RFID technology. A foot-mounted inertial measuring units (IMUs)-based position estimation method, is aided by the received signal strengths (RSSs) obtained from several active RFID tags placed at known locations in a building. In contrast to other authors that integrate IMUs and RSS with a loose Kalman filter (KF)-based coupling (by using the residuals of inertial- and RSS-calculated positions), we present a tight KF-based INS/RFID integration, using the residuals between the INS-predicted reader-to-tag ranges and the ranges derived from a generic RSS path-loss model. Our approach also includes other drift reduction methods such as zero velocity updates (ZUPTs) at foot stance detections, zero angular-rate updates (ZARUs) when the user is motionless, and heading corrections using magnetometers. A complementary extended Kalman filter (EKF), throughout its 15-element error state vector, compensates the position, velocity and attitude errors of the INS solution, as well as IMU biases. This methodology is valid for any kind of motion (forward, lateral or backward walk, at different speeds), and does not require an offline calibration for the user gait. The integrated INS+RFID methodology eliminates the typical drift of IMU-alone solutions (approximately 1% of the total traveled distance), resulting in typical positioning errors along the walking path (no matter its length) of approximately 1.5 m

Multimodal Sensor Data Integration for Indoor Positioning in Ambient-Assisted Living Environments

A reliable Indoor Positioning System (IPS) is a crucial part of the Ambient-Assisted Living (AAL) concept. The use of Wi-Fi fingerprinting techniques to determine the location of the user, based on the Received Signal Strength Indication (RSSI) mapping, avoids the need to deploy a dedicated positioning infrastructure but comes with its own issues. Heterogeneity of devices and RSSI variability in space and time due to environment changing conditions pose a challenge to positioning systems based on this technique. The primary purpose of this research is to examine the viability of leveraging other sensors in aiding the positioning system to provide more accurate predictions. In particular, the experiments presented in this work show that Inertial Motion Units (IMU), which are present by default in smart devices such as smartphones or smartwatches, can increase the performance of Indoor Positioning Systems in AAL environments. Furthermore, this paper assesses a set of techniques to predict the future performance of the positioning system based on the training data, as well as complementary strategies such as data scaling and the use of consecutive Wi-Fi scanning to further improve the reliability of the IPS predictions. This research shows that a robust positioning estimation can be derived from such strategies

Human Motion Analysis with Wearable Inertial Sensors

High-resolution, quantitative data obtained by a human motion capture system can be used to better understand the cause of many diseases for effective treatments. Talking about the daily care of the aging population, two issues are critical. One is to continuously track motions and position of aging people when they are at home, inside a building or in the unknown environment; the other is to monitor their health status in real time when they are in the free-living environment. Continuous monitoring of human movement in their natural living environment potentially provide more valuable feedback than these in laboratory settings. However, it has been extremely challenging to go beyond laboratory and obtain accurate measurements of human physical activity in free-living environments. Commercial motion capture systems produce excellent in-studio capture and reconstructions, but offer no comparable solution for acquisition in everyday environments. Therefore in this dissertation, a wearable human motion analysis system is developed for continuously tracking human motions, monitoring health status, positioning human location and recording the itinerary. In this dissertation, two systems are developed for seeking aforementioned two goals: tracking human body motions and positioning a human. Firstly, an inertial-based human body motion tracking system with our developed inertial measurement unit (IMU) is introduced. By arbitrarily attaching a wearable IMU to each segment, segment motions can be measured and translated into inertial data by IMUs. A human model can be reconstructed in real time based on the inertial data by applying high efficient twists and exponential maps techniques. Secondly, for validating the feasibility of developed tracking system in the practical application, model-based quantification approaches for resting tremor and lower extremity bradykinesia in Parkinson’s disease are proposed. By estimating all involved joint angles in PD symptoms based on reconstructed human model, angle characteristics with corresponding medical ratings are employed for training a HMM classifier for quantification. Besides, a pedestrian positioning system is developed for tracking user’s itinerary and positioning in the global frame. Corresponding tests have been carried out to assess the performance of each system

Indoor navigation for the visually impaired : enhancements through utilisation of the Internet of Things and deep learning

Wayfinding and navigation are essential aspects of independent living that heavily rely on the sense of vision. Walking in a complex building requires knowing exact location to find a suitable path to the desired destination, avoiding obstacles and monitoring orientation and movement along the route. People who do not have access to sight-dependent information, such as that provided by signage, maps and environmental cues, can encounter challenges in achieving these tasks independently. They can rely on assistance from others or maintain their independence by using assistive technologies and the resources provided by smart environments. Several solutions have adapted technological innovations to combat navigation in an indoor environment over the last few years. However, there remains a significant lack of a complete solution to aid the navigation requirements of visually impaired (VI) people. The use of a single technology cannot provide a solution to fulfil all the navigation difficulties faced. A hybrid solution using Internet of Things (IoT) devices and deep learning techniques to discern the patterns of an indoor environment may help VI people gain confidence to travel independently. This thesis aims to improve the independence and enhance the journey of VI people in an indoor setting with the proposed framework, using a smartphone. The thesis proposes a novel framework, Indoor-Nav, to provide a VI-friendly path to avoid obstacles and predict the user s position. The components include Ortho-PATH, Blue Dot for VI People (BVIP), and a deep learning-based indoor positioning model. The work establishes a novel collision-free pathfinding algorithm, Orth-PATH, to generate a VI-friendly path via sensing a grid-based indoor space. Further, to ensure correct movement, with the use of beacons and a smartphone, BVIP monitors the movements and relative position of the moving user. In dark areas without external devices, the research tests the feasibility of using sensory information from a smartphone with a pre-trained regression-based deep learning model to predict the user s absolute position. The work accomplishes a diverse range of simulations and experiments to confirm the performance and effectiveness of the proposed framework and its components. The results show that Indoor-Nav is the first type of pathfinding algorithm to provide a novel path to reflect the needs of VI people. The approach designs a path alongside walls, avoiding obstacles, and this research benchmarks the approach with other popular pathfinding algorithms. Further, this research develops a smartphone-based application to test the trajectories of a moving user in an indoor environment

Multimodal Sensing for Robust and Energy-Efficient Context Detection with Smart Mobile Devices

Adoption of smart mobile devices (smartphones, wearables, etc.) is rapidly growing. There are already over 2 billion smartphone users worldwide [1] and the percentage of smartphone users is expected to be over 50% in the next five years [2]. These devices feature rich sensing capabilities which allow inferences about mobile device user’s surroundings and behavior. Multiple and diverse sensors common on such mobile devices facilitate observing the environment from different perspectives, which helps to increase robustness of inferences and enables more complex context detection tasks. Though a larger number of sensing modalities can be beneficial for more accurate and wider mobile context detection, integrating these sensor streams is non-trivial. This thesis presents how multimodal sensor data can be integrated to facilitate ro- bust and energy efficient mobile context detection, considering three important and challenging detection tasks: indoor localization, indoor-outdoor detection and human activity recognition. This thesis presents three methods for multimodal sensor inte- gration, each applied for a different type of context detection task considered in this thesis. These are gradually decreasing in design complexity, starting with a solution based on an engineering approach decomposing context detection to simpler tasks and integrating these with a particle filter for indoor localization. This is followed by man- ual extraction of features from different sensors and using an adaptive machine learn- ing technique called semi-supervised learning for indoor-outdoor detection. Finally, a method using deep neural networks capable of extracting non-intuitive features di- rectly from raw sensor data is used for human activity recognition; this method also provides higher degree of generalization to other context detection tasks. Energy efficiency is an important consideration in general for battery powered mo- bile devices and context detection is no exception. In the various context detection tasks and solutions presented in this thesis, particular attention is paid to this issue by relying largely on sensors that consume low energy and on lightweight computations. Overall, the solutions presented improve on the state of the art in terms of accuracy and robustness while keeping the energy consumption low, making them practical for use on mobile devices

Context Awareness for Navigation Applications

This thesis examines the topic of context awareness for navigation applications and asks the question, “What are the benefits and constraints of introducing context awareness in navigation?” Context awareness can be defined as a computer’s ability to understand the situation or context in which it is operating. In particular, we are interested in how context awareness can be used to understand the navigation needs of people using mobile computers, such as smartphones, but context awareness can also benefit other types of navigation users, such as maritime navigators. There are countless other potential applications of context awareness, but this thesis focuses on applications related to navigation. For example, if a smartphone-based navigation system can understand when a user is walking, driving a car, or riding a train, then it can adapt its navigation algorithms to improve positioning performance. We argue that the primary set of tools available for generating context awareness is machine learning. Machine learning is, in fact, a collection of many different algorithms and techniques for developing “computer systems that automatically improve their performance through experience” [1]. This thesis examines systematically the ability of existing algorithms from machine learning to endow computing systems with context awareness. Specifically, we apply machine learning techniques to tackle three different tasks related to context awareness and having applications in the field of navigation: (1) to recognize the activity of a smartphone user in an indoor office environment, (2) to recognize the mode of motion that a smartphone user is undergoing outdoors, and (3) to determine the optimal path of a ship traveling through ice-covered waters. The diversity of these tasks was chosen intentionally to demonstrate the breadth of problems encompassed by the topic of context awareness. During the course of studying context awareness, we adopted two conceptual “frameworks,” which we find useful for the purpose of solidifying the abstract concepts of context and context awareness. The first such framework is based strongly on the writings of a rhetorician from Hellenistic Greece, Hermagoras of Temnos, who defined seven elements of “circumstance”. We adopt these seven elements to describe contextual information. The second framework, which we dub the “context pyramid” describes the processing of raw sensor data into contextual information in terms of six different levels. At the top of the pyramid is “rich context”, where the information is expressed in prose, and the goal for the computer is to mimic the way that a human would describe a situation. We are still a long way off from computers being able to match a human’s ability to understand and describe context, but this thesis improves the state-of-the-art in context awareness for navigation applications. For some particular tasks, machine learning has succeeded in outperforming humans, and in the future there are likely to be tasks in navigation where computers outperform humans. One example might be the route optimization task described above. This is an example of a task where many different types of information must be fused in non-obvious ways, and it may be that computer algorithms can find better routes through ice-covered waters than even well-trained human navigators. This thesis provides only preliminary evidence of this possibility, and future work is needed to further develop the techniques outlined here. The same can be said of the other two navigation-related tasks examined in this thesis

Indoor positioning system for wireless sensor networks

Tese de Doutoramento - Programa Doutoral em Engenharia Electrónica e ComputadoresPositioning technologies are ubiquitous nowadays. From the implementation of the global positioning system (GPS) until now, its evolution, acceptance and spread has been unanimous, due to the underlying advantages the system brings. Currently, these systems are present in many different scenarios, from the home to the movie theatre, at work, during a walk in the park. Many applications provide useful information, based on the current position of the user, in order to provide results of interest. Positioning systems can be implemented in a wide range of contexts: in hospitals to locate equipment and guide patients to the necessary resources, or in public spaces like museums, to guide tourists during visits. They can also be used in a gymnasium to point the user to his next workout machine and, simultaneously, gather information regarding his fitness plan. In a congress or conference, the positioning system can be used to provide information to its participants about the on-going presentations. Devices can also be monitored to prevent thefts. Privacy and security issues are also important in positioning systems. A user might not want to be localized or its location to be known, permanently or during a time interval, in different locations. This information is therefore sensitive to the user and influences directly the acceptance of the system itself. Concerning outdoor systems, GPS is in fact the system of reference. However, this system cannot be used in indoor environment, due to the high attenuation of the satellite signals from non-line-of-sight conditions. Another issue related to GPS is the power consumption. The integration of these devices with wireless sensor networks becomes prohibitive, due to the low power consumption profile associated with devices in this type of networks. As such, this work proposes an indoor positioning system for wireless sensor networks, having in consideration the low energy consumption and low computational capacity profile. The proposed indoor positioning system is composed of two modules: the received signal strength positioning module and the stride and heading positioning module. For the first module, an experimental performance comparison between several received signal strength based algorithms was conducted in order to assess its performance in a predefined indoor environment. Modifications to the algorithm with higher performance were implemented and evaluated, by introducing a model of the effect of the human body in the received signal strength. In the case of the second module, a stride and heading system was proposed, which comprises two subsystems: the stride detection and stride length estimation system to detect strides and infer the travelled distance, and an attitude and heading reference system to provide the full three-dimensional orientation stride-by-stride. The stride detection enabled the identification of the gait cycle and detected strides with an error percentage between 0% and 0.9%. For the stride length estimation two methods were proposed, a simplified method, and an improved method with higher computational requirements than the former. The simplified method estimated the total distance with an error between 6.7% and 7.7% of total travelled distance. The improved method achieved an error between 1.2% and 3.7%. Both the stride detection and the improved stride length estimation methods were compared to other methods in the literature with favourable results. For the second subsystem, this work proposed a quaternion-based complementary filter. A generic formulation allows a simple parameterization of the filter, according to the amount of external influences (accelerations and magnetic interferences) that are expected, depending on the location that the device is to be attached on the human body. The generic formulation enables the inclusion/exclusion of components, thus allowing design choices according to the needs of applications in wireless sensor networks. The proposed method was compared to two other existing solutions in terms of robustness to interferences and execution time, also presenting a favourable outcome.Os sistemas de posicionamento fazem parte do quotidiano. Desde a implementação do sistema GPS (Global Positioning System) até aos dias que correm, a evolução, aceitação e disseminação destes sistemas foi unânime, derivada das vantagens subjacentes da sua utilização. Hoje em dia, eles estão presentes nos mais variados cenários, desde o lar até́ à sala de cinema, no trabalho, num passeio ao ar livre. São várias as aplicações que nos fornecem informação útil, usando como base a descrição da posição atual, de modo a produzir resultados de maior interesse para os utilizadores. Os sistemas de posicionamento podem ser implementados nos mais variados contextos, como por exemplo: nos hospitais, para localizar equipamento e guiar os pacientes aos recursos necessários, ou nas grandes superfícies públicas, como por exemplo museus, para guiar os turistas durante as visitas. Podem ser igualmente utilizados num ginásio para indicar ao utilizador qual a máquina para onde se deve dirigir durante o seu treino e, simultaneamente, obter informação acerca desta mesma máquina. Num congresso ou conferência, o sistema de localização pode ser utilizado para fornecer informação aos seus participantes sobre as apresentações que estão a decorrer no momento. Os dispositivos também podem ser monitorizados para prevenir roubos. Existem também questões de privacidade e segurança associados aos sistemas de posicionamento. Um utilizador poderá não desejar ser localizado ou que a sua localização seja conhecida, permanentemente ou num determinado intervalo de tempo, num ou em vários locais. Esta informação é por isso sensível ao utilizador e influencia diretamente a aceitação do próprio sistema. No que diz respeito aos sistemas utilizados no exterior, o GPS (ou posicionamento por satélite) é de facto o sistema mais utilizado. No entanto, em ambiente interior este sistema não pode ser usado, por causa da grande atenuação dos sinais provenientes dos satélites devido à falta de linha de vista. Um outro problema associado ao recetor GPS está relacionado com as suas características elétricas, nomeadamente os consumos energéticos. A integração destes dispositivos nas redes de sensores sem fios torna-se proibitiva, devido ao perfil de baixo consumo associado a estas redes. Este trabalho propõe um sistema de posicionamento para redes de sensores sem fio em ambiente interior, tendo em conta o perfil de baixo consumo de potência e baixa capacidade de processamento. O sistema proposto é constituído por dois módulos: o modulo de posicionamento por potência de sinal recebido e o módulo de navegação inercial pedestre. Para o primeiro módulo foi feita uma comparação experimental entre vários algoritmos que utilizam a potência do sinal recebido, de modo a avaliar a sua utilização num ambiente interior pré-definido. Ao algoritmo com melhor prestação foram implementadas e testadas modificações, utilizando um modelo do efeito do corpo na potência do sinal recebido. Para o segundo módulo foi proposto um sistema de navegação inercial pedestre. Este sistema é composto por dois subsistemas: o subsistema de deteção de passos e estimação de distância percorrida; e o subsistema de orientação que fornece a direção do movimento do utilizador, passo a passo. O sistema de deteção de passos proposto permite a identificação das fases da marcha, detetando passos com um erro entre 0% e 0.9%. Para o sistema de estimação da distância foram propostos dois métodos: um método simplificado de baixa complexidade e um método melhorado, mas com maiores requisitos computacionais quando comparado com o primeiro. O método simplificado estima a distância total com erros entre 6.7% e 7.7% da distância percorrida. O método melhorado por sua vez alcança erros entre 1.2% e 3.7%. Ambos os sistemas foram comparados com outros sistemas da literatura apresentando resultados favoráveis. Para o sistema de orientação, este trabalho propõe um filtro complementar baseado em quaterniões. É utilizada uma formulação genérica que permite uma parametrização simples do filtro, de acordo com as influências externas (acelerações e interferências magnéticas) que são expectáveis, dependendo da localização onde se pretende colocar o dispositivo no corpo humano. O algoritmo desenvolvido permite a inclusão/exclusão de componentes, permitindo por isso liberdade de escolha para melhor satisfazer as necessidades das aplicações em redes de sensores sem fios. O método proposto foi comparado com outras soluções em termos de robustez a interferências e tempo de execução, apresentando também resultados positivos

Mobile localization : approach and applications

textLocalization is critical to a number of wireless network applications. In many situations GPS is not suitable. This dissertation (i) develops novel localization schemes for wireless networks by explicitly incorporating mobility information and (ii) applies localization to physical analytics i.e., understanding shoppers' behavior within retail spaces by leveraging inertial sensors, Wi-Fi and vision enabled by smart glasses. More specifically, we first focus on multi-hop mobile networks, analyze real mobility traces and observe that they exhibit temporal stability and low-rank structure. Motivated by these observations, we develop novel localization algorithms to effectively capture and also adapt to different degrees of these properties. Using extensive simulations and testbed experiments, we demonstrate the accuracy and robustness of our new schemes. Second, we focus on localizing a single mobile node, which may not be connected with multiple nodes (e.g., without network connectivity or only connected with an access point). We propose trajectory-based localization using Wi-Fi or magnetic field measurements. We show that these measurements have the potential to uniquely identify a trajectory. We then develop a novel approach that leverages multi-level wavelet coefficients to first identify the trajectory and then localize to a point on the trajectory. We show that this approach is highly accurate and power efficient using indoor and outdoor experiments. Finally, localization is a critical step in enabling a lot of applications --- an important one is physical analytics. Physical analytics has the potential to provide deep-insight into shoppers' interests and activities and therefore better advertisements, recommendations and a better shopping experience. To enable physical analytics, we build ThirdEye system which first achieves zero-effort localization by leveraging emergent devices like the Google-Glass to build AutoLayout that fuses video, Wi-Fi, and inertial sensor data, to simultaneously localize the shoppers while also constructing and updating the product layout in a virtual coordinate space. Further, ThirdEye comprises of a range of schemes that use a combination of vision and inertial sensing to study mobile users' behavior while shopping, namely: walking, dwelling, gazing and reaching-out. We show the effectiveness of ThirdEye through an evaluation in two large retail stores in the United States.Computer Science