Very Low Power Neural Network FPGA Accelerators for Tag-Less Remote Person Identification Using Capacitive Sensors

Human detection, identification, and monitoring are essential for many applications aiming to make smarter the indoor environments, where most people spend much of their time (like home, office, transportation, or public spaces). The capacitive sensors can meet stringent privacy, power, cost, and unobtrusiveness requirements, they do not rely on wearables or specific human interactions, but they may need significant on-board data processing to increase their performance. We comparatively analyze in terms of overall processing time and energy several data processing implementations of multilayer perceptron neural networks (NNs) on board capacitive sensors. The NN architecture, optimized using augmented experimental data, consists of six 17-bit inputs, two hidden layers with eight neurons each, and one four-bit output. For the software (SW) NN implementation, we use two STMicroelectronics STM32 low-power ARM microcontrollers (MCUs): one MCU optimized for power and one for performance. For hardware (HW) implementations, we use four ultralow-power field-programmable gate arrays (FPGAs), with different sizes, dedicated computation blocks, and data communication interfaces (one FPGA from the Lattice iCE40 family and three FPGAs from the Microsemi IGLOO family). Our shortest SW implementation latency is 54.4 µs and the lowest energy per inference is 990 nJ, while the shortest HW implementation latency is 1.99 µs and the lowest energy is 39 nJ (including the data transfer between MCU and FPGA). The FPGAs active power ranges between 6.24 and 34.7 mW, while their static power is between 79 and 277 µW. They compare very favorably with the static power consumption of Xilinx and Altera low-power device families, which is around 40 mW. The experimental results show that NN inferences offloaded to external FPGAs have lower latency and energy than SW ones (even when using HW multipliers), and the FPGAs with dedicated computational blocks (multiply-accumulate) perform best

Neural Networks for Indoor Human Activity Reconstructions

Low cost, ubiquitous, tagless, and privacy aware indoor monitoring is essential to many existing or future applications, such as assisted living of elderly persons. We explore how well different types of neural networks in basic configurations can extract location and movement information from noisy experimental data (with both high-pitch and slow drift noise) obtained from capacitive sensors operating in loading mode at ranges much longer that the diagonal of their plates. Through design space exploration, we optimize and analyze the location and trajectory tracking inference performance of multilayer perceptron (MLP), autoregressive feedforward, 1D Convolutional (1D-CNN), and Long-Short Term Memory (LSTM) neural networks on experimental data collected using four capacitive sensors with 16 cm x 16 cm plates deployed on the boundaries of a 3 m x 3 m open space in our laboratory. We obtain the minimum error using a 1D-CNN [0.251 m distance Root Mean Square Error (RMSE) and 0.307 m Average Distance Error (ADE)] and the smoothest trajectory inference using an LSTM, albeit with higher localization errors (0.281 m RMSE and 0.326 m ADE). 1D Convolutional and window-based neural networks have best inference accuracy and smoother trajectory reconstruction. LSTMs seem to infer best the person movement dynamics

Device-free indoor localisation with non-wireless sensing techniques : a thesis by publications presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Electronics and Computer Engineering, Massey University, Albany, New Zealand

Global Navigation Satellite Systems provide accurate and reliable outdoor positioning to support a large number of applications across many sectors. Unfortunately, such systems do not operate reliably inside buildings due to the signal degradation caused by the absence of a clear line of sight with the satellites. The past two decades have therefore seen intensive research into the development of Indoor Positioning System (IPS). While considerable progress has been made in the indoor localisation discipline, there is still no widely adopted solution. The proliferation of Internet of Things (IoT) devices within the modern built environment provides an opportunity to localise human subjects by utilising such ubiquitous networked devices. This thesis presents the development, implementation and evaluation of several passive indoor positioning systems using ambient Visible Light Positioning (VLP), capacitive-flooring, and thermopile sensors (low-resolution thermal cameras). These systems position the human subject in a device-free manner (i.e., the subject is not required to be instrumented). The developed systems improve upon the state-of-the-art solutions by offering superior position accuracy whilst also using more robust and generalised test setups. The developed passive VLP system is one of the first reported solutions making use of ambient light to position a moving human subject. The capacitive-floor based system improves upon the accuracy of existing flooring solutions as well as demonstrates the potential for automated fall detection. The system also requires very little calibration, i.e., variations of the environment or subject have very little impact upon it. The thermopile positioning system is also shown to be robust to changes in the environment and subjects. Improvements are made over the current literature by testing across multiple environments and subjects whilst using a robust ground truth system. Finally, advanced machine learning methods were implemented and benchmarked against a thermopile dataset which has been made available for other researchers to use

Enhanced Exploration of Neural Network Models for Indoor Human Monitoring

Indoor human monitoring can enable or enhance a wide range of applications, from medical to security and home or building automation. For effective ubiquitous deployment, the monitoring system should be easy to install and unobtrusive, reliable, low cost, tagless, and privacy-aware. Long-range capacitive sensors are good candidates, but they can be susceptible to environmental electromagnetic noise and require special signal processing. Neural networks (NNs), especially 1D convolutional neural networks (1D-CNNs), excel at extracting information and rejecting noise, but they lose important relationships in max/average pooling operations. We investigate the performance of NN architectures for time series analysis without this shortcoming, the capsule networks that use dynamic routing, and the temporal convolutional networks (TCNs) that use dilated convolutions to preserve input resolution across layers and extend their receptive field with fewer layers. The networks are optimized for both inference accuracy and resource consumption using two independent state-of-the-art methods, neural architecture search and knowledge distillation. Experimental results show that the TCN architecture performs the best, achieving 12.7% lower inference loss with 73.3% less resource consumption than the best 1D-CNN when processing noisy capacitive sensor data for indoor human localization and tracking

Machine Learning Techniques for Device-Free Indoor Person Tracking

L'abstract è presente nell'allegato / the abstract is in the attachmen

Neural Networks for Indoor Person Tracking With Infrared Sensors

Indoor localization has many pervasive applications, like energy management, health monitoring, and security. Tagless localization detects directly the human body, like passive infrared sensing, and is the most amenable to different users and use cases. We evaluate the localization and tracking performance, as well as resource and processing requirements of various neural network (NN) types using directly the data from a low resolution 16-pixel thermopile sensor array in a 3 m x 3 m room. Out of the multilayer perceptron, autoregressive, 1D-CNN, and LSTM NN architectures that we test, the latter require more resources but can accurately locate and capture best the person movement dynamics, while the 1D-CNN provides the best compromise between localization accuracy (9.6 cm RMSE) and movement tracking smoothness with the least resources, and seem more suited for embedded applications

A Tagless Indoor Localization System Based on Capacitive Sensing Technology

Accurate indoor person localization is essential for several services, such as assisted living. We introduce a tagless indoor person localization system based on capacitive sensing and localization algorithms that can determine the location with less than 0.2 m average error in a 3 m × 3 m room and has recall and precision better than 70%. We also discuss the effects of various noise types on the measurements and ways to reduce them using filters suitable for on-sensor implementation to lower communication energy consumption. We also compare the performance of several standard localization algorithms in terms of localization error, recall, precision, and accuracy of detection of the movement trajectory

Body-Area Capacitive or Electric Field Sensing for Human Activity Recognition and Human-Computer Interaction: A Comprehensive Survey

Due to the fact that roughly sixty percent of the human body is essentially composed of water, the human body is inherently a conductive object, being able to, firstly, form an inherent electric field from the body to the surroundings and secondly, deform the distribution of an existing electric field near the body. Body-area capacitive sensing, also called body-area electric field sensing, is becoming a promising alternative for wearable devices to accomplish certain tasks in human activity recognition and human-computer interaction. Over the last decade, researchers have explored plentiful novel sensing systems backed by the body-area electric field. On the other hand, despite the pervasive exploration of the body-area electric field, a comprehensive survey does not exist for an enlightening guideline. Moreover, the various hardware implementations, applied algorithms, and targeted applications result in a challenging task to achieve a systematic overview of the subject. This paper aims to fill in the gap by comprehensively summarizing the existing works on body-area capacitive sensing so that researchers can have a better view of the current exploration status. To this end, we first sorted the explorations into three domains according to the involved body forms: body-part electric field, whole-body electric field, and body-to-body electric field, and enumerated the state-of-art works in the domains with a detailed survey of the backed sensing tricks and targeted applications. We then summarized the three types of sensing frontends in circuit design, which is the most critical part in body-area capacitive sensing, and analyzed the data processing pipeline categorized into three kinds of approaches. Finally, we described the challenges and outlooks of body-area electric sensing

Assessing the feasibility of augmenting fall detection systems by relying on UWB-based position tracking and a home robot

Falls in the home environment are a primary cause of injury in older adults. According to the U.S. Centers for Disease Control and Prevention, every year, one in four adults 65 years of age and older reports experiencing a fall. A variety of different technologies have been proposed to detect fall events. However, the need to detect all fall instances (i.e., to avoid false negatives) has led to the development of systems marked by high sensitivity and hence a significant number of false alarms. The occurrence of false alarms causes frequent and unnecessary calls to emergency response centers, which are critical resources that should be utilized only when necessary. Besides, false alarms decrease the level of confidence of end-users in the fall detection system with a negative impact on their compliance with using the system (e.g., wearing the sensor enabling the detection of fall events). Herein, we present a novel approach aimed to augment traditional fall detection systems that rely on wearable sensors and fall detection algorithms. The proposed approach utilizes a UWB-based tracking system and a home robot. When the fall detection system generates an alarm, the alarm is relayed to a base station that utilizes a UWB-based tracking system to identify where the older adult and the robot are so as to enable navigating the environment using the robot and reaching the older adult to check if he/she experienced a fall. This approach prevents unnecessary calls to emergency response centers while enabling a tele-presence using the robot when appropriate. In this paper, we report the results of a novel fall detection algorithm, the characteristics of the alarm notification system, and the accuracy of the UWB-based tracking system that we implemented. The fall detection algorithm displayed a sensitivity of 99.0% and a specificity of 97.8%. The alarm notification system relayed all simulated alarm notification instances with a maximum delay of 106 ms. The UWB-based tracking system was found to be suitable to locate radio tags both in line-of-sight and in no-line-of-sight conditions. This result was obtained by using a machine learning-based algorithm that we developed to detect and compensate for the multipath effect in no-line-of-sight conditions. When using this algorithm, the error affecting the estimated position of the radio tags was smaller than 0.2 m, which is satisfactory for the application at hand