UWB and WiFi Systems as Passive Opportunistic Activity Sensing Radars

Human Activity Recognition (HAR) is becoming increasingly important in smart homes and healthcare applications such as assisted-living and remote health monitoring. In this paper, we use Ultra-Wideband (UWB) and commodity WiFi systems for the passive sensing of human activities. These systems are based on a receiver-only radar network that detects reflections of ambient Radio-Frequency (RF) signals from humans in the form of Channel Impulse Response (CIR) and Channel State Information (CSI). An experiment was performed whereby the transmitter and receiver were separated by a fixed distance in a Line-of-Sight (LoS) setting. Five activities were performed in between them, namely, sitting, standing, lying down, standing from the floor and walking. We use the high-resolution CIRs provided by the UWB modules as features in machine and deep learning algorithms for classifying the activities. Experimental results show that a classification performance with an F1-score as high as 95.53% is achieved using processed UWB CIR data as features. Furthermore, we analysed the classification performance in the same physical layout using CSI data extracted from a dedicated WiFi Network Interface Card (NIC). In this case, maximum F1-scores of 92.24% and 80.89% are obtained when amplitude CSI data and spectrograms are used as features, respectively

UWB and WiFi Systems as Passive Opportunistic Activity Sensing Radars

Human Activity Recognition (HAR) is becoming increasingly important in smart homes and healthcare applications such as assisted-living and remote health monitoring. In this paper, we use Ultra-Wideband (UWB) and commodity WiFi systems for the passive sensing of human activities. These systems are based on a receiver-only radar network that detects reflections of ambient Radio-Frequency (RF) signals from humans in the form of Channel Impulse Response (CIR) and Channel State Information (CSI). An experiment was performed whereby the transmitter and receiver were separated by a fixed distance in a Line-of-Sight (LoS) setting. Five activities were performed in between them, namely, sitting, standing, lying down, standing from the floor and walking. We use the high-resolution CIRs provided by the UWB modules as features in machine and deep learning algorithms for classifying the activities. Experimental results show that a classification performance with an F1-score as high as 95.53% is achieved using processed UWB CIR data as features. Furthermore, we analysed the classification performance in the same physical layout using CSI data extracted from a dedicated WiFi Network Interface Card (NIC). In this case, maximum F1-scores of 92.24% and 80.89% are obtained when amplitude CSI data and spectrograms are used as features, respectively

InContexto: Multisensor Architecture to Obtain People Context from Smartphones

The way users intectact with smartphones is changing after the improvements made in their embedded sensors. Increasingly, these devices are being employed as tools to observe individuals habits. Smartphones provide a great set of embedded sensors, such as accelerometer, digital compass, gyroscope, GPS, microphone, and camera. This paper aims to describe a distributed architecture, called inContexto, to recognize user context information using mobile phones. Moreover, it aims to infer physical actions performed by users such as walking, running, and still. Sensory data is collected by HTC magic application made in Android OS, and it was tested achieving about 97% of accuracy classifying five different actions (still, walking and running).This work was supported in part by Projects CICYT TIN2011-28620-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485), and DPS2008-07029- C02-02.Publicad

Fall Detection Using Channel State Information from WiFi Devices

Falls among the independently living elderly population are a major public health worry, leading to injuries, loss of confidence to live independently and even to death. Each year, one in three people aged 65 and older falls and one in five of them suffers fatal or non fatal injuries. Therefore, detecting a fall early and alerting caregivers can potentially save lives and increase the standard of living. Existing solutions, e.g. push-button, wearables, cameras, radar, pressure and vibration sensors, have limited public adoption either due to the requirement for wearing the device at all times or installing specialized and expensive infrastructure. In this thesis, a device-free, low cost indoor fall detection system using commodity WiFi devices is presented. The system uses physical layer Channel State Information (CSI) to detect falls. Commercial WiFi hardware is cheap and ubiquitous and CSI provides a wealth of information which helps in maintaining good fall detection accuracy even in challenging environments. The goals of the research in this thesis are the design, implementation and experimentation of a device-free fall detection system using CSI extracted from commercial WiFi devices. To achieve these objectives, the following contributions are made herein. A novel time domain human presence detection scheme is developed as a precursor to detecting falls. As the next contribution, a novel fall detection system is designed and developed. Finally, two main enhancements to the fall detection system are proposed to improve the resilience to changes in operating environment. Experiments were performed to validate system performance in diverse environments. It can be argued that through collection of real world CSI traces, understanding the behavior of CSI during human motion, the development of a signal processing tool-set to facilitate the recognition of falls and validation of the system using real world experiments significantly advances the state of the art by providing a more robust fall detection scheme

NON-CONTACT TECHNIQUES FOR HUMAN VITAL SIGN DETECTION AND GAIT ANALYSIS

Human vital signs including respiratory rate, heart rate, oxygen saturation, blood pressure, and body temperature are important physiological parameters that are used to track and monitor human health condition. Another important biological parameter of human health is human gait. Human vital sign detection and gait investigations have been attracted many scientists and practitioners in various fields such as sport medicine, geriatric medicine, bio-mechanic and bio-medical engineering and has many biological and medical applications such as diagnosis of health issues and abnormalities, elderly care and health monitoring, athlete performance analysis, and treatment of joint problems. Thoroughly tracking and understanding the normal motion of human limb joints can help to accurately monitor human subjects or patients over time to provide early flags of possible complications in order to aid in a proper diagnosis and development of future comprehensive treatment plans. With the spread of COVID-19 around the world, it has been getting more important than ever to employ technology that enables us to detect human vital signs in a non-contact way and helps protect both patients and healthcare providers from potentially life-threatening viruses, and have the potential to also provide a convenient way to monitor people health condition, remotely. A popular technique to extract biological parameters from a distance is to use cameras. Radar systems are another attractive solution for non-contact human vital signs monitoring and gait investigation that track and monitor these biological parameters without invading people privacy. The goal of this research is to develop non-contact methods that is capable of extracting human vital sign parameters and gait features accurately. To do that, in this work, optical systems including cameras and proper filters have been developed to extract human respiratory rate, heart rate, and oxygen saturation. Feasibility of blood pressure extraction using the developed optical technique has been investigated, too. Moreover, a wideband and low-cost radar system has been implemented to detect single or multiple human subject’s respiration and heart rate in dark or from behind the wall. The performance of the implemented radar system has been enhanced and it has been utilized for non-contact human gait analysis. Along with the hardware, advanced signal processing schemes have been enhanced and applied to the data collected using the aforementioned radar system. The data processing algorithms have been extended for multi-subject scenarios with high accuracy for both human vital sign detection and gait analysis. In addition, different configurations of this and high-performance radar system including mono-static and MIMO have been designed and implemented with great success. Many sets of exhaustive experiments have been conducted using different human subjects and various situations and accurate reference sensors have been used to validate the performance of the developed systems and algorithms

Edge Artificial Intelligence for Real-Time Target Monitoring

The key enabling technology for the exponentially growing cellular communications sector is location-based services. The need for location-aware services has increased along with the number of wireless and mobile devices. Estimation problems, and particularly parameter estimation, have drawn a lot of interest because of its relevance and engineers' ongoing need for higher performance. As applications expanded, a lot of interest was generated in the accurate assessment of temporal and spatial properties. In the thesis, two different approaches to subject monitoring are thoroughly addressed. For military applications, medical tracking, industrial workers, and providing location-based services to the mobile user community, which is always growing, this kind of activity is crucial. In-depth consideration is given to the viability of applying the Angle of Arrival (AoA) and Receiver Signal Strength Indication (RSSI) localization algorithms in real-world situations. We presented two prospective systems, discussed them, and presented specific assessments and tests. These systems were put to the test in diverse contexts (e.g., indoor, outdoor, in water...). The findings showed the localization capability, but because of the low-cost antenna we employed, this method is only practical up to a distance of roughly 150 meters. Consequently, depending on the use-case, this method may or may not be advantageous. An estimation algorithm that enhances the performance of the AoA technique was implemented on an edge device. Another approach was also considered. Radar sensors have shown to be durable in inclement weather and bad lighting conditions. Frequency Modulated Continuous Wave (FMCW) radars are the most frequently employed among the several sorts of radar technologies for these kinds of applications. Actually, this is because they are low-cost and can simultaneously provide range and Doppler data. In comparison to pulse and Ultra Wide Band (UWB) radar sensors, they also need a lower sample rate and a lower peak to average ratio. The system employs a cutting-edge surveillance method based on widely available FMCW radar technology. The data processing approach is built on an ad hoc-chain of different blocks that transforms data, extract features, and make a classification decision before cancelling clutters and leakage using a frame subtraction technique, applying DL algorithms to Range-Doppler (RD) maps, and adding a peak to cluster assignment step before tracking targets. In conclusion, the FMCW radar and DL technique for the RD maps performed well together for indoor use-cases. The aforementioned tests used an edge device and Infineon Technologies' Position2Go FMCW radar tool-set

Machine Learning for Human Activity Detection in Smart Homes

Recognizing human activities in domestic environments from audio and active power consumption sensors is a challenging task since on the one hand, environmental sound signals are multi-source, heterogeneous, and varying in time and on the other hand, the active power consumption varies significantly for similar type electrical appliances. Many systems have been proposed to process environmental sound signals for event detection in ambient assisted living applications. Typically, these systems use feature extraction, selection, and classification. However, despite major advances, several important questions remain unanswered, especially in real-world settings. A part of this thesis contributes to the body of knowledge in the field by addressing the following problems for ambient sounds recorded in various real-world kitchen environments: 1) which features, and which classifiers are most suitable in the presence of background noise? 2) what is the effect of signal duration on recognition accuracy? 3) how do the SNR and the distance between the microphone and the audio source affect the recognition accuracy in an environment in which the system was not trained? We show that for systems that use traditional classifiers, it is beneficial to combine gammatone frequency cepstral coefficients and discrete wavelet transform coefficients and to use a gradient boosting classifier. For systems based on deep learning, we consider 1D and 2D CNN using mel-spectrogram energies and mel-spectrograms images, as inputs, respectively and show that the 2D CNN outperforms the 1D CNN. We obtained competitive classification results for two such systems and validated the performance of our algorithms on public datasets (Google Brain/TensorFlow Speech Recognition Challenge and the 2017 Detection and Classification of Acoustic Scenes and Events Challenge). Regarding the problem of the energy-based human activity recognition in a household environment, machine learning techniques to infer the state of household appliances from their energy consumption data are applied and rule-based scenarios that exploit these states to detect human activity are used. Since most activities within a house are related with the operation of an electrical appliance, this unimodal approach has a significant advantage using inexpensive smart plugs and smart meters for each appliance. This part of the thesis proposes the use of unobtrusive and easy-install tools (smart plugs) for data collection and a decision engine that combines energy signal classification using dominant classifiers (compared in advanced with grid search) and a probabilistic measure for appliance usage. It helps preserving the privacy of the resident, since all the activities are stored in a local database. DNNs received great research interest in the field of computer vision. In this thesis we adapted different architectures for the problem of human activity recognition. We analyze the quality of the extracted features, and more specifically how model architectures and parameters affect the ability of the automatically extracted features from DNNs to separate activity classes in the final feature space. Additionally, the architectures that we applied for our main problem were also applied to text classification in which we consider the input text as an image and apply 2D CNNs to learn the local and global semantics of the sentences from the variations of the visual patterns of words. This work helps as a first step of creating a dialogue agent that would not require any natural language preprocessing. Finally, since in many domestic environments human speech is present with other environmental sounds, we developed a Convolutional Recurrent Neural Network, to separate the sound sources and applied novel post-processing filters, in order to have an end-to-end noise robust system. Our algorithm ranked first in the Apollo-11 Fearless Steps Challenge.Horizon 2020 research and innovation program under the Marie Sklodowska-Curie grant agreement No. 676157, project ACROSSIN