Higher order feature extraction and selection for robust human gesture recognition using CSI of COTS Wi-Fi devices

Device-free human gesture recognition (HGR) using commercial o the shelf (COTS) Wi-Fi devices has gained attention with recent advances in wireless technology. HGR recognizes the human activity performed, by capturing the reflections ofWi-Fi signals from moving humans and storing them as raw channel state information (CSI) traces. Existing work on HGR applies noise reduction and transformation to pre-process the raw CSI traces. However, these methods fail to capture the non-Gaussian information in the raw CSI data due to its limitation to deal with linear signal representation alone. The proposed higher order statistics-based recognition (HOS-Re) model extracts higher order statistical (HOS) features from raw CSI traces and selects a robust feature subset for the recognition task. HOS-Re addresses the limitations in the existing methods, by extracting third order cumulant features that maximizes the recognition accuracy. Subsequently, feature selection methods derived from information theory construct a robust and highly informative feature subset, fed as input to the multilevel support vector machine (SVM) classifier in order to measure the performance. The proposed methodology is validated using a public database SignFi, consisting of 276 gestures with 8280 gesture instances, out of which 5520 are from the laboratory and 2760 from the home environment using a 10 5 cross-validation. HOS-Re achieved an average recognition accuracy of 97.84%, 98.26% and 96.34% for the lab, home and lab + home environment respectively. The average recognition accuracy for 150 sign gestures with 7500 instances, collected from five di erent users was 96.23% in the laboratory environment.Taylor's University through its TAYLOR'S PhD SCHOLARSHIP Programmeinfo:eu-repo/semantics/publishedVersio

Sensing the care:Advancing unobtrusive sensing solutions to support informal caregivers of older adults with cognitive impairment

Older adults (65 years and above) make up a growing proportion of the world's population which is anticipated to increase further in the coming decades. As individuals age, they often become more vulnerable to cognitive impairments, necessitating a diverse array of care and support services from their caregivers to uphold their quality of life. However, the scarcity of professional caregivers and care facilities, compounded by the preference of many older adults to remain in their own homes, places a significant burden on informal caregivers, adversely affecting their physical, mental, and social well-being. To assist informal caregivers, numerous sensing solutions have been developed. However, many of these solutions are not optimally suited for older adult care, particularly in cases of cognitive impairments. In that regard, the overarching aim of this thesis was to develop and evaluate the Unobtrusive Sensing Solution (USS) for in-home monitoring of older adults with cognitive impairment (OwCI) who live alone in their own houses to ease the support of their informal caregivers. In the 'Explore and Scope' part, a scoping review was conducted to identify available unobtrusive sensing technology that can be implemented in older adult care. Subsequently, in the 'Develop and Test' part, Wi-Fi CSI technology was utilized to collect a dataset illustrating physical agitation activities (Wi-Gitation). However, upon evaluation of the Wi-Gitation dataset, a challenge of generalization across different domains (or environments) was identified. To address this, the Inter-data Selected Sequential Transfer Learning framework was proposed and implemented. Lastly, in the 'Design to Communicate' part, the thesis focused on identifying the needs and requirements of informal caregivers of OwCI towards USSs. These needs and requirements were gathered through interviews and surveys, informing the development of a Lo-Fi prototype for an interaction platform. Overall, the results obtained in this thesis not only enhance the development of Wi-Fi CSI (specifically for OwCI care) but also provide valuable insights into the informational and design requirements of informal caregivers, thereby promoting the context-aware development of USSs

Moisture estimation for precision agriculture through RF sensing

Convenient, non-obtrusive, low-cost, and accurate sensing of fruit moisture content is crucial for the scientific studies of Pomology and Viticulture and their associated agriculture. It can provide early indicators of yield estimation and crop health as well as providing data for food production and precision farming systems. With a focus on grapes, we introduce SING, a scheme that senses grape moisture content by utilizing RF signals but without physical contact with the fruit. In this thesis, we extend the investigation of the theoretical relationship between the dielectric properties and the moisture content of agricultural products to establish a sensing model in the 5 GHz band. To make the work practical, we are first to measure the dielectric properties of grape bunches (not individually as that would be destructive), presenting a unique measurement challenge as internal grapes are hidden. In doing so, we demonstrate that our technique precisely estimates moisture content to a high degree of accuracy (90%). Current RF sensing models to estimate moisture are destructive; they require samples to be constrained in containers. Our work is first to dispense with such impracticalities, and, without contact with the object, accurately measures non-uniform grape clusters in open space. We demonstrate that SING is superior to existing work in its ability to accurately measure the dielectric properties of non-uniform fruit objects and test this through both lab-based experimentation and preliminary outdoor vineyard tests. We also examine the transferability of SING’s approach to real-world scenarios.Open Acces

Employing multi-modal sensors for personalised smart home health monitoring.

Smart home systems are employed worldwide for a variety of automated monitoring tasks. FITsense is a system that performs personalised smart home health monitoring using sensor data. In this thesis, we expand upon this system by identifying the limits of health monitoring using simple IoT sensors, and establishing deployable solutions for new rich sensing technologies. The FITsense system collects data from FitHomes and generates behavioural insights for health monitoring. To allow the system to expand to arbitrary home layouts, sensing applications must be delivered while relying on sparse "ground truth" data. An enhanced data representation was tested for improving activity recognition performance by encoding observed temporal dependencies. Experiments showed an improvement in activity recognition accuracy over baseline data representations with standard classifiers. Channel State Information (CSI) was chosen as our rich sensing technology for its ambient nature and potential deployability. We developed a novel Python toolkit, called CSIKit, to handle various CSI software implementations, including automatic detection for off-the-shelf CSI formats. Previous researchers proposed a method to address AGC effects on COTS CSI hardware, which we tested and found to improve correlation with a baseline without AGC. This implementation was included in the public release of CSIKit. Two sensing applications were delivered using CSIKit to demonstrate its functionality. Our statistical approach to motion detection with CSI data showed a 32% increase in accuracy over an infrared sensor-based solution using data from 2 unique environments. We also demonstrated the first CSI activity recognition application on a Raspberry Pi 4, which achieved an accuracy of 92% with 11 activity classes. An application was then trained to support movement detection using data from all COTS CSI hardware. This was combined with our signal divider implementation to compare CSI wireless and sensing performance characteristics. The IWL5300 exhibited the most consistent wireless performance, while the ESP32 was found to produce viable CSI data for sensing applications. This establishes the ESP32 as a low-cost high-value hardware solution for CSI sensing. To complete this work, an in-home study was performed using real-world sensor data. An ESP32-based CSI sensor was developed to be integrated into our IoT network. This sensor was tested in a FitHome environment to identify how the data from our existing simple sensors could aid sensor development. We performed an experiment to demonstrate that annotations for CSI data could be gathered with infrared motion sensors. Results showed that our new CSI sensor collected real-world data of similar utility to that collected manually in a controlled environment

Intelligent Multi-Modal Sensing-Communication Integration: Synesthesia of Machines

In the era of sixth-generation (6G) wireless communications, integrated sensing and communications (ISAC) is recognized as a promising solution to upgrade the physical system by endowing wireless communications with sensing capability. Existing ISAC is mainly oriented to static scenarios with radio-frequency (RF) sensors being the primary participants, thus lacking a comprehensive environment feature characterization and facing a severe performance bottleneck in dynamic environments. To date, extensive surveys on ISAC have been conducted but are limited to summarizing RF-based radar sensing. Currently, some research efforts have been devoted to exploring multi-modal sensing-communication integration but still lack a comprehensive review. Therefore, we generalize the concept of ISAC inspired by human synesthesia to establish a unified framework of intelligent multi-modal sensing-communication integration and provide a comprehensive review under such a framework in this paper. The so-termed Synesthesia of Machines (SoM) gives the clearest cognition of such intelligent integration and details its paradigm for the first time. We commence by justifying the necessity of the new paradigm. Subsequently, we offer a definition of SoM and zoom into the detailed paradigm, which is summarized as three operation modes. To facilitate SoM research, we overview the prerequisite of SoM research, i.e., mixed multi-modal (MMM) datasets. Then, we introduce the mapping relationships between multi-modal sensing and communications. Afterward, we cover the technological review on SoM-enhance-based and SoM-concert-based applications. To corroborate the superiority of SoM, we also present simulation results related to dual-function waveform and predictive beamforming design. Finally, we propose some potential directions to inspire future research efforts.Comment: This paper has been accepted by IEEE Communications Surveys & Tutorial

Systems with Massive Number of Antennas: Distributed Approaches

As 5G is entering maturity, the research interest has shifted towards 6G, and specially the new use cases that the future telecommunication infrastructure needs to support. These new use cases encompass much higher requirements, specifically: higher communication data-rates, larger number of users, higher accuracy in localization, possibility to wirelessly charge devices, among others.The radio access network (RAN) has already gone through an evolution on the path towards 5G. One of the main changes was a large increment of the number of antennas in the base-station. Some of them may even reach 100 elements, in what is commonly referred as Massive MIMO. New proposals for 6G RAN point in the direction of continuing this path of increasing the number of antennas, and locate them throughout a certain area of service. Different technologies have been proposed in this direction, such as: cell-free Massive MIMO, distributed MIMO, and large intelligent surface (LIS). In this thesis we focus on LIS, whose conducted theoretical studies promise the fulfillment of the aforementioned requirements.While the theoretical capabilities of LIS have been conveniently analyzed, little has been done in terms of implementing this type of systems. When the number of antennas grow to hundreds or thousands, there are numerous challenges that need to be solved for a successful implementation. The most critical challenges are the interconnection data-rate and the computational complexity.In the present thesis we introduce the implementation challenges, and show that centralized processing architectures are no longer adequate for this type of systems. We also present different distributed processing architectures and show the benefits of this type of schemes. This work aims at giving a system-design guideline that helps the system designer to make the right decisions when designing these type of systems. For that, we provide algorithms, performance analysis and comparisons, including first order evaluation of the interconnection data-rate, processing latency, memory and energy consumption. These numbers are based on models and available data in the literature. Exact values depend on the selected technology, and will be accurately determined after building and testing these type of systems.The thesis concentrates mostly on the topic of communication, with additional exploration of other areas, such as localization. In case of localization, we benefit from the high spatial resolution of a very-large array that provides very rich channel state information (CSI). A CSI-based fingerprinting via neural network technique is selected for this case with promising results. As the communication and localization services are based on the acquisition of CSI, we foresee a common system architecture capable of supporting both cases. Further work in this direction is recommended, with the possibility of including other applications such as sensing.The obtained results indicate that the implementation of these very-large array systems is feasible, but the challenges are numerous. The proposed solutions provide encouraging results that need to be verified with hardware implementations and real measurements

Interwoven Waves:Enhancing the Scalability and Robustness of Wi-Fi Channel State Information for Human Activity Recognition

This PhD dissertation investigates the future of unobtrusive radio wave-based sensing, specifically focusing on Wi-Fi sensing in realistic healthcare scenarios. Wi-Fi sensing leverages the analysis of multi-path reflections of radio waves to monitor human activities and physiological states, providing a scalable solution without intruding on daily life.Wi-Fi-based sensing, particularly through channel state information, fits well in healthcare due to its ubiquitous presence and unobtrusiveness. As our society ages and populations grow, continuous health monitoring becomes increasingly critical. Existing solutions like wearable devices, audiovisual technologies, and expensive infrastructure modifications each have limitations, such as forgetting to wear devices, privacy invasions, and high costs. Channel state information-based sensing offers a promising alternative, enabling remote monitoring without the need for additional infrastructure changes.Nevertheless, implementing channel state information-based sensing in already congested Wi-Fi bands could present challenges in the future. Current solutions often exacerbate congestion by adding random noise, which can degrade network performance. These solutions also tend to address niche problems in idealistic settings, making it difficult to justify their use in everyday environments due to potential impacts on network latency and overall user experience.To realise the potential of Wi-Fi sensing, future solutions must integrate seamlessly with wireless communication networks, ensuring that sensing and communication processes coexist and collaborate effectively. This dissertation categorises the relationship between sensing and communication into three models: parasitic, opportunistic, and mutualistic. In the parasitic model, sensing operates independently of the wireless infrastructure, potentially adding noise and congestion. The opportunistic model leverages existing traffic flows, avoiding adverse effects on communication. The mutualistic model aims for a balance, enhancing both sensing and communication without compromising either function.The primary research objective is to enhance the robustness and scalability of channel state information-based sensing for human activity recognition, facilitating seamless integration into home environments with minimal impact on existing infrastructure. Overall, this dissertation provides an exploration of the challenges and solutions for unobtrusive Wi-Fi sensing in healthcare, paving the way for future advancements in the field