4,059 research outputs found
Gesture Recognition with mmWave Wi-Fi Access Points: Lessons Learned
In recent years, channel state information (CSI) at sub-6 GHz has been widely
exploited for Wi-Fi sensing, particularly for activity and gesture recognition.
In this work, we instead explore mmWave (60 GHz) Wi-Fi signals for gesture
recognition/pose estimation. Our focus is on the mmWave Wi-Fi signals so that
they can be used not only for high data rate communication but also for
improved sensing e.g., for extended reality (XR) applications. For this reason,
we extract spatial beam signal-to-noise ratios (SNRs) from the periodic beam
training employed by IEEE 802.11ad devices. We consider a set of 10
gestures/poses motivated by XR applications. We conduct experiments in two
environments and with three people.As a comparison, we also collect CSI from
IEEE 802.11ac devices. To extract features from the CSI and the beam SNR, we
leverage a deep neural network (DNN). The DNN classifier achieves promising
results on the beam SNR task with state-of-the-art 96.7% accuracy in a single
environment, even with a limited dataset. We also investigate the robustness of
the beam SNR against CSI across different environments. Our experiments reveal
that features from the CSI generalize without additional re-training, while
those from beam SNRs do not. Therefore, re-training is required in the latter
case
Antenna Response Consistency Driven Self-supervised Learning for WIFI-based Human Activity Recognition
Self-supervised learning (SSL) for WiFi-based human activity recognition (HAR) holds great promise due to its ability to address the challenge of insufficient labeled data. However, directly transplanting SSL algorithms, especially contrastive learning, originally designed for other domains to CSI data, often fails to achieve the expected performance. We attribute this issue to the inappropriate alignment criteria, which disrupt the semantic distance consistency between the feature space and the input space. To address this challenge, we introduce \textbf{A}ntenna \textbf{R}esponse \textbf{C}onsistency (ARC) as a solution to define proper alignment criteria. ARC is designed to retain semantic information from the input space while introducing robustness to real-world noise. Moreover, we substantiate the effectiveness of ARC through a comprehensive set of experiments, demonstrating its capability to enhance the performance of self-supervised learning for WiFi-based HAR by achieving an increase of over 5\% in accuracy in most cases and achieving a best accuracy of 94.97\%
Transfer: Cross Modality Knowledge Transfer using Adversarial Networks -- A Study on Gesture Recognition
Knowledge transfer across sensing technology is a novel concept that has been
recently explored in many application domains, including gesture-based human
computer interaction. The main aim is to gather semantic or data driven
information from a source technology to classify / recognize instances of
unseen classes in the target technology. The primary challenge is the
significant difference in dimensionality and distribution of feature sets
between the source and the target technologies. In this paper, we propose
TRANSFER, a generic framework for knowledge transfer between a source and a
target technology. TRANSFER uses a language-based representation of a hand
gesture, which captures a temporal combination of concepts such as handshape,
location, and movement that are semantically related to the meaning of a word.
By utilizing a pre-specified syntactic structure and tokenizer, TRANSFER
segments a hand gesture into tokens and identifies individual components using
a token recognizer. The tokenizer in this language-based recognition system
abstracts the low-level technology-specific characteristics to the machine
interface, enabling the design of a discriminator that learns
technology-invariant features essential for recognition of gestures in both
source and target technologies. We demonstrate the usage of TRANSFER for three
different scenarios: a) transferring knowledge across technology by learning
gesture models from video and recognizing gestures using WiFi, b) transferring
knowledge from video to accelerometer, and d) transferring knowledge from
accelerometer to WiFi signals
SenseFi: A library and benchmark on deep-learning-empowered WiFi human sensing
Over the recent years, WiFi sensing has been rapidly developed for privacy-preserving, ubiquitous human-sensing applications, enabled by signal processing and deep-learning methods. However, a comprehensive public benchmark for deep learning in WiFi sensing, similar to that available for visual recognition, does not yet exist. In this article, we review recent progress in topics ranging from WiFi hardware platforms to sensing algorithms and propose a new library with a comprehensive benchmark, SenseFi. On this basis, we evaluate various deep-learning models in terms of distinct sensing tasks, WiFi platforms, recognition accuracy, model size, computational complexity, and feature transferability. Extensive experiments are performed whose results provide valuable insights into model design, learning strategy, and training techniques for real-world applications. In summary, SenseFi is a comprehensive benchmark with an open-source library for deep learning in WiFi sensing research that offers researchers a convenient tool to validate learning-based WiFi-sensing methods on multiple datasets and platforms.Nanyang Technological UniversityPublished versionThis research is supported by NTU Presidential Postdoctoral Fellowship, ‘‘Adaptive Multi-modal Learning for Robust Sensing and Recognition in Smart Cities’’ project fund (020977-00001), at the Nanyang Technological University, Singapore
MIMOCrypt: Multi-User Privacy-Preserving Wi-Fi Sensing via MIMO Encryption
Wi-Fi signals may help realize low-cost and non-invasive human sensing, yet
it can also be exploited by eavesdroppers to capture private information. Very
few studies rise to handle this privacy concern so far; they either jam all
sensing attempts or rely on sophisticated technologies to support only a single
sensing user, rendering them impractical for multi-user scenarios. Moreover,
these proposals all fail to exploit Wi-Fi's multiple-in multiple-out (MIMO)
capability. To this end, we propose MIMOCrypt, a privacy-preserving Wi-Fi
sensing framework to support realistic multi-user scenarios. To thwart
unauthorized eavesdropping while retaining the sensing and communication
capabilities for legitimate users, MIMOCrypt innovates in exploiting MIMO to
physically encrypt Wi-Fi channels, treating the sensed human activities as
physical plaintexts. The encryption scheme is further enhanced via an
optimization framework, aiming to strike a balance among i) risk of
eavesdropping, ii) sensing accuracy, and iii) communication quality, upon
securely conveying decryption keys to legitimate users. We implement a
prototype of MIMOCrypt on an SDR platform and perform extensive experiments to
evaluate its effectiveness in common application scenarios, especially
privacy-sensitive human gesture recognition.Comment: IEEE S&P 2024, 19 pages, 22 figures, including meta reviews and
response
SoK: Inference Attacks and Defenses in Human-Centered Wireless Sensing
Human-centered wireless sensing aims to understand the fine-grained
environment and activities of a human using the diverse wireless signals around
her. The wireless sensing community has demonstrated the superiority of such
techniques in many applications such as smart homes, human-computer
interactions, and smart cities. Like many other technologies, wireless sensing
is also a double-edged sword. While the sensed information about a human can be
used for many good purposes such as enhancing life quality, an adversary can
also abuse it to steal private information about the human (e.g., location,
living habits, and behavioral biometric characteristics). However, the
literature lacks a systematic understanding of the privacy vulnerabilities of
wireless sensing and the defenses against them.
In this work, we aim to bridge this gap. First, we propose a framework to
systematize wireless sensing-based inference attacks. Our framework consists of
three key steps: deploying a sniffing device, sniffing wireless signals, and
inferring private information. Our framework can be used to guide the design of
new inference attacks since different attacks can instantiate these three steps
differently. Second, we propose a defense-in-depth framework to systematize
defenses against such inference attacks. The prevention component of our
framework aims to prevent inference attacks via obfuscating the wireless
signals around a human, while the detection component aims to detect and
respond to attacks. Third, based on our attack and defense frameworks, we
identify gaps in the existing literature and discuss future research
directions
- …