Real-Time Radar-Based Gesture Detection and Recognition Built in an Edge-Computing Platform

In this paper, a real-time signal processing frame-work based on a 60 GHz frequency-modulated continuous wave (FMCW) radar system to recognize gestures is proposed. In order to improve the robustness of the radar-based gesture recognition system, the proposed framework extracts a comprehensive hand profile, including range, Doppler, azimuth and elevation, over multiple measurement-cycles and encodes them into a feature cube. Rather than feeding the range-Doppler spectrum sequence into a deep convolutional neural network (CNN) connected with recurrent neural networks, the proposed framework takes the aforementioned feature cube as input of a shallow CNN for gesture recognition to reduce the computational complexity. In addition, we develop a hand activity detection (HAD) algorithm to automatize the detection of gestures in real-time case. The proposed HAD can capture the time-stamp at which a gesture finishes and feeds the hand profile of all the relevant measurement-cycles before this time-stamp into the CNN with low latency. Since the proposed framework is able to detect and classify gestures at limited computational cost, it could be deployed in an edge-computing platform for real-time applications, whose performance is notedly inferior to a state-of-the-art personal computer. The experimental results show that the proposed framework has the capability of classifying 12 gestures in real-time with a high F1-score.Comment: Accepted for publication in IEEE Sensors Journal. A video is available on https://youtu.be/IR5NnZvZBL

Automatic Radar-based Gesture Detection and Classification via a Region-based Deep Convolutional Neural Network

In this paper, a region-based deep convolutional neural network (R-DCNN) is proposed to detect and classify gestures measured by a frequency-modulated continuous wave radar system. Micro-Doppler (μD) signatures of gestures are exploited, and the resulting spectrograms are fed into a neural network. We are the first to use the R-DCNN for radar-based gesture recognition, such that multiple gestures could be automatically detected and classified without manually clipping the data streams according to each hand movement in advance. Further, along with the μD signatures, we incorporate phase-difference information of received signals from an L-shaped antenna array to enhance the classification accuracy. Finally, the classification results show that the proposed network trained with spectrogram and phase-difference information can guarantee a promising performance for nine gestures