Personnel recognition and gait classification based on multistatic micro-doppler signatures using deep convolutional neural networks

In this letter, we propose two methods for personnel recognition and gait classification using deep convolutional neural networks (DCNNs) based on multistatic radar micro-Doppler signatures. Previous DCNN-based schemes have mainly focused on monostatic scenarios, whereas directional diversity offered by multistatic radar is exploited in this letter to improve classification accuracy. We first propose the voted monostatic DCNN (VMo-DCNN) method, which trains DCNNs on each receiver node separately and fuses the results by binary voting. By merging the fusion step into the network architecture, we further propose the multistatic DCNN (Mul-DCNN) method, which performs slightly better than VMo-DCNN. These methods are validated on real data measured with a 2.4-GHz multistatic radar system. Experimental results show that the Mul-DCNN achieves over 99% accuracy in armed/unarmed gait classification using only 20% training data and similar performance in two-class personnel recognition using 50% training data, which are higher than the accuracy obtained by performing DCNN on a single radar node

Toward Unobtrusive In-home Gait Analysis Based on Radar Micro-Doppler Signatures

Objective: In this paper, we demonstrate the applicability of radar for gait classification with application to home security, medical diagnosis, rehabilitation and assisted living. Aiming at identifying changes in gait patterns based on radar micro-Doppler signatures, this work is concerned with solving the intra motion category classification problem of gait recognition. Methods: New gait classification approaches utilizing physical features, subspace features and sum-of-harmonics modeling are presented and their performances are evaluated using experimental K-band radar data of four test subjects. Five different gait classes are considered for each person, including normal, pathological and assisted walks. Results: The proposed approaches are shown to outperform existing methods for radar-based gait recognition which utilize physical features from the cadence-velocity data representation domain as in this paper. The analyzed gait classes are correctly identified with an average accuracy of 93.8%, where a classification rate of 98.5% is achieved for a single gait class. When applied to new data of another individual a classification accuracy on the order of 80% can be expected. Conclusion: Radar micro-Doppler signatures and their Fourier transforms are well suited to capture changes in gait. Five different walking styles are recognized with high accuracy. Significance: Radar-based sensing of human gait is an emerging technology with multi-faceted applications in security and health care industries. We show that radar, as a contact-less sensing technology, can supplement existing gait diagnostic tools with respect to long-term monitoring and reproducibility of the examinations.Comment: 11 pages, 6 figure

Representation of Radar Micro-Dopplers Using Customized Dictionaries

Human motions give rise to frequency modulations, known as micro-Dopplers, to continuous wave radar signals. Micro-Doppler signals have been extensively researched for the classification of different types of human motions as well as to distinguish humans from other moving targets. However, there are two main scenarios where the performance of existing algorithms deteriorates significantly—one, when the channel consists of multiple moving targets resulting in distorted signatures, and two, when the systems conditions during the training stage deviate significantly from the conditions during the test stage. In this chapter, it is demonstrated that both of these limitations can be overcome by representing the radar data through customized dictionaries, fine-tuned to provide sparser representations of the data, than traditional data-independent dictionaries such as Fourier or wavelets. The performances of the algorithms are evaluated with both simulated and measured radar data gathered from moving humans in indoor line-of-sight conditions

Toward Deep Learning-Based Human Target Analysis

In this chapter, we describe methods toward deep learning-based human target analysis. Firstly, human target analysis in 2D and 3D domains of radar signal is introduced. Furthermore, range-Doppler surface for human target analysis using ultra-wideband radar is described. The construction of range-Doppler surface involves range-Doppler imaging, adaptive threshold detection, and isosurface extraction. In comparison with micro-Doppler profiles and high-resolution range profiles, range-Doppler surface contains range, Doppler, and time information simultaneously. An ellipsoid-based human motion model is designed for validation. Range-Doppler surfaces simulated for different human activities are demonstrated and discussed. With the rapid emergence of deep learning, the development of radar target recognition has been accelerated. We describe several deep learning algorithms for human target analysis. Finally, a few future research considerations are listed to spark inspiration

Radar-Based Multi-Target Classification Using Deep Learning

Real-time, radar-based human activity and target recognition has several applications in various fields. Examples include hand gesture recognition, border and home surveillance, pedestrian recognition for automotive safety and fall detection for assisted living. This dissertation sought to improve the speed and accuracy of a previously developed model classifying human activity and targets using radar data for outdoor surveillance purposes. An improvement in accuracy and speed of classification helps surveillance systems to provide reliable results on time. For example, the results can be used to intercept trespassers, poachers or smugglers. To achieve these objectives, radar data was collected using a C-band pulse-Doppler radar and converted to spectrograms using the Short-time Fourier transform (STFT) algorithm. Spectrograms of the following classes were utilised in classification: one human walking, two humans walking, one human running, moving vehicles, a swinging sphere and clutter/noise. A seven-layer residual network was proposed, which utilised batch normalisation (BN), global average pooling (GAP), and residual connections to achieve a classification accuracy of 92.90% and 87.72% on the validation and test data, respectively. Compared to the previously proposed model, this represented a 10% improvement in accuracy on the validation data and a 3% improvement on the test data. Applying model quantisation provided up to 3.8 times speedup in inference, with a less than 0.4% accuracy drop on both the validation and test data. The quantised model could support a range of up to 89.91 kilometres in real-time, allowing it to be used in radars that operate within this range

DopNet:A Deep Convolutional Neural Network to Recognize Armed and Unarmed Human Targets

The work presented in this paper aims to distinguish between armed or unarmed personnel using multi-static radar data and advanced Doppler processing. We propose two modified Deep Convolutional Neural Networks (DCNN) termed SCDopNet and MC-DopNet for mono-static and multi-static micro- Doppler signature (μ-DS) classification. Differentiating armed and unarmed walking personnel is challenging due to the effect of aspect angle and channel diversity in real-world scenarios. In addition, DCNN easily overfits the relatively small-scale μ-DS dataset. To address these problems, the work carried out in this paper makes three key contributions: first, two effective schemes including data augmentation operation and a regularization term are proposed to train SC-DopNet from scratch. Next, a factor analysis of the SC-DopNet are conducted based on various operating parameters in both the processing and radar operations. Thirdly, to solve the problem of aspect angle diversity for μ-DS classification, we design MC-DopNet for multi-static μ- DS which is embedded with two new fusion schemes termed as Greedy Importance Reweighting (GIR) and `21-Norm. These two schemes are based on two different strategies and have been evaluated experimentally: GIR uses a “win by sacrificing worst case” whilst `21-Norm adopts a “win by sacrificing best case” approach. The SC-DopNet outperforms the non-deep methods by 12.5% in average and the proposed MC-DopNet with two fusion methods outperforms the conventional binary voting by 1.2% in average. Note that we also argue and discuss how to utilize the statistics of SC-DopNet results to infer the selection of fusion strategies for MC-DopNet under different experimental scenarios

Signal classification at discrete frequencies using machine learning

Incidents such as the 2018 shut down of Gatwick Airport due to a small Unmanned Aerial System (UAS) airfield incursion, have shown that we don’t have routine and consistent detection and classification methods in place to recognise unwanted signals in an airspace. Today, incidents of this nature are taking place around the world regularly. The first stage in mitigating a threat is to know whether a threat is present. This thesis focuses on the detection and classification of Global Navigation Satellite Systems (GNSS) jamming radio frequency (RF) signal types and small commercially available UAS RF signals using machine learning for early warning systems. RF signals can be computationally heavy and sometimes sensitive to collect. With neural networks requiring a lot of information to train from scratch, the thesis explores the use of transfer learning from the object detection field to lessen this burden by using graphical representations of the signal in the frequency and time domain. The thesis shows that utilising the benefits of transfer learning with both supervised and unsupervised learning and graphical signal representations, can provide high accuracy detection and classification, down to the fidelity of whether a small UAS is flying or stationary. By treating the classification of RF signals as an image classification problem, this thesis has shown that transfer learning through CNN feature extraction reduces the need for large datasets while still providing high accuracy results. CNN feature extraction and transfer learning was also shown to improve accuracy as a precursor to unsupervised learning but at a cost of time, while raw images provided a good overall solution for timely clustering. Lastly the thesis has shown that the implementation of machine learning models using a raspberry pi and software defined radio (SDR) provides a viable option for low cost early warning systems

Human Action Recognition from Active Acoustics: Physics Modelling for Representation Learning and Inference Using Generative Probabilistic Graphical Models

This dissertation explores computational methods to address the problem of physics-based modeling and ultimately doing inference from data in multiple modalities where there exists large amounts of low dimensional data complementary to a much smaller set of high dimensional data. In this instance the low dimensional timeseries data are active acoustics from a micro-Doppler sensor that include no or very limited spatial information, and the high dimensional data is RGB-Depth skeleton data from a Microsoft Kinect sensor. The task is that of human action recognition from the active acoustic data. To accomplish this, statistical models, trained simultaneously on both the micro-Doppler modulations induced by human actions and symbolic representations of skeletal poses, are developed. This enables the model to learn correlations between the rich temporal structure of the micro-Doppler modulations and the high-dimensional motion sequences of human action. During runtime, the model then relies purely on the active acoustic data to infer the human action. In order to adapt this methodology to situations not observed in the training data, a physical model of the human body is combined with a physics-based simulation of the Doppler phenomenon to predict the acoustic data for a sequence of skeletal poses and a con gurable sensor geometry. The physics model is then combined with a generative statistical model for human actions to create a generative physics-based model of micro-Doppler modulations for human action