Practical classification of different moving targets using automotive radar and deep neural networks

In this work, the authors present results for classification of different classes of targets (car, single and multiple people, bicycle) using automotive radar data and different neural networks. A fast implementation of radar algorithms for detection, tracking, and micro-Doppler extraction is proposed in conjunction with the automotive radar transceiver TEF810X and microcontroller unit SR32R274 manufactured by NXP Semiconductors. Three different types of neural networks are considered, namely a classic convolutional network, a residual network, and a combination of convolutional and recurrent network, for different classification problems across the four classes of targets recorded. Considerable accuracy (close to 100% in some cases) and low latency of the radar pre-processing prior to classification (∼0.55 s to produce a 0.5 s long spectrogram) are demonstrated in this study, and possible shortcomings and outstanding issues are discussed

Personnel recognition and gait classification based on multistatic micro-doppler signatures using deep convolutional neural networks

In this letter, we propose two methods for personnel recognition and gait classification using deep convolutional neural networks (DCNNs) based on multistatic radar micro-Doppler signatures. Previous DCNN-based schemes have mainly focused on monostatic scenarios, whereas directional diversity offered by multistatic radar is exploited in this letter to improve classification accuracy. We first propose the voted monostatic DCNN (VMo-DCNN) method, which trains DCNNs on each receiver node separately and fuses the results by binary voting. By merging the fusion step into the network architecture, we further propose the multistatic DCNN (Mul-DCNN) method, which performs slightly better than VMo-DCNN. These methods are validated on real data measured with a 2.4-GHz multistatic radar system. Experimental results show that the Mul-DCNN achieves over 99% accuracy in armed/unarmed gait classification using only 20% training data and similar performance in two-class personnel recognition using 50% training data, which are higher than the accuracy obtained by performing DCNN on a single radar node

Radar and RGB-depth sensors for fall detection: a review

This paper reviews recent works in the literature on the use of systems based on radar and RGB-Depth (RGB-D) sensors for fall detection, and discusses outstanding research challenges and trends related to this research field. Systems to detect reliably fall events and promptly alert carers and first responders have gained significant interest in the past few years in order to address the societal issue of an increasing number of elderly people living alone, with the associated risk of them falling and the consequences in terms of health treatments, reduced well-being, and costs. The interest in radar and RGB-D sensors is related to their capability to enable contactless and non-intrusive monitoring, which is an advantage for practical deployment and users’ acceptance and compliance, compared with other sensor technologies, such as video-cameras, or wearables. Furthermore, the possibility of combining and fusing information from The heterogeneous types of sensors is expected to improve the overall performance of practical fall detection systems. Researchers from different fields can benefit from multidisciplinary knowledge and awareness of the latest developments in radar and RGB-D sensors that this paper is discussing

Radar for Assisted Living in the Context of Internet of Things for Health and Beyond

This paper discusses the place of radar for assisted living in the context of IoT for Health and beyond. First, the context of assisted living and the urgency to address the problem is described. The second part gives a literature review of existing sensing modalities for assisted living and explains why radar is an upcoming preferred modality to address this issue. The third section presents developments in machine learning that helps improve performances in classification especially with deep learning with a reflection on lessons learned from it. The fourth section introduces recent published work from our research group in the area that shows promise with multimodal sensor fusion for classification and long short-term memory applied to early stages in the radar signal processing chain. Finally, we conclude with open challenges still to be addressed in the area and open to future research directions in animal welfare

Micro-Doppler Based Human-Robot Classification Using Ensemble and Deep Learning Approaches

Radar sensors can be used for analyzing the induced frequency shifts due to micro-motions in both range and velocity dimensions identified as micro-Doppler ($\boldsymbol{\mu}$-D) and micro-Range ($\boldsymbol{\mu}$-R), respectively. Different moving targets will have unique $\boldsymbol{\mu}$-D and $\boldsymbol{\mu}$-R signatures that can be used for target classification. Such classification can be used in numerous fields, such as gait recognition, safety and surveillance. In this paper, a 25 GHz FMCW Single-Input Single-Output (SISO) radar is used in industrial safety for real-time human-robot identification. Due to the real-time constraint, joint Range-Doppler (R-D) maps are directly analyzed for our classification problem. Furthermore, a comparison between the conventional classical learning approaches with handcrafted extracted features, ensemble classifiers and deep learning approaches is presented. For ensemble classifiers, restructured range and velocity profiles are passed directly to ensemble trees, such as gradient boosting and random forest without feature extraction. Finally, a Deep Convolutional Neural Network (DCNN) is used and raw R-D images are directly fed into the constructed network. DCNN shows a superior performance of 99\% accuracy in identifying humans from robots on a single R-D map.Comment: 6 pages, accepted in IEEE Radar Conference 201

Cross-Frequency Classification of Indoor Activities with DNN Transfer Learning

Remote, non-contact recognition of human motion and activities is central to health monitoring in assisted living facilities, but current systems face the problems of training compatibility, minimal training data sets and a lack of interoperability between radar sensors at different frequencies. This paper represents a first work to consider the efficacy of deep neural networks (DNNs) and transfer learning to bridge the gap in phenomenology that results when multiple types of radars simultaneously observe human activity. Six different human activities are recorded indoors simultaneously with 5.8 GHz and 25 GHz radars. Firstly, the bottleneck feature performance of the DNNs show that a baseline of 76% is achieved. On models trained only with 25 GHz data when 5.8 GHz data is used for testing 81% accuracy is achieved. in absence of a large dataset for radar at a certain frequency, we demonstrate information from a different frequency radar is better suited for generating the classification models than optical images and by using time-velocity diagrams (TVD), a degree of interoperability can be achieved

Cross-Frequency Classification of Indoor Activities with DNN Transfer Learning

Remote, non-contact recognition of human motion and activities is central to health monitoring in assisted living facilities, but current systems face the problems of training compatibility, minimal training data sets and a lack of interoperability between radar sensors at different frequencies. This paper represents a first work to consider the efficacy of deep neural networks (DNNs) and transfer learning to bridge the gap in phenomenology that results when multiple types of radars simultaneously observe human activity. Six different human activities are recorded indoors simultaneously with 5.8 GHz and 25 GHz radars. Firstly, the bottleneck feature performance of the DNNs show that a baseline of 76% is achieved. On models trained only with 25 GHz data when 5.8 GHz data is used for testing 81% accuracy is achieved. in absence of a large dataset for radar at a certain frequency, we demonstrate information from a different frequency radar is better suited for generating the classification models than optical images and by using time-velocity diagrams (TVD), a degree of interoperability can be achieved

Edge Artificial Intelligence for Real-Time Target Monitoring

The key enabling technology for the exponentially growing cellular communications sector is location-based services. The need for location-aware services has increased along with the number of wireless and mobile devices. Estimation problems, and particularly parameter estimation, have drawn a lot of interest because of its relevance and engineers' ongoing need for higher performance. As applications expanded, a lot of interest was generated in the accurate assessment of temporal and spatial properties. In the thesis, two different approaches to subject monitoring are thoroughly addressed. For military applications, medical tracking, industrial workers, and providing location-based services to the mobile user community, which is always growing, this kind of activity is crucial. In-depth consideration is given to the viability of applying the Angle of Arrival (AoA) and Receiver Signal Strength Indication (RSSI) localization algorithms in real-world situations. We presented two prospective systems, discussed them, and presented specific assessments and tests. These systems were put to the test in diverse contexts (e.g., indoor, outdoor, in water...). The findings showed the localization capability, but because of the low-cost antenna we employed, this method is only practical up to a distance of roughly 150 meters. Consequently, depending on the use-case, this method may or may not be advantageous. An estimation algorithm that enhances the performance of the AoA technique was implemented on an edge device. Another approach was also considered. Radar sensors have shown to be durable in inclement weather and bad lighting conditions. Frequency Modulated Continuous Wave (FMCW) radars are the most frequently employed among the several sorts of radar technologies for these kinds of applications. Actually, this is because they are low-cost and can simultaneously provide range and Doppler data. In comparison to pulse and Ultra Wide Band (UWB) radar sensors, they also need a lower sample rate and a lower peak to average ratio. The system employs a cutting-edge surveillance method based on widely available FMCW radar technology. The data processing approach is built on an ad hoc-chain of different blocks that transforms data, extract features, and make a classification decision before cancelling clutters and leakage using a frame subtraction technique, applying DL algorithms to Range-Doppler (RD) maps, and adding a peak to cluster assignment step before tracking targets. In conclusion, the FMCW radar and DL technique for the RD maps performed well together for indoor use-cases. The aforementioned tests used an edge device and Infineon Technologies' Position2Go FMCW radar tool-set

Data portability for activities of daily living and fall detection in different environments using radar micro-doppler

The health status of an older or vulnerable person can be determined by looking into the additive effects of aging as well as any associated diseases. This status can lead the person to a situation of ‘unstable incapacity’ for normal aging and is determined by the decrease in response to the environment and to specific pathologies with apparent decrease of independence in activities of daily living (ADL). In this paper, we use micro-Doppler images obtained using a frequency-modulated continuous wave radar (FMCW) operating at 5.8 GHz with 400 MHz bandwidth as the sensor to perform assessment of this health status. The core idea is to develop a generalized system where the data obtained for ADL can be portable across different environments and groups of subjects, and critical events such as falls in mature individuals can be detected. In this context, we have conducted comprehensive experimental campaigns at nine different locations including four laboratory environments and five elderly care homes. A total of 99 subjects participated in the experiments where 1453 micro-Doppler signatures were recorded for six activities. Different machine learning, deep learning algorithms and transfer learning technique were used to classify the ADL. The support vector machine (SVM), K-nearest neighbor (KNN) and convolutional neural network (CNN) provided adequate classification accuracies for particular scenarios; however, the autoencoder neural network outperformed the mentioned classifiers by providing classification accuracy of ~ 88%. The proposed system for fall detection in elderly people can be deployed in care centers and is application for any indoor settings with various age group of people. For future work, we would focus on monitoring multiple older adults, concurrently in indoor settings using continuous radar sensor data stream which is limitation of the present system