In this paper the application, performance and results of a fully discrete micro-Doppler feature classication processing chain utilising Krawtchouk moment invariants are presented. The approach demonstrates to be capable of running on low power hardware such as the Raspberry Pi 2. The effectiveness of the proposed approach is veried through the use of real K-band data in real-time

Aßmann, A.

Clemente, Carmine

Izzo, A.

University of Strathclyde Institutional Repository

Aßmann, A. and Izzo, A. and Clemente, Carmine (2016) Efficient micro-doppler based pedestrian activity classification for ADAS systems using Krawtchouk moments. In: 11th IMA International Conference on Mathematics in Signal Processing, 2016-12-12 - 2016-12-14, IET Austin Court. , This version is available at https://strathprints.strath.ac.uk/66617/Strathprints is  designed  to  allow  users  to  access  the  research  output  of  the  University  of Strathclyde. Unless otherwise explicitly stated on the manuscript, Copyright © and Moral Rights for the papers on this site are retained by the individual authors and/or other copyright owners. Please check the manuscript for details of any other licences that may have been applied. You may  not  engage  in  further  distribution  of  the  material  for  any  profitmaking  activities  or  any commercial gain. You may freely distribute both the url (https://strathprints.strath.ac.uk/) and the content of this paper for research or private study, educational, or not-for-profit purposes without prior permission or charge. Any correspondence concerning this service should be sent to the Strathprints administrator: strathprints@strath.ac.ukThe Strathprints institutional repository (https://strathprints.strath.ac.uk) is a digital archive of University of Strathclyde research outputs. It has been developed to disseminate open access research outputs, expose data about those outputs, and enable the management and persistent access to Strathclyde's intellectual output.1Efficient Micro-Doppler based pedestrianactivity classification for ADAS systemsusing Krawtchouk momentsBy A. Aßmann1, A. Izzo2 and C. Clemente21Centre for Doctoral Training in Applied Photonics, Heriot-Watt University, Edinburgh,UK,2Centre in Signal & Image Processing, University of Strathclyde, Glasgow, UKAbstractIn this paper the application, performance and results of a fully discrete micro-Dopplerfeature classification processing chain utilising Krawtchouk moment invariants are pre-sented. The approach demonstrates to be capable of running on low power hardware suchas the Raspberry Pi 2. The effectiveness of the proposed approach is verified through theuse of real K-band data in real-time.1. IntroductionFuture automotive autonomous systems require contextual information about the sceneat hand to allow complex decision making in the context of self-driving cars. Secondarymotion in reference to the bulk motion described by the Doppler effect are known asmicro-Doppler (m-D) modulations (Chen (2011)), which can be captured by using time-frequency radar signatures containing unique information about moving non-rigid targetssuch as pedestrians in the line-of-sight (LOS). In contrast to information captured bycamera systems, radar signatures are insensitive to light conditions, weather conditionsand relative distance. They can be therefore a robust identifier for scene and targetclassification, which can be used in safety critical applications if they meet performancerequirements.The main contribution of this paper is the presentation of a novel radar m-D classifi-cation tool-chain capable of running on low-power hardware using Krawtchouk momentsenabling real-time operation of the classification algorithm by Clemente et al. (2016).Captured real radar data is sensitive to noise and variant to the initial phase of capture,rotation, scale and translation. Further, they are highly complex and thus it is com-putationally expensive to perform any further processing, specifically in the context ofclassification. To address these issues, a processing tool chain has been developed utilisingfive steps to reduce the data complexity and ultimately yielding a robust feature vec-tor. With the ability to highly parallelise these steps on compatible hardware, it enablesreal-time pedestrian classification with a Raspberry Pi 2 on simultaneously captured realK-band radar data on a trained classifier containing four classes of pedestrians.The remainder of this paper is organised as follows. Section 2 introduces the featureextraction using Krawtchouk moments. Section 3 describes the potential application inautonomous systems, while Section 4 presents the performance of the system. Section 5concludes this paper.11th International Conference on Mathematics in Signal Processing 12  14 December 2016, Austin Court, Birmingham, UKEfficient m-D pedestrian classification via Krawtchouk moments 22. Feature ExtractionSafety critical systems require information with as little delay as possible and in nearreal-time. It is therefore of interest to use feature vectors with a minimal number of com-ponents, while maintaining a robust set of information. It is further advantageous if thefeature extraction is computationally efficient and ideally based on as many pre-computedcomponents as possible. One such processing chain has been proposed (Clemente et al.(2016)) to extract a robust feature vector with pre-computed Krawtchouk polynomialsresulting in Krawtchouk moment invariants, which are further invariant to rotation, scaleand translation of the image.Given a discrete sequence of complex radar data x(n) = I(n) + jQ(n), this vector isnormalised and downsampled to reduce the initial amount of data and limit the relevantfrequency range dependent on the expected target signatures, resulting in the signal s˜(n).The spectrogram is computed using a discrete short-time Fourier transform resulting inχ(ν, k) = |STFT (x)|2 =∣∣∣∣∣N−1∑n=0(s˜(n) · h(n− k) · e−j2πνnN )∣∣∣∣∣2, k = 0, ...,K − 1 , (2.1)where ν is the normalized frequency and h is a windowing function of choice to smooththe edges of the each iterative signal window (Clemente et al. (2015)).Invariance to the initial phase of motion is achieved by measuring the cadence ofDoppler frequencies in χ(ν, k), resulting in the cadence-velocity diagram (CVD),∆(ν, ǫ) =∣∣∣∣∣K−1∑k=0(χ(ν, k) · e−j2πkǫK )∣∣∣∣∣ , (2.2)where ǫ is the cadence frequency and the size of this image is defined by the choice of theFourier transform resolution, resulting in a N×K size, where N is the vertical dimensionof χ(ν, k) (i.e. the Doppler frequency spectrum). For efficient classification the CVD mustbe further compressed while maintaining discrimination between separate feature classesand similarity within a particular class (Clemente et al. (2015), Bjo˝rklund (2012)).To achieve this, geometrical moment invariants, as introduced by Hu (1962), can beused to transform the CVD into a compact feature vector. An orthogonal form usingKrawtchouk polynomials was presented by Yap et al. (2003) and further applied byClemente et al. (2016) to radar imaging in a discrete fashion, by applying pre-computedweighted Krawtchouk polynomials K¯ of the order (p,q) to ∆(ν, ǫ) (2.2) resulting in theKrawtchouk moments of a CVD,Qpq =P−1∑ν=0Q−1∑ǫ=0K¯p(ǫ; p1, P − 1)K¯q(ν; p2, Q− 1)∆(ν, ǫ). (2.3)The above expression can be readily vectorised toFKrawtchouk = [Q00, ..., Qpq]. (2.4)This unique feature vector is further normalised to remove the impact of singular featuresover others and hence potential bias in the classification process, resulting in the finalfeature vector of dimension (p+ 1)× (q + 1) beingF˜ =F− µFσF, (2.5)11th International Conference on Mathematics in Signal Processing 12  14 December 2016, Austin Court, Birmingham, UKAPPLICATIONS 3Figure 1: Processing chain of a Python based efficient mD classification on an embeddedsystem, where ts is the sample time and tp is the total processing time of the algorithm.where µF is the mean and σF is the standard deviation of F. This robust feature vectoris invariant to rotation, scale and translation of an image and can be computed discretelyand efficiently for classification purposes (see Clemente et al. (2016) for more detail).3. ApplicationsThe procedure in section 2 was implemented in a modular processing tool-chain in Pythoncapable of running on a variety of hardware including low-power embedded systems. Themodular nature of the processing chain allows parallel computing to be employed forboth complete processes and further sub-processes itself. The standard processing flowis shown in Figure 1.The primary requirement for this processing chain is a radar sensor which providesphase information in the form of I/Q radar data, which can be interfaced with thedesired processing system.This enables flexible deployment of m-D analysis and radar analysis in a variety ofapplications. The experiments in this paper were conducted using a 24 GHz radar sensorat a height of 1 metre being a typical automotive placement above the front bumper.It allows robust analysis of the scene at hand to detect and classify objects in motion.The focus in the following section was given to pedestrian motion. This requires aneffective Doppler frequency range of ±750 Hz for pedestrians running in front of a vehicle(Ghaleb et al. (2008)). It is further suggested by Bjo˝rklund (2012) to capture a fullcycle of pedestrian motion, which was modestly estimated at 3 seconds. Being able toclassify pedestrians and their current type of motion can be an important measure forautonomous systems and driver assistance as it allows to not only to make an assessmentof the current scene but further allows to predict future movement of pedestrians basedon their direction and their relative velocity making it a potentially useful measure infuture automotive systems.4. Experimental ResultsData was recorded for the four classes as shown in Table 1. The data was then processedto CVDs (examples shown in Figure 2 with associated spectrograms) and classified inKrawtchouck moment data sets. The integrity of the data was verified using a MonteCarlo approach and a k-Nearest-Neighbour (kNN) classifier with a 70/30 split. Hence,11th International Conference on Mathematics in Signal Processing 12  14 December 2016, Austin Court, Birmingham, UKEfficient m-D pedestrian classification via Krawtchouk moments 4Class Time recorded Training yieldIndividual walking 352.9 s 77Individual running 319.8 s 73Group walking 179.4 s 61Group running 221.9 s 56All classes 1074 s 267Table 1: Recorded amount of raw data for the 4 tested classesPlatform k=1 k=3 k=5 k=7Accuracy all 98.17% 97.75% 96.39% 93.33%Average time x86 (Intel C2D T7500, 4GB RAM) 0.173 s 0.176 s 0.172 s 0.178 sAverage time BeagleBone Black 2.60 s 2.73 s 2.64 s 2.74 sAverage time Raspberry Pi 2 (1 GHz) 1.72 s 1.78 s 1.75 s 1.80 sTable 2: Accuracy verification of kNN-classifier and training data for a range of k-Neighbours70% of the training data used was randomly selected and 30% was randomly selectedas the testing set. The oﬄine performance was then tested on 3 platforms, as shownin Table 2. The integrity of the data set was sufficient at above 90%. Further, given asampling time ts of 3 seconds the Raspberry Pi 2 as a modern and low-power embeddedsystems is capable of running the classification in parallel to the data acquisition and cantherefore provide classification predictions in quasi-real-time. It further indicates that thesampling time can be further reduced to match the processing time of the hardware tobe used, illustrating the scalability of the proposed tool-chain.The Raspberry Pi 2 and the radar sensor were successfully tested in a live environmentbeing powered by a 5000 mAh battery, illustrating the low power requirements of thesystem with a projected runtime of the entire system at above an hour, demonstratingthe power efficiency crucial in e.g. electric cars. The live testing showed no slow-down incomparison to the aforementioned oﬄine performance.5. ConclusionIn this paper the application, performance and results of a fully discrete micro-Dopplerfeature classification processing chain utilising Krawtchouk moment invariants was pre-sented. The approach demonstrates to be capable of running on low power hardware suchas the Raspberry Pi 2. The effectiveness of the proposed approach was verified throughthe use of real K-band data in quasi-real-time.11th International Conference on Mathematics in Signal Processing 12  14 December 2016, Austin Court, Birmingham, UKCONCLUSION 5REFERENCESChen, V.C. The Micro-Doppler Effect in Radar Artech House Radar Library, Artech House,2011.Clemente, C., Pallotta, L., Maio, A. Soraghan, J. & Farina, A A novel algorithm forradar classification based on Doppler characteristics exploiting pseudo-Zernike polynomialsIEEE Transactions on Aerospace and Electronic Systems vol. 51, pp. 417-430, Jan. 2015.Clemente, C., Pallotta, L., Gaglione, D., Maio, A. & Soraghan, J. Automatic recogni-tion of military vehicles with Krawtchouk moments IEEE Transactions on Aerospace andElectronic Systems, 31.07.2016.Bjo˝rklund, S., Johansson, T. & Petersson H. May 2012 Evaluation of a microdoppler clas-sification method on mm-wave data Radar Conference (RADAR), 2012 IEEE, pp. 934939.Hu, M.-K. Visual pattern recognition by moment invariants IRE Trans- actions on InformationTheory vol. 8, no. 2, pp. 179187, Feb. 1962.Yap, P.-T., Paramesran, R. & Ong, S.-H. Image analysis by Krawtchouk moments IEEETransactions on Image Processing vol. 12, no. 11, pp. 1367 1377, Nov. 2003.Ghaleb,A., Vignaud L., & Nicolas J. Micro-doppler analysis of wheels and pedestrians inisar imaging Signal Processing, IET, vol. 2, no. 3, pp. 301311, Sep. 2008.11th International Conference on Mathematics in Signal Processing 12  14 December 2016, Austin Court, Birmingham, UKEfficient m-D pedestrian classification via Krawtchouk moments 6(a) (b)(c) (d)(e) (f)(g) (h)Figure 2: Spectrogram (LHS) and CVD (RHS) examples for recorded motion classes:(a,b) Single pedestrian walking, (c,d) multiple pedestrians walking, (e,f) single pedestrianrunning and (g,h) multiple pedestrians running.11th International Conference on Mathematics in Signal Processing 12  14 December 2016, Austin Court, Birmingham, UK

Efficient micro-doppler based pedestrian activity classification for ADAS systems using Krawtchouk moments

https://strathprints.strath.ac.uk/66617/1/Abmann_ICMSP_2016_Efficient_micro_doppler_based_pedestrian_activity_classification.pdf

Efficient micro-doppler based pedestrian activity classification for ADAS systems using Krawtchouk moments

Abstract

Similar works

Full text

Available Versions

University of Strathclyde Institutional Repository