Radar-based Application of Pedestrian and Cyclist Micro-Doppler Signatures for Automotive Safety Systems

Die sensorbasierte Erfassung des Nahfeldes im Kontext des hochautomatisierten Fahrens erfährt einen spürbaren Trend bei der Integration von Radarsensorik. Fortschritte in der Mikroelektronik erlauben den Einsatz von hochauflösenden Radarsensoren, die durch effiziente Verfahren sowohl im Winkel als auch in der Entfernung und im Doppler die Messgenauigkeit kontinuierlich ansteigen lassen. Dadurch ergeben sich neuartige Möglichkeiten bei der Bestimmung der geometrischen und kinematischen Beschaffenheit ausgedehnter Ziele im Fahrzeugumfeld, die zur gezielten Entwicklung von automotiven Sicherheitssystemen herangezogen werden können. Im Rahmen dieser Arbeit werden ungeschützte Verkehrsteilnehmer wie Fußgänger und Radfahrer mittels eines hochauflösenden Automotive-Radars analysiert. Dabei steht die Erscheinung des Mikro-Doppler-Effekts, hervorgerufen durch das hohe Maß an kinematischen Freiheitsgraden der Objekte, im Vordergrund der Betrachtung. Die durch den Mikro-Doppler-Effekt entstehenden charakteristischen Radar-Signaturen erlauben eine detailliertere Perzeption der Objekte und können in direkten Zusammenhang zu ihren aktuellen Bewegungszuständen gesetzt werden. Es werden neuartige Methoden vorgestellt, die die geometrischen und kinematischen Ausdehnungen der Objekte berücksichtigen und echtzeitfähige Ansätze zur Klassifikation und Verhaltensindikation realisieren. Wird ein ausgedehntes Ziel (z.B. Radfahrer) von einem Radarsensor detektiert, können aus dessen Mikro-Doppler-Signatur wesentliche Eigenschaften bezüglich seines Bewegungszustandes innerhalb eines Messzyklus erfasst werden. Die Geschwindigkeitsverteilungen der sich drehenden Räder erlauben eine adaptive Eingrenzung der Tretbewegung, deren Verhalten essentielle Merkmale im Hinblick auf eine vorausschauende Unfallprädiktion aufweist. Ferner unterliegen ausgedehnte Radarziele einer Orientierungsabhängigkeit, die deren geometrischen und kinematischen Profile direkt beeinflusst. Dies kann sich sowohl negativ auf die Klassifikations-Performance als auch auf die Verwertbarkeit von Parametern auswirken, die eine Absichtsbekundung des Radarziels konstituieren. Am Beispiel des Radfahrers wird hierzu ein Verfahren vorgestellt, das die orientierungsabhängigen Parameter in Entfernung und Doppler normalisiert und die gemessenen Mehrdeutigkeiten kompensiert. Ferner wird in dieser Arbeit eine Methodik vorgestellt, die auf Grundlage des Mikro- Doppler-Profils eines Fußgängers dessen Beinbewegungen über die Zeit schätzt (Tracking) und wertvolle Objektinformationen hinsichtlich seines Bewegungsverhaltens offenbart. Dazu wird ein Bewegungsmodell entwickelt, das die nichtlineare Fortbewegung des Beins approximiert und dessen hohes Maß an biomechanischer Variabilität abbildet. Durch die Einbeziehung einer wahrscheinlichkeitsbasierten Datenassoziation werden die Radar-Detektionen ihren jeweils hervorrufenden Quellen (linkes und rechtes Bein) zugeordnet und eine Trennung der Gliedmaßen realisiert. Im Gegensatz zu bisherigen Tracking-Verfahren weist die vorgestellte Methodik eine Steigerung in der Genauigkeit der Objektinformationen auf und stellt damit einen entscheidenden Vorteil für zukünftige Fahrerassistenzsysteme dar, um deutlich schneller auf kritische Verkehrssituationen reagieren zu können.:1 Introduction 1 1.1 Automotive environmental perception 2 1.2 Contributions of this work 4 1.3 Thesis overview 6 2 Automotive radar 9 2.1 Physical fundamentals 9 2.1.1 Radar cross section 9 2.1.2 Radar equation 10 2.1.3 Micro-Doppler effect 11 2.2 Radar measurement model 15 2.2.1 FMCW radar 15 2.2.2 Chirp sequence modulation 17 2.2.3 Direction-of-arrival estimation 22 2.3 Signal processing 25 2.3.1 Target properties 26 2.3.2 Target extraction 28 Power detection 28 Clustering 30 2.3.3 Real radar data example 31 2.4 Conclusion 33 3 Micro-Doppler applications of a cyclist 35 3.1 Physical fundamentals 35 3.1.1 Micro-Doppler signatures of a cyclist 35 3.1.2 Orientation dependence 36 3.2 Cyclist feature extraction 38 3.2.1 Adaptive pedaling extraction 38 Ellipticity constraints 38 Ellipse fitting algorithm 39 3.2.2 Experimental results 42 3.3 Normalization of the orientation dependence 44 3.3.1 Geometric correction 44 3.3.2 Kinematic correction 45 3.3.3 Experimental results 45 3.4 Conclusion 47 3.5 Discussion and outlook 47 4 Micro-Doppler applications of a pedestrian 49 4.1 Pedestrian detection 49 4.1.1 Human kinematics 49 4.1.2 Micro-Doppler signatures of a pedestrian 51 4.1.3 Experimental results 52 Radially moving pedestrian 52 Crossing pedestrian 54 4.2 Pedestrian feature extraction 57 4.2.1 Frequency-based limb separation 58 4.2.2 Extraction of body parts 60 4.2.3 Experimental results 62 4.3 Pedestrian tracking 64 4.3.1 Probabilistic state estimation 65 4.3.2 Gaussian filters 67 4.3.3 The Kalman filter 67 4.3.4 The extended Kalman filter 69 4.3.5 Multiple-object tracking 71 4.3.6 Data association 74 4.3.7 Joint probabilistic data association 80 4.4 Kinematic-based pedestrian tracking 84 4.4.1 Kinematic modeling 84 4.4.2 Tracking motion model 87 4.4.3 4-D radar point cloud 91 4.4.4 Tracking implementation 92 4.4.5 Experimental results 96 Longitudinal trajectory 96 Crossing trajectory with sudden turn 98 4.5 Conclusion 102 4.6 Discussion and outlook 103 5 Summary and outlook 105 5.1 Developed algorithms 105 5.1.1 Adaptive pedaling extraction 105 5.1.2 Normalization of the orientation dependence 105 5.1.3 Model-based pedestrian tracking 106 5.2 Outlook 106 Bibliography 109 List of Acronyms 119 List of Figures 124 List of Tables 125 Appendix 127 A Derivation of the rotation matrix 2.26 127 B Derivation of the mixed radar signal 2.52 129 C Calculation of the marginal association probabilities 4.51 131 Curriculum Vitae 135Sensor-based detection of the near field in the context of highly automated driving is experiencing a noticeable trend in the integration of radar sensor technology. Advances in microelectronics allow the use of high-resolution radar sensors that continuously increase measurement accuracy through efficient processes in angle as well as distance and Doppler. This opens up novel possibilities in determining the geometric and kinematic nature of extended targets in the vehicle environment, which can be used for the specific development of automotive safety systems. In this work, vulnerable road users such as pedestrians and cyclists are analyzed using a high-resolution automotive radar. The focus is on the appearance of the micro-Doppler effect, caused by the objects’ high kinematic degree of freedom. The characteristic radar signatures produced by the micro-Doppler effect allow a clearer perception of the objects and can be directly related to their current state of motion. Novel methods are presented that consider the geometric and kinematic extents of the objects and realize real-time approaches to classification and behavioral indication. When a radar sensor detects an extended target (e.g., bicyclist), its motion state’s fundamental properties can be captured from its micro-Doppler signature within a measurement cycle. The spinning wheels’ velocity distributions allow an adaptive containment of the pedaling motion, whose behavior exhibits essential characteristics concerning predictive accident prediction. Furthermore, extended radar targets are subject to orientation dependence, directly affecting their geometric and kinematic profiles. This can negatively affect both the classification performance and the usability of parameters constituting the radar target’s intention statement. For this purpose, using the cyclist as an example, a method is presented that normalizes the orientation-dependent parameters in range and Doppler and compensates for the measured ambiguities. Furthermore, this paper presents a methodology that estimates a pedestrian’s leg motion over time (tracking) based on the pedestrian’s micro-Doppler profile and reveals valuable object information regarding his motion behavior. To this end, a motion model is developed that approximates the leg’s nonlinear locomotion and represents its high degree of biomechanical variability. By incorporating likelihood-based data association, radar detections are assigned to their respective evoking sources (left and right leg), and limb separation is realized. In contrast to previous tracking methods, the presented methodology shows an increase in the object information’s accuracy. It thus represents a decisive advantage for future driver assistance systems in order to be able to react significantly faster to critical traffic situations.:1 Introduction 1 1.1 Automotive environmental perception 2 1.2 Contributions of this work 4 1.3 Thesis overview 6 2 Automotive radar 9 2.1 Physical fundamentals 9 2.1.1 Radar cross section 9 2.1.2 Radar equation 10 2.1.3 Micro-Doppler effect 11 2.2 Radar measurement model 15 2.2.1 FMCW radar 15 2.2.2 Chirp sequence modulation 17 2.2.3 Direction-of-arrival estimation 22 2.3 Signal processing 25 2.3.1 Target properties 26 2.3.2 Target extraction 28 Power detection 28 Clustering 30 2.3.3 Real radar data example 31 2.4 Conclusion 33 3 Micro-Doppler applications of a cyclist 35 3.1 Physical fundamentals 35 3.1.1 Micro-Doppler signatures of a cyclist 35 3.1.2 Orientation dependence 36 3.2 Cyclist feature extraction 38 3.2.1 Adaptive pedaling extraction 38 Ellipticity constraints 38 Ellipse fitting algorithm 39 3.2.2 Experimental results 42 3.3 Normalization of the orientation dependence 44 3.3.1 Geometric correction 44 3.3.2 Kinematic correction 45 3.3.3 Experimental results 45 3.4 Conclusion 47 3.5 Discussion and outlook 47 4 Micro-Doppler applications of a pedestrian 49 4.1 Pedestrian detection 49 4.1.1 Human kinematics 49 4.1.2 Micro-Doppler signatures of a pedestrian 51 4.1.3 Experimental results 52 Radially moving pedestrian 52 Crossing pedestrian 54 4.2 Pedestrian feature extraction 57 4.2.1 Frequency-based limb separation 58 4.2.2 Extraction of body parts 60 4.2.3 Experimental results 62 4.3 Pedestrian tracking 64 4.3.1 Probabilistic state estimation 65 4.3.2 Gaussian filters 67 4.3.3 The Kalman filter 67 4.3.4 The extended Kalman filter 69 4.3.5 Multiple-object tracking 71 4.3.6 Data association 74 4.3.7 Joint probabilistic data association 80 4.4 Kinematic-based pedestrian tracking 84 4.4.1 Kinematic modeling 84 4.4.2 Tracking motion model 87 4.4.3 4-D radar point cloud 91 4.4.4 Tracking implementation 92 4.4.5 Experimental results 96 Longitudinal trajectory 96 Crossing trajectory with sudden turn 98 4.5 Conclusion 102 4.6 Discussion and outlook 103 5 Summary and outlook 105 5.1 Developed algorithms 105 5.1.1 Adaptive pedaling extraction 105 5.1.2 Normalization of the orientation dependence 105 5.1.3 Model-based pedestrian tracking 106 5.2 Outlook 106 Bibliography 109 List of Acronyms 119 List of Figures 124 List of Tables 125 Appendix 127 A Derivation of the rotation matrix 2.26 127 B Derivation of the mixed radar signal 2.52 129 C Calculation of the marginal association probabilities 4.51 131 Curriculum Vitae 13

Reliable detection and characterisation of dim target via track-before-detect

Detection of manoeuvring and small objects is a challenging task in radar surveillance applications. Small objects in high noise background induce low signal to noise ratio (SNR) reflections. Conventional methods detect such objects by integrating multiple reflections in the same range-bearing and doppler bins in sampled versions of received signals. When the objects manoeuvre, however, these methods are likely to fail to detect them because the integration is performed without taking into account the possibility of the object movements across resolution bins. Furthermore, slowly manoeuvring objects create detection difficulties in discriminating them from radar clutter. Reflections of such objects contain micro-Doppler shifts generated by their propulsion devices. These shifts can characterise specific types of objects. In this case, estimation of these shifts is a challenging task because the front-end signals at the receiver are low SNR reflections and are the superposition of all reflections from the entire object and the noise background. Conventional estimators for this purpose only use reflections collected in a coherent processing interval (CPI) and produce poor estimate outputs. In order to achieve the desired accuracy, one requires more reflections than those collected in a CPI. This thesis mainly considers the aforementioned two difficulties and aims to develop efficient algorithms, which can detect low SNR and manoeuvring objects by incorporating long-time pulse integration and micro-doppler estimation. Main contributions in this thesis are based on the following two algorithms. The first work considers the detection of manoeuvring and small objects with radars. The radar systems are considered both co-located and separated transmitter/receiver pairs, i.e., monostatic and bistatic configurations, respectively, as well as multistatic settings involving both types. The proposed detection algorithm is capable of coherently integrating reflected signals within a CPI in all these configurations and continuing integration for an arbitrarily long time across consecutive CPIs. This approach estimates the complex value of the reflection coefficients for the integration while simultaneously estimating the object trajectory. Compounded with this simultaneous tracking and reflection coefficient estimation is the estimation of the unknown time reference shift of the separated transmitters necessary for coherent processing. The detection is made by using the resulting integration value in a Neyman-Pearson test against a constant false alarm rate threshold. The second work focuses on micro-Doppler signature estimation of manoeuvring and small rotor based unmanned aerial vehicle (UAV) systems with a monostatic radar. The micro-Doppler signature is considered rotation frequencies generated by rotating rotor blades of the UAVs. This estimation uses a maximum likelihood (ML) approach that finds rotation frequencies to maximise a likelihood function conditioned on an object trajectory, complex reflection coefficients, and rotation frequencies. In particular, the proposed algorithm uses an expectation-maximisation (EM) approach such that the expectation of the likelihood mentioned above is approximated by using the state distributions generated from Bayesian recursive filtering for the trajectory estimation. The reflection coefficients and the rotation frequencies are estimated by maximising this approximated expectation. As a result, this algorithm is capable of simultaneously tracking the trajectory and estimating the reflection coefficients and the rotation frequencies of the UAVs before the decision on the object presence is made

High-Dimensional Information Detection based on Correlation Imaging Theory

Radar is a device that uses electromagnetic(EM) waves to detect targets; it can measure the position parameters and motion parameters and extract target characteristics information by analyzing the reflected signal from the target. From the perspective of the radar theoretical basis of physics, the more than 70 years of development of radar are based on the EM field fluctuation theory of physics. Many theories have been developed towards one-dimensional signal processing. For example, a variety of threshold filtering have widely used as methods to resist interference during detection. The optimal state estimation describes the propagation process of the statistical characteristics of the target over time in the probability domain. Compressed sensing greatly improves the reconstructing efficiency of the sparse signal. These theories are one-dimensional information processing. The information obtained by them is a deterministic description of the EM field. The correlated imaging technique is from the high-order coherence property of the EM field, which uses the fluctuation characteristic of the EM field to realize non-local imaging. Correlated imaging radar, a combination of correlated imaging techniques and modern information theory, will provide a novel remote sensing detection and imaging method. More importantly, correlated imaging radar is a new research field. Therefore, a complete theoretical frame and application system should be urgently built up and improved. Based on the coherence theory of the EM field, the work in this thesis explores the method of determining the statistical characteristics of the EM field so that the high dimensional target information can be detected, including theoretical analysis, principle design, imaging modes, target detecting models, image reconstruction algorithms, the enhancement of visibility, and system design. The simulations and real experiments are set up to prove the theory's validity and the systems' feasibility

Experimental studies on shock wave interactions with flexible surfaces and development of flow diagnostic tools

Nowadays, light-weight composite materials have increasingly used for high-speed flight vehicles to improve their performance and efficiency. At supersonic speed, sonic fatigue, panel flutter, severe instabilities, and even catastrophic structural failure would occur due to the shock wave impingement on several flexible components of a given structural system either internally or externally. Therefore, investigation on shock wave interaction with flexible surfaces is crucial for the safety and performance of high-speed flight vehicles. This work aims to investigate the mechanism of shock wave interaction with flexible surfaces with and without the presence of the boundary layer. The first part involves the shock wave generated by supersonic starting jets interaction with flexible surfaces and the other one focuses on shock wave and boundary layer interaction (SBLI) over flexible surfaces. A novel miniature and cost-effective shock tube driven by detonation transmission tubing was designed and manufactured to simulate the supersonic starting jet and investigate the interaction of a supersonic starting jet with flexible surfaces. To investigate the characterization of this novel type shock tube, the pressure-time measurement in the driven section and the time-resolved shadowgraph were performed. The result shows that the flow structure from the open end of the shock tube driven by detonation transmission tubing agrees with that of conventional compressed-gas driven shock tubes. Moreover, this novel type of shock tube has good repeatability of less than 3% with a Mach number range of 1.29-1.58 when the weight of the NONEL explosive mixture varies from 3.6mg to 12.6mg. An unsteady background oriented schlieren (BOS) measurement system and a sprayable Polymer-Ceramic unsteady pressure sensitive paint (PC-PSP) system were developed. The preliminary BOS result in a supersonic wind tunnel shows that the sensitivity of the BOS system is good enough to visualize weak density variations caused by expansion waves, boundary layer, and weak oblique shocks. Additionally, compared with the commercial PC-PSP from Innovative Scientific Solutions Incorporated (ISSI), the in-house developed unsteady PSP system has higher pressure sensitivity, lower temperature sensitivity, and photo-degradation rate. To identify the shock movement, distortion and unsteadiness during the processes of the supersonic starting jet impingement and shock wave boundary layer interaction (SBLI) over flexible surfaces, an image processing scheme involving background subtraction in the frequency domain, filtering, resampling, edge detection, adaptive threshold, contour detection, feature extraction, and fitting was proposed and applied to process shadowgraph and schlieren sequences automatically. A large shadowgraph data set characterized by low signal to noise ratio (SNR) and small spatial resolution (312×260-pixel), was used to validate the proposed scheme. The result proves that the aforementioned image processing scheme can detect, track, localize, and fit shock waves in a subpixel accuracy. The mechanism of the interaction between the initial shock wave from a supersonic starting jet and flexible surfaces was investigated based on a square shock tube driven by detonation transmitting tube. Compared with that of the solid plate case, flexible surfaces can delay the shock reflection process because of the flexible panel deformation generated by the pressure difference between the top and the bottom. The delay time is around 8µs in the case of 0.1mm thick flexible surface, whereas it declines to around 4µs in the case of 0.3mm thick flexible surface because of the lower flexibility and deformation magnitude. However, interestingly, the propagation velocity of the reflected shock wave is basically the same for the solid plate and flexible panels, which means the flexible surface doesn’t reduce the strength of the reflection wave, although it delays its propagation. Also, there is not an apparent difference in the velocity of the reflected shock wave in the case of different incident shock Mach numbers when Ms varying from 1.22 to 1.54. These experimental results from this study are useful for validating numerical codes that are used for understanding fluid-structure interaction processes

Remote Sensing

This dual conception of remote sensing brought us to the idea of preparing two different books; in addition to the first book which displays recent advances in remote sensing applications, this book is devoted to new techniques for data processing, sensors and platforms. We do not intend this book to cover all aspects of remote sensing techniques and platforms, since it would be an impossible task for a single volume. Instead, we have collected a number of high-quality, original and representative contributions in those areas

Remote Sensing of the Oceans

This book covers different topics in the framework of remote sensing of the oceans. Latest research advancements and brand-new studies are presented that address the exploitation of remote sensing instruments and simulation tools to improve the understanding of ocean processes and enable cutting-edge applications with the aim of preserving the ocean environment and supporting the blue economy. Hence, this book provides a reference framework for state-of-the-art remote sensing methods that deal with the generation of added-value products and the geophysical information retrieval in related fields, including: Oil spill detection and discrimination; Analysis of tropical cyclones and sea echoes; Shoreline and aquaculture area extraction; Monitoring coastal marine litter and moving vessels; Processing of SAR, HF radar and UAV measurements

Twenty-Second Lunar and Planetary Science Conference

The papers in this collection were written for general presentation, avoiding jargon and unnecessarily complex terms. Some of the topics covered include: planetary evolution, planetary satellites, planetary composition, planetary surfaces, planetary geology, volcanology, meteorite impacts and composition, and cosmic dust. Particular emphasis is placed on Mars and the Moon

Significant Accomplishments in Science and Technology

The proceedings of a symposium on significant accomplishments in science and technology are presented. The symposium was held at the Goddard Space Flight Center in December 1973. The subjects discussed are as follows: (1) cometary physics, (2) X-ray and gamma ray astronomy, (3) solar and terrestrial physics, (4) spacecraft technology, (5) Earth Resources Technology Satellite, (6) earth and ocean physics, (6) communications and navigation, (7) mission operations and data systems, and (8) networks systems and operations

Apollo-Soyuz test project. Volume 1: Astronomy, earth atmosphere and gravity field, life sciences, and materials processing

The joint U.S.-USSR experiments and the U.S. conducted unilateral experiments performed during the Apollo Soyuz Test Project are described. Scientific concepts and experiment design and operation are discussed along with scientific results of postflight analysis