Practical classification of different moving targets using automotive radar and deep neural networks

In this work, the authors present results for classification of different classes of targets (car, single and multiple people, bicycle) using automotive radar data and different neural networks. A fast implementation of radar algorithms for detection, tracking, and micro-Doppler extraction is proposed in conjunction with the automotive radar transceiver TEF810X and microcontroller unit SR32R274 manufactured by NXP Semiconductors. Three different types of neural networks are considered, namely a classic convolutional network, a residual network, and a combination of convolutional and recurrent network, for different classification problems across the four classes of targets recorded. Considerable accuracy (close to 100% in some cases) and low latency of the radar pre-processing prior to classification (∼0.55 s to produce a 0.5 s long spectrogram) are demonstrated in this study, and possible shortcomings and outstanding issues are discussed

Study of electromagnetic wave propagation and scattering in Low-THz automotive radar

The development of a new generation of sensors for autonomous vehicles requires the increase of the number of automotive radars on the roads, leading to an inevitable problem of overcrowding of the electromagnetic spectrum in the allocated 77 GHz band. The solution proposed in this research is the migration of the automotive radar operation frequency towards the low-THz band. This thesis reports, firstly, an experimental study on the feasibility of deploying automotive radars working at frequencies above 100 GHz. The study analyses the possible additional attenuation of the electromagnetic waves in adverse weather conditions and the differences in targets reflectivities, in comparison to the performances of current automotive radars. A comprehensive library of reflectivity signatures of a number of road actors is established, to provide a basis for the development of low-THz automotive radars. Secondarily, the thesis discusses and demonstrates the advantages of the employment of low-THz signals to improve the imaging capability of automotive radars, to allow identification and classification of road targets based on high resolution images and micro-Doppler signatures

Fusion of deep representations in multistatic radar networks to counteract the presence of synthetic jamming

Micro-Doppler signatures are extremely valuable in the classification of a wide range of targets. This paper investigates the effects of jamming on the micro-Doppler classification performance and explores a potential deep topology enabling low-bandwidth data fusion between nodes in a multistatic radar network. The topology is based on an array of three independent deep neural networks (DNNs) functioning cooperatively to achieve joint classification. In addition to this, a further DNN is trained to detect the presence of jamming, and from this, it attempts to remedy the degradation effects in the data fusion process. This is applied to the real experimental data gathered with the multistatic radar system, NetRAD, of a human operating with seven combinations of holding a rifle-like object and a heavy backpack that is slung on their shoulders. The resilience of the proposed network is tested by applying synthetic jamming signals into specific radar nodes and observing the networks' ability to respond to these undesired effects. The results of this are compared with a traditional voting system topology, serving as a convenient baseline for this paper

Radar-based Application of Pedestrian and Cyclist Micro-Doppler Signatures for Automotive Safety Systems

Die sensorbasierte Erfassung des Nahfeldes im Kontext des hochautomatisierten Fahrens erfährt einen spürbaren Trend bei der Integration von Radarsensorik. Fortschritte in der Mikroelektronik erlauben den Einsatz von hochauflösenden Radarsensoren, die durch effiziente Verfahren sowohl im Winkel als auch in der Entfernung und im Doppler die Messgenauigkeit kontinuierlich ansteigen lassen. Dadurch ergeben sich neuartige Möglichkeiten bei der Bestimmung der geometrischen und kinematischen Beschaffenheit ausgedehnter Ziele im Fahrzeugumfeld, die zur gezielten Entwicklung von automotiven Sicherheitssystemen herangezogen werden können. Im Rahmen dieser Arbeit werden ungeschützte Verkehrsteilnehmer wie Fußgänger und Radfahrer mittels eines hochauflösenden Automotive-Radars analysiert. Dabei steht die Erscheinung des Mikro-Doppler-Effekts, hervorgerufen durch das hohe Maß an kinematischen Freiheitsgraden der Objekte, im Vordergrund der Betrachtung. Die durch den Mikro-Doppler-Effekt entstehenden charakteristischen Radar-Signaturen erlauben eine detailliertere Perzeption der Objekte und können in direkten Zusammenhang zu ihren aktuellen Bewegungszuständen gesetzt werden. Es werden neuartige Methoden vorgestellt, die die geometrischen und kinematischen Ausdehnungen der Objekte berücksichtigen und echtzeitfähige Ansätze zur Klassifikation und Verhaltensindikation realisieren. Wird ein ausgedehntes Ziel (z.B. Radfahrer) von einem Radarsensor detektiert, können aus dessen Mikro-Doppler-Signatur wesentliche Eigenschaften bezüglich seines Bewegungszustandes innerhalb eines Messzyklus erfasst werden. Die Geschwindigkeitsverteilungen der sich drehenden Räder erlauben eine adaptive Eingrenzung der Tretbewegung, deren Verhalten essentielle Merkmale im Hinblick auf eine vorausschauende Unfallprädiktion aufweist. Ferner unterliegen ausgedehnte Radarziele einer Orientierungsabhängigkeit, die deren geometrischen und kinematischen Profile direkt beeinflusst. Dies kann sich sowohl negativ auf die Klassifikations-Performance als auch auf die Verwertbarkeit von Parametern auswirken, die eine Absichtsbekundung des Radarziels konstituieren. Am Beispiel des Radfahrers wird hierzu ein Verfahren vorgestellt, das die orientierungsabhängigen Parameter in Entfernung und Doppler normalisiert und die gemessenen Mehrdeutigkeiten kompensiert. Ferner wird in dieser Arbeit eine Methodik vorgestellt, die auf Grundlage des Mikro- Doppler-Profils eines Fußgängers dessen Beinbewegungen über die Zeit schätzt (Tracking) und wertvolle Objektinformationen hinsichtlich seines Bewegungsverhaltens offenbart. Dazu wird ein Bewegungsmodell entwickelt, das die nichtlineare Fortbewegung des Beins approximiert und dessen hohes Maß an biomechanischer Variabilität abbildet. Durch die Einbeziehung einer wahrscheinlichkeitsbasierten Datenassoziation werden die Radar-Detektionen ihren jeweils hervorrufenden Quellen (linkes und rechtes Bein) zugeordnet und eine Trennung der Gliedmaßen realisiert. Im Gegensatz zu bisherigen Tracking-Verfahren weist die vorgestellte Methodik eine Steigerung in der Genauigkeit der Objektinformationen auf und stellt damit einen entscheidenden Vorteil für zukünftige Fahrerassistenzsysteme dar, um deutlich schneller auf kritische Verkehrssituationen reagieren zu können.:1 Introduction 1 1.1 Automotive environmental perception 2 1.2 Contributions of this work 4 1.3 Thesis overview 6 2 Automotive radar 9 2.1 Physical fundamentals 9 2.1.1 Radar cross section 9 2.1.2 Radar equation 10 2.1.3 Micro-Doppler effect 11 2.2 Radar measurement model 15 2.2.1 FMCW radar 15 2.2.2 Chirp sequence modulation 17 2.2.3 Direction-of-arrival estimation 22 2.3 Signal processing 25 2.3.1 Target properties 26 2.3.2 Target extraction 28 Power detection 28 Clustering 30 2.3.3 Real radar data example 31 2.4 Conclusion 33 3 Micro-Doppler applications of a cyclist 35 3.1 Physical fundamentals 35 3.1.1 Micro-Doppler signatures of a cyclist 35 3.1.2 Orientation dependence 36 3.2 Cyclist feature extraction 38 3.2.1 Adaptive pedaling extraction 38 Ellipticity constraints 38 Ellipse fitting algorithm 39 3.2.2 Experimental results 42 3.3 Normalization of the orientation dependence 44 3.3.1 Geometric correction 44 3.3.2 Kinematic correction 45 3.3.3 Experimental results 45 3.4 Conclusion 47 3.5 Discussion and outlook 47 4 Micro-Doppler applications of a pedestrian 49 4.1 Pedestrian detection 49 4.1.1 Human kinematics 49 4.1.2 Micro-Doppler signatures of a pedestrian 51 4.1.3 Experimental results 52 Radially moving pedestrian 52 Crossing pedestrian 54 4.2 Pedestrian feature extraction 57 4.2.1 Frequency-based limb separation 58 4.2.2 Extraction of body parts 60 4.2.3 Experimental results 62 4.3 Pedestrian tracking 64 4.3.1 Probabilistic state estimation 65 4.3.2 Gaussian filters 67 4.3.3 The Kalman filter 67 4.3.4 The extended Kalman filter 69 4.3.5 Multiple-object tracking 71 4.3.6 Data association 74 4.3.7 Joint probabilistic data association 80 4.4 Kinematic-based pedestrian tracking 84 4.4.1 Kinematic modeling 84 4.4.2 Tracking motion model 87 4.4.3 4-D radar point cloud 91 4.4.4 Tracking implementation 92 4.4.5 Experimental results 96 Longitudinal trajectory 96 Crossing trajectory with sudden turn 98 4.5 Conclusion 102 4.6 Discussion and outlook 103 5 Summary and outlook 105 5.1 Developed algorithms 105 5.1.1 Adaptive pedaling extraction 105 5.1.2 Normalization of the orientation dependence 105 5.1.3 Model-based pedestrian tracking 106 5.2 Outlook 106 Bibliography 109 List of Acronyms 119 List of Figures 124 List of Tables 125 Appendix 127 A Derivation of the rotation matrix 2.26 127 B Derivation of the mixed radar signal 2.52 129 C Calculation of the marginal association probabilities 4.51 131 Curriculum Vitae 135Sensor-based detection of the near field in the context of highly automated driving is experiencing a noticeable trend in the integration of radar sensor technology. Advances in microelectronics allow the use of high-resolution radar sensors that continuously increase measurement accuracy through efficient processes in angle as well as distance and Doppler. This opens up novel possibilities in determining the geometric and kinematic nature of extended targets in the vehicle environment, which can be used for the specific development of automotive safety systems. In this work, vulnerable road users such as pedestrians and cyclists are analyzed using a high-resolution automotive radar. The focus is on the appearance of the micro-Doppler effect, caused by the objects’ high kinematic degree of freedom. The characteristic radar signatures produced by the micro-Doppler effect allow a clearer perception of the objects and can be directly related to their current state of motion. Novel methods are presented that consider the geometric and kinematic extents of the objects and realize real-time approaches to classification and behavioral indication. When a radar sensor detects an extended target (e.g., bicyclist), its motion state’s fundamental properties can be captured from its micro-Doppler signature within a measurement cycle. The spinning wheels’ velocity distributions allow an adaptive containment of the pedaling motion, whose behavior exhibits essential characteristics concerning predictive accident prediction. Furthermore, extended radar targets are subject to orientation dependence, directly affecting their geometric and kinematic profiles. This can negatively affect both the classification performance and the usability of parameters constituting the radar target’s intention statement. For this purpose, using the cyclist as an example, a method is presented that normalizes the orientation-dependent parameters in range and Doppler and compensates for the measured ambiguities. Furthermore, this paper presents a methodology that estimates a pedestrian’s leg motion over time (tracking) based on the pedestrian’s micro-Doppler profile and reveals valuable object information regarding his motion behavior. To this end, a motion model is developed that approximates the leg’s nonlinear locomotion and represents its high degree of biomechanical variability. By incorporating likelihood-based data association, radar detections are assigned to their respective evoking sources (left and right leg), and limb separation is realized. In contrast to previous tracking methods, the presented methodology shows an increase in the object information’s accuracy. It thus represents a decisive advantage for future driver assistance systems in order to be able to react significantly faster to critical traffic situations.:1 Introduction 1 1.1 Automotive environmental perception 2 1.2 Contributions of this work 4 1.3 Thesis overview 6 2 Automotive radar 9 2.1 Physical fundamentals 9 2.1.1 Radar cross section 9 2.1.2 Radar equation 10 2.1.3 Micro-Doppler effect 11 2.2 Radar measurement model 15 2.2.1 FMCW radar 15 2.2.2 Chirp sequence modulation 17 2.2.3 Direction-of-arrival estimation 22 2.3 Signal processing 25 2.3.1 Target properties 26 2.3.2 Target extraction 28 Power detection 28 Clustering 30 2.3.3 Real radar data example 31 2.4 Conclusion 33 3 Micro-Doppler applications of a cyclist 35 3.1 Physical fundamentals 35 3.1.1 Micro-Doppler signatures of a cyclist 35 3.1.2 Orientation dependence 36 3.2 Cyclist feature extraction 38 3.2.1 Adaptive pedaling extraction 38 Ellipticity constraints 38 Ellipse fitting algorithm 39 3.2.2 Experimental results 42 3.3 Normalization of the orientation dependence 44 3.3.1 Geometric correction 44 3.3.2 Kinematic correction 45 3.3.3 Experimental results 45 3.4 Conclusion 47 3.5 Discussion and outlook 47 4 Micro-Doppler applications of a pedestrian 49 4.1 Pedestrian detection 49 4.1.1 Human kinematics 49 4.1.2 Micro-Doppler signatures of a pedestrian 51 4.1.3 Experimental results 52 Radially moving pedestrian 52 Crossing pedestrian 54 4.2 Pedestrian feature extraction 57 4.2.1 Frequency-based limb separation 58 4.2.2 Extraction of body parts 60 4.2.3 Experimental results 62 4.3 Pedestrian tracking 64 4.3.1 Probabilistic state estimation 65 4.3.2 Gaussian filters 67 4.3.3 The Kalman filter 67 4.3.4 The extended Kalman filter 69 4.3.5 Multiple-object tracking 71 4.3.6 Data association 74 4.3.7 Joint probabilistic data association 80 4.4 Kinematic-based pedestrian tracking 84 4.4.1 Kinematic modeling 84 4.4.2 Tracking motion model 87 4.4.3 4-D radar point cloud 91 4.4.4 Tracking implementation 92 4.4.5 Experimental results 96 Longitudinal trajectory 96 Crossing trajectory with sudden turn 98 4.5 Conclusion 102 4.6 Discussion and outlook 103 5 Summary and outlook 105 5.1 Developed algorithms 105 5.1.1 Adaptive pedaling extraction 105 5.1.2 Normalization of the orientation dependence 105 5.1.3 Model-based pedestrian tracking 106 5.2 Outlook 106 Bibliography 109 List of Acronyms 119 List of Figures 124 List of Tables 125 Appendix 127 A Derivation of the rotation matrix 2.26 127 B Derivation of the mixed radar signal 2.52 129 C Calculation of the marginal association probabilities 4.51 131 Curriculum Vitae 13

A New Form of Interlocking Developing Technology for Level Crossings and Depots with International Applications

There are multiple large rail infrastructure projects planned or currently being undertaken within the United Kingdom. Many of these projects aim to reduce the continual issue of limited or overcapacity service. These projects involve an expansion of Rail lines, introducing faster lines, improved stations in towns and cities and better communication networks. Some major projects like Control Period 6 (CP6) are being managed by Network Rail; where projects are initiated throughout Great Britain. Many projects are managed outside Great Britain e.g., Trans-European Transport Network Program, which is planning for expansion of Rail lines (almost double) for High-Speed Rails (category I and II). These projects will increase the number of junctions and Level Crossings. A Level Crossing is where a Rail Line is crossed by a road or a walkway without the use of a tunnel or bridge. The misuse from the road users account for nearly 90% of the fatalities and near misses at Level Crossings. During 2016/2017, the Rail Network recorded 6 fatalities, about 400 near-misses and more than 77 incidents of shock and trauma. Accidents at Level Crossings represent 8% of the total accidents from the whole Rail Network. Office of Rail and Road (ORR) suggested that among these accidents at Level Crossings 90% of them are pedestrians. Such high numbers of accidents, fatalities and high risk have alarmed authorities. These authorities found it necessary to invest time and utilise given resources to improve the safety system at a Level Crossing using the safer and reliable interlocking system. The interlocking system is a feature of a control system that makes the state of two functions mutually independent. The primary function of Interlocking is to ensure that trains are safe from collision and derailment. Considering the risk associated with the Level Crossing system, the new proposed interlocking system should utilise the sensing system available at a Level Crossing to significantly reduce implementation cost and comply with the given standards and Risk Assessments. The new proposed interlocking system is designed to meet the “Safety Integrity Level- SIL” and possibly use the “2oo2” approach for its application at a Level Crossing, where the operational cycle is automated or train driver is alarmed for risk situations. Importantly, the new proposed system should detect and classify small objects and provide a reasonable solution to the current risk associated with Level Crossing, which was impossible with the traditional sensing systems. The present work discusses the sensors and algorithms used and has the potential to detect and classify objects within a Level Crossing area. The review of existing solutions e.g Inductive Loops and other major sensors allows the reader to understand why RADAR and Video Cameras are preferable choices of a sensing system for a Level Crossing. Video data provides sufficient information for the proposed algorithm to detect and classify objects at Level Crossings without the need of a manual “operator”. The RADAR sensing system can provide information using micro-Doppler signatures, which are generated from small regular movements of an obstacle. The two sensors will make the system a two-layer resilient system. The processed information from these two sensing systems is used as the “2oo2” logic system for Interlocking for automating the operational cycle or alarm the train drive using effective communication e.g., GSM-R. These two sensors provide sufficient information for the proposed algorithm, which will allow the system to automatically make an “intelligent decision” and proceed with a safe Level Crossing operational cycle. Many existing traditional algorithms depend on pixels values, which are compared with background pixels. This approach cannot detect complex textures, adapt to a dynamic background or avoid detection of unnecessary harmless objects. To avoid these problems, the proposed work utilises “Deep Learning” technology integrated with the proposed Vision and RADAR system. The Deep Learning technology can learn representations from labelled pixels; hence it does not depend on background pixels. The Deep 3 | P a g e Learning technology can classify, detect and localise objects at a Level Crossing area. It can classify and differentiate between a child and a small inanimate object, which was impossible with traditional algorithms. The system can detect an object regardless of its position, orientation and scale without any additional training because it learns representation from the data and does not rely on background pixels. The proposed system e.g., Deep Learning technology is integrated with the existing Vision System and RADAR installed at a Level Crossing, hence implementation cost is significantly reduced as well. The proposed work address two main aspects of training a model using Deep Learning technology; training from scratch and training using Transfer Learning techniques. Results are demonstrated for Image Classification, Object Detection and micro-Doppler signals from RADAR. An architecture of Convolutional Neural Network from scratch is trained consisting of Input Layer, Convolution, Pooling and Dropout Layer. The model achieves an accuracy of about 66.78%. Different notable models are trained using Transfer Learning techniques and their results are mentioned along with the MobileNet model, which achieves the highest accuracy of 91.9%. The difference between Image Classification and Object Detection is discussed and results for Object Detection are mentioned as well, where the Loss metrics are used to evaluate the performance of the Object Detector. MobileNet achieves the smallest loss metric of about 0.092. These results clearly show the effectiveness and preferability of these models for their applicability at Level Crossings. Another Convolutional Neural Network is trained using micro-Doppler signatures from the Radar system. The model trained using the micro-Doppler signature achieved an accuracy of 92%. The present work also addresses the Risk Assessment associated with the installation and maintenance of the system using Deep Learning technology. RAMS (Reliability, Availability, Maintainability and Safety) management system is used to address the General and Specific Risks associated with the sensing system integrated with the Deep Learning technology. Finally, the work is concluded with the preferred choice, its application, results and associated Risk Assessment. Deep Learning is an evolving field with new improvements being introduced constantly. Any new challenges and problems should be monitored regularly. Some future work is discussed as well. To further improve the model's accuracy, the dataset from the same distribution should be gathered with the cooperation of relevant Railway authorities. Also, the RADAR dataset could be generated rather than simulated to further include diversity and avoid any biases in the dataset during the training process. Also, the proposed system can be implemented and used in different applications within the Rail Industry e.g., passenger census and classification of passengers at the platform as discussed in the work

Fusion of Deep Representations in Multistatic Radar Networks to Counteract the Presence of Synthetic Jamming

Micro-Doppler signatures are extremely valuable in the classification of a wide range of targets. This work investigates the effects of jamming on micro-Doppler classification performance and explores a potential deep topology enabling low bandwidth data fusion between nodes in a multistatic radar network. The topology is based on an array of three independent deep neural networks (DNNs) functioning cooperatively to achieve joint classification. In addition to this, a further DNN is trained to detect the presence of jamming and from this it attempts to remedy the degradation effects in the data fusion process. This is applied to real experimental data gathered with the multistatic radar system NetRAD, of a human operating with seven combinations of holding a rifle-like object and a heavy backpack which is slung on their shoulders. The resilience of the proposed network is tested by applying synthetic jamming signals into specific radar nodes and observing the networks’ ability to respond to these undesired effects. The results of this are compared with a traditional voting system topology, serving as a convenient baseline for this work

Pre-Trained Driving in Localized Surroundings with Semantic Radar Information and Machine Learning

Entlang der Signalverarbeitungskette von Radar Detektionen bis zur Fahrzeugansteuerung, diskutiert diese Arbeit eine semantischen Radar Segmentierung, einen darauf aufbauenden Radar SLAM, sowie eine im Verbund realisierte autonome Parkfunktion. Die Radarsegmentierung der (statischen) Umgebung wird durch ein Radar-spezifisches neuronales Netzwerk RadarNet erreicht. Diese Segmentierung ermöglicht die Entwicklung des semantischen Radar Graph-SLAM SERALOC. Auf der Grundlage der semantischen Radar SLAM Karte wird eine beispielhafte autonome Parkfunktionalität in einem realen Versuchsträger umgesetzt. Entlang eines aufgezeichneten Referenzfades parkt die Funktion ausschließlich auf Basis der Radar Wahrnehmung mit bisher unerreichter Positioniergenauigkeit. Im ersten Schritt wird ein Datensatz von 8.2 · 10^6 punktweise semantisch gelabelten Radarpunktwolken über eine Strecke von 2507.35m generiert. Es sind keine vergleichbaren Datensätze dieser Annotationsebene und Radarspezifikation öffentlich verfügbar. Das überwachte Training der semantischen Segmentierung RadarNet erreicht 28.97% mIoU auf sechs Klassen. Außerdem wird ein automatisiertes Radar-Labeling-Framework SeRaLF vorgestellt, welches das Radarlabeling multimodal mittels Referenzkameras und LiDAR unterstützt. Für die kohärente Kartierung wird ein Radarsignal-Vorfilter auf der Grundlage einer Aktivierungskarte entworfen, welcher Rauschen und andere dynamische Mehrwegreflektionen unterdrückt. Ein speziell für Radar angepasstes Graph-SLAM-Frontend mit Radar-Odometrie Kanten zwischen Teil-Karten und semantisch separater NDT Registrierung setzt die vorgefilterten semantischen Radarscans zu einer konsistenten metrischen Karte zusammen. Die Kartierungsgenauigkeit und die Datenassoziation werden somit erhöht und der erste semantische Radar Graph-SLAM für beliebige statische Umgebungen realisiert. Integriert in ein reales Testfahrzeug, wird das Zusammenspiel der live RadarNet Segmentierung und des semantischen Radar Graph-SLAM anhand einer rein Radar-basierten autonomen Parkfunktionalität evaluiert. Im Durchschnitt über 42 autonome Parkmanöver (∅3.73 km/h) bei durchschnittlicher Manöverlänge von ∅172.75m wird ein Median absoluter Posenfehler von 0.235m und End-Posenfehler von 0.2443m erreicht, der vergleichbare Radar-Lokalisierungsergebnisse um ≈ 50% übertrifft. Die Kartengenauigkeit von veränderlichen, neukartierten Orten über eine Kartierungsdistanz von ∅165m ergibt eine ≈ 56%-ige Kartenkonsistenz bei einer Abweichung von ∅0.163m. Für das autonome Parken wurde ein gegebener Trajektorienplaner und Regleransatz verwendet

University catalog, 2016-2017

The catalog is a comprehensive reference for your academic studies. It includes a list of all degree programs offered at MU, including bachelors, masters, specialists, doctorates, minors, certificates, and emphasis areas. It details the university wide requirements, the curricular requirements for each program, and in some cases provides a sample plan of study. The catalog includes a complete listing and description of approved courses. It also provides information on academic policies, contact information for supporting offices, and a complete listing of faculty members. -- Page 3