Super-Resolution Radar Imaging with Sparse Arrays Using a Deep Neural Network Trained with Enhanced Virtual Data

This paper introduces a method based on a deep neural network (DNN) that is perfectly capable of processing radar data from extremely thinned radar apertures. The proposed DNN processing can provide both aliasing-free radar imaging and super-resolution. The results are validated by measuring the detection performance on realistic simulation data and by evaluating the Point-Spread-function (PSF) and the target-separation performance on measured point-like targets. Also, a qualitative evaluation of a typical automotive scene is conducted. It is shown that this approach can outperform state-of-the-art subspace algorithms and also other existing machine learning solutions. The presented results suggest that machine learning approaches trained with sufficiently sophisticated virtual input data are a very promising alternative to compressed sensing and subspace approaches in radar signal processing. The key to this performance is that the DNN is trained using realistic simulation data that perfectly mimic a given sparse antenna radar array hardware as the input. As ground truth, ultra-high resolution data from an enhanced virtual radar are simulated. Contrary to other work, the DNN utilizes the complete radar cube and not only the antenna channel information at certain range-Doppler detections. After training, the proposed DNN is capable of sidelobe- and ambiguity-free imaging. It simultaneously delivers nearly the same resolution and image quality as would be achieved with a fully occupied array.Comment: 15 pages, 12 figures, Accepted to IEEE Journal of Microwave

An Adversarial Super-Resolution Remedy for Radar Design Trade-offs

Radar is of vital importance in many fields, such as autonomous driving, safety and surveillance applications. However, it suffers from stringent constraints on its design parametrization leading to multiple trade-offs. For example, the bandwidth in FMCW radars is inversely proportional with both the maximum unambiguous range and range resolution. In this work, we introduce a new method for circumventing radar design trade-offs. We propose the use of recent advances in computer vision, more specifically generative adversarial networks (GANs), to enhance low-resolution radar acquisitions into higher resolution counterparts while maintaining the advantages of the low-resolution parametrization. The capability of the proposed method was evaluated on the velocity resolution and range-azimuth trade-offs in micro-Doppler signatures and FMCW uniform linear array (ULA) radars, respectively.Comment: Accepted in EUSIPCO 2019, 5 page

Coherent, super resolved radar beamforming using self-supervised learning

High resolution automotive radar sensors are required in order to meet the high bar of autonomous vehicles needs and regulations. However, current radar systems are limited in their angular resolution causing a technological gap. An industry and academic trend to improve angular resolution by increasing the number of physical channels, also increases system complexity, requires sensitive calibration processes, lowers robustness to hardware malfunctions and drives higher costs. We offer an alternative approach, named Radar signal Reconstruction using Self Supervision (R2-S2), which significantly improves the angular resolution of a given radar array without increasing the number of physical channels. R2-S2 is a family of algorithms which use a Deep Neural Network (DNN) with complex range-Doppler radar data as input and trained in a self-supervised method using a loss function which operates in multiple data representation spaces. Improvement of 4x in angular resolution was demonstrated using a real-world dataset collected in urban and highway environments during clear and rainy weather conditions.Comment: 28 pages 10 figure

Investigation of Non-coherent Discrete Target Range Estimation Techniques for High-precision Location

Ranging is an essential and crucial task for radar systems. How to solve the range-detection problem effectively and precisely is massively important. Meanwhile, unambiguity and high resolution are the points of interest as well. Coherent and non-coherent techniques can be applied to achieve range estimation, and both of them have advantages and disadvantages. Coherent estimates offer higher precision but are more vulnerable to noise and clutter and phase wrap errors, particularly in a complex or harsh environment, while the non-coherent approaches are simpler but provide lower precision. With the purpose of mitigating inaccuracy and perturbation in range estimation, miscellaneous techniques are employed to achieve optimally precise detection. Numerous elegant processing solutions stemming from non-coherent estimate are now introduced into the coherent realm, and vice versa. This thesis describes two non-coherent ranging estimate techniques with novel algorithms to mitigate the instinct deficit of non-coherent ranging approaches. One technique is based on peak detection and realised by Kth-order Polynomial Interpolation, while another is based on Z-transform and realised by Most-likelihood Chirp Z-transform. A two-stage approach for the fine ranging estimate is applied to the Discrete Fourier transform domain of both algorithms. An N-point Discrete Fourier transform is implemented to attain a coarse estimation; an accurate process around the point of interest determined in the first stage is conducted. For KPI technique, it interpolates around the peak of Discrete Fourier transform profiles of the chirp signal to achieve accurate interpolation and optimum precision. For Most-likelihood Chirp Z-transform technique, the Chirp Z-transform accurately implements the periodogram where only a narrow band spectrum is processed. Furthermore, the concept of most-likelihood estimator is introduced to combine with Chirp Z-transform to acquire better ranging performance. Cramer-Rao lower bound is presented to evaluate the performance of these two techniques from the perspective of statistical signal processing. Mathematical derivation, simulation modelling, theoretical analysis and experimental validation are conducted to assess technique performance. Further research will be pushed forward to algorithm optimisation and system development of a location system using non-coherent techniques and make a comparison to a coherent approach

Millimetre-wave radar development for high resolution detection

Automotive technology today is focusing on autonomous vehicle development. The sensors for these systems include radars due to their robustness against adverse weather conditions such as rain, fog, ash or snow. In this constant search for advancement, high resolution systems play a central role in target detection and avoidance. In this PhD project, these methods have been researched and engineered to leverage the best radar resolution for collision avoidance systems. The first part of this thesis will focus on the existing systems consisting of the state-of-the-art at the time of writing and explain what makes a high resolution radar and how it can cover the whole field of view. The second part will focus on how a non-uniform sparse radar system was simulated, developed and benchmarked for improved radar performance up to 40% better than conventional designs. The third part will focus on signal processing techniques and how these methods have achieved high resolution and detection: large virtual aperture array using Multiple Input Multiple Output (MIMO) systems, beampattern multiplication to improve side-lobe levels and compressive sensing. Also, the substrate-integrated waveguide (SIW) antennas which have been fabricated provide a bandwidth of 1.5GHz for the transmitter and 2GHz at the receiver. This has resulted in a range resolution of 10 cm. The four part of this thesis presents the measurements which have been carried out at the facilities within Heriot-Watt University and also at Netherlands Organisation for Applied Scientific Research (TNO). The results were better than expected since a two transmitter four receiver system was able to detect targets which have been separated at 2.2◦ in angle in the horizontal plane. Also, compressive sensing was used as a high resolution method for obtaining fine target detection and in combination with the multiplication method showed improved detection performance with a 20 dB side-lobe level suppression. The measurement results from the 6-months placements are presented and compared with the state-of the art, revealing that the developed radar is comparable in performance to high-grade automotive radars developed in the industry

Novel Hybrid-Learning Algorithms for Improved Millimeter-Wave Imaging Systems

Increasing attention is being paid to millimeter-wave (mmWave), 30 GHz to 300 GHz, and terahertz (THz), 300 GHz to 10 THz, sensing applications including security sensing, industrial packaging, medical imaging, and non-destructive testing. Traditional methods for perception and imaging are challenged by novel data-driven algorithms that offer improved resolution, localization, and detection rates. Over the past decade, deep learning technology has garnered substantial popularity, particularly in perception and computer vision applications. Whereas conventional signal processing techniques are more easily generalized to various applications, hybrid approaches where signal processing and learning-based algorithms are interleaved pose a promising compromise between performance and generalizability. Furthermore, such hybrid algorithms improve model training by leveraging the known characteristics of radio frequency (RF) waveforms, thus yielding more efficiently trained deep learning algorithms and offering higher performance than conventional methods. This dissertation introduces novel hybrid-learning algorithms for improved mmWave imaging systems applicable to a host of problems in perception and sensing. Various problem spaces are explored, including static and dynamic gesture classification; precise hand localization for human computer interaction; high-resolution near-field mmWave imaging using forward synthetic aperture radar (SAR); SAR under irregular scanning geometries; mmWave image super-resolution using deep neural network (DNN) and Vision Transformer (ViT) architectures; and data-level multiband radar fusion using a novel hybrid-learning architecture. Furthermore, we introduce several novel approaches for deep learning model training and dataset synthesis.Comment: PhD Dissertation Submitted to UTD ECE Departmen

Sensors for Vital Signs Monitoring

Sensor technology for monitoring vital signs is an important topic for various service applications, such as entertainment and personalization platforms and Internet of Things (IoT) systems, as well as traditional medical purposes, such as disease indication judgments and predictions. Vital signs for monitoring include respiration and heart rates, body temperature, blood pressure, oxygen saturation, electrocardiogram, blood glucose concentration, brain waves, etc. Gait and walking length can also be regarded as vital signs because they can indirectly indicate human activity and status. Sensing technologies include contact sensors such as electrocardiogram (ECG), electroencephalogram (EEG), photoplethysmogram (PPG), non-contact sensors such as ballistocardiography (BCG), and invasive/non-invasive sensors for diagnoses of variations in blood characteristics or body fluids. Radar, vision, and infrared sensors can also be useful technologies for detecting vital signs from the movement of humans or organs. Signal processing, extraction, and analysis techniques are important in industrial applications along with hardware implementation techniques. Battery management and wireless power transmission technologies, the design and optimization of low-power circuits, and systems for continuous monitoring and data collection/transmission should also be considered with sensor technologies. In addition, machine-learning-based diagnostic technology can be used for extracting meaningful information from continuous monitoring data

Signal processing architectures for automotive high-resolution MIMO radar systems

To date, the digital signal processing for an automotive radar sensor has been handled in an efficient way by general purpose signal processors and microcontrollers. However, increasing resolution requirements for automated driving on the one hand, as well as rapidly growing numbers of manufactured sensors on the other hand, can provoke a paradigm change in the near future. The design and development of highly specialized hardware accelerators could become a viable option - at least for the most demanding processing steps with data rates of several gigabits per second. In this work, application-specific signal processing architectures for future high-resolution multiple-input and multiple-output (MIMO) radar sensors are designed, implemented, investigated and optimized. A focus is set on real-time performance such that even sophisticated algorithms can be computed sufficiently fast. The full processing chain from the received baseband signals to a list of detections is considered, comprising three major steps: Spectrum analysis, target detection and direction of arrival estimation. The developed architectures are further implemented on a field-programmable gate array (FPGA) and important measurements like resource consumption, power dissipation or data throughput are evaluated and compared with other examples from literature. A substantial dataset, based on more than 3600 different parametrizations and variants, has been established with the help of a model-based design space exploration and is provided as part of this work. Finally, an experimental radar sensor has been built and is used under real-world conditions to verify the effectiveness of the proposed signal processing architectures.Bisher wurde die digitale Signalverarbeitung für automobile Radarsensoren auf eine effiziente Art und Weise von universell verwendbaren Mikroprozessoren bewältigt. Jedoch können steigende Anforderungen an das Auflösungsvermögen für hochautomatisiertes Fahren einerseits, sowie schnell wachsende Stückzahlen produzierter Sensoren andererseits, einen Paradigmenwechsel in naher Zukunft bewirken. Die Entwicklung von hochgradig spezialisierten Hardwarebeschleunigern könnte sich als eine praktikable Alternative etablieren - zumindest für die anspruchsvollsten Rechenschritte mit Datenraten von mehreren Gigabits pro Sekunde. In dieser Arbeit werden anwendungsspezifische Signalverarbeitungsarchitekturen für zukünftige, hochauflösende, MIMO Radarsensoren entworfen, realisiert, untersucht und optimiert. Der Fokus liegt dabei stets auf der Echtzeitfähigkeit, sodass selbst anspruchsvolle Algorithmen in einer ausreichend kurzen Zeit berechnet werden können. Die komplette Signalverarbeitungskette, beginnend von den empfangenen Signalen im Basisband bis hin zu einer Liste von Detektion, wird in dieser Arbeit behandelt. Die Kette gliedert sich im Wesentlichen in drei größere Teilschritte: Spektralanalyse, Zieldetektion und Winkelschätzung. Des Weiteren werden die entwickelten Architekturen auf einem FPGA implementiert und wichtige Kennzahlen wie Ressourcenverbrauch, Stromverbrauch oder Datendurchsatz ausgewertet und mit anderen Beispielen aus der Literatur verglichen. Ein umfangreicher Datensatz, welcher mehr als 3600 verschiedene Parametrisierungen und Varianten beinhaltet, wurde mit Hilfe einer modellbasierten Entwurfsraumexploration erstellt und ist in dieser Arbeit enthalten. Schließlich wurde ein experimenteller Radarsensor aufgebaut und dazu benutzt, die entworfenen Signalverarbeitungsarchitekturen unter realen Umgebungsbedingungen zu verifizieren

Using Radio Frequency and Motion Sensing to Improve Camera Sensor Systems

Camera-based sensor systems have advanced significantly in recent years. This advancement is a combination of camera CMOS (complementary metal-oxide-semiconductor) hardware technology improvement and new computer vision (CV) algorithms that can better process the rich information captured. As the world becoming more connected and digitized through increased deployment of various sensors, cameras have become a cost-effective solution with the advantages of small sensor size, intuitive sensing results, rich visual information, and neural network-friendly. The increased deployment and advantages of camera-based sensor systems have fueled applications such as surveillance, object detection, person re-identification, scene reconstruction, visual tracking, pose estimation, and localization. However, camera-based sensor systems have fundamental limitations such as extreme power consumption, privacy-intrusive, and inability to see-through obstacles and other non-ideal visual conditions such as darkness, smoke, and fog. In this dissertation, we aim to improve the capability and performance of camera-based sensor systems by utilizing additional sensing modalities such as commodity WiFi and mmWave (millimeter wave) radios, and ultra-low-power and low-cost sensors such as inertial measurement units (IMU). In particular, we set out to study three problems: (1) power and storage consumption of continuous-vision wearable cameras, (2) human presence detection, localization, and re-identification in both indoor and outdoor spaces, and (3) augmenting the sensing capability of camera-based systems in non-ideal situations. We propose to use an ultra-low-power, low-cost IMU sensor, along with readily available camera information, to solve the first problem. WiFi devices will be utilized in the second problem, where our goal is to reduce the hardware deployment cost and leverage existing WiFi infrastructure as much as possible. Finally, we will use a low-cost, off-the-shelf mmWave radar to extend the sensing capability of a camera in non-ideal visual sensing situations.Doctor of Philosoph

Direction of arrival estimation using a multiple-input-multiple-output radar with applications to automobiles

The thesis at hand investigates the direction of arrival (DOA) estimation using a Multiple-Input-Multiple-Output (MIMO) radar system. The application of MIMO radars in automobiles is studied. A MIMO radar consists of several transmitting (Tx) and receiving (Rx) antennas. We focus on a time division multiplexed (TDM) MIMO radar with colocated Tx and Rx antennas. The motivation is the use of a radar as a security system in automotive applications, e.g. to identify a dangerous situation and react automatically. Security systems must be very reliable. Hence, besides a good estimation of the distance and velocity, a high performance in DOA estimation is necessary. This is a demanding task, since only a small number of antennas is used and the radar is limited to a small geometrical size. Compared to the corresponding Single-Input-Multiple-Output (SIMO) radar, a MIMO radar with colocated antennas can achieve a higher accuracy in DOA estimation due to its larger virtual aperture. Therefore it is a promising technique for the use in automobiles. The obtained results of this thesis enable us to find optimal TDM schemes which yield a very high DOA accuracy for targets which are stationary as well as for targets which are moving relative to the radar system. The results are not confined to MIMO radars in automobiles, but can be used in other applications as well.In der vorliegenden Arbeit wird die Winkelschätzung (auch Einfallsrichtung genannt, engl. Direction of Arrival (DOA)) mit Hilfe eines Multiple-Input-Multiple-Output (MIMO) Radars untersucht. Darüber hinaus wird die Verwendung eines MIMO Radars in automobilien Anwendungen betrachtet. Ein MIMO Radar besteht aus mehreren Sende- (Tx) und Empfangsantennen (Rx). Wir betrachten insbesondere MIMO Radare die im Zeitmultiplexverfahren (engl. time division multiplex (TDM)) betrieben werden und geometrisch nahe beieinander liegende Antennen (engl. colocated) besitzen. Die Motivation dieser Untersuchungen ist die Verwendung von Radarsystemen als Sicherheitssysteme in Fahrzeugen, z.B. um eine gefährliche Situation zu detektieren und darauf automatisch zu reagieren. Sicherheitssysteme müssen sehr zuverlässig sein. Daher ist neben einer genauen Abstands- und Geschwindigkeitsschätzung auch eine hohe Performance in der Winkelschätzung nötig. Dies ist eine anspruchsvolle Aufgabe, da nur eine geringe Anzahl an Antennen zur Verfügung steht und das Radarsystem nur eine kleine geometrische Größe aufweisen darf. Im Vergleich zu einem entsprechenden Single-Input-Multiple-Output (SIMO) Radar kann ein colocated MIMO Radar aufgrund seiner größeren virtuellen Apertur eine höhere Winkelgenauigkeit erreichen. Daher ist es eine vielversprechende Technik für die Anwendung in Fahrzeugen. Die Ergebnisse dieser Arbeit ermöglichen uns optimale Zeitmultiplexverfahren zu finden, welche sowohl für stationäre Objekte als auch für Objekte die sich relativ zum Radar bewegen, eine hohe Winkelgenauigkeit erreichen. Die Ergebnisse beschränken sich nicht nur auf Radare in Fahrzeugen, sondern können auch in anderen Anwendungen verwendet werden