Novel Hybrid-Learning Algorithms for Improved Millimeter-Wave Imaging Systems

Increasing attention is being paid to millimeter-wave (mmWave), 30 GHz to 300 GHz, and terahertz (THz), 300 GHz to 10 THz, sensing applications including security sensing, industrial packaging, medical imaging, and non-destructive testing. Traditional methods for perception and imaging are challenged by novel data-driven algorithms that offer improved resolution, localization, and detection rates. Over the past decade, deep learning technology has garnered substantial popularity, particularly in perception and computer vision applications. Whereas conventional signal processing techniques are more easily generalized to various applications, hybrid approaches where signal processing and learning-based algorithms are interleaved pose a promising compromise between performance and generalizability. Furthermore, such hybrid algorithms improve model training by leveraging the known characteristics of radio frequency (RF) waveforms, thus yielding more efficiently trained deep learning algorithms and offering higher performance than conventional methods. This dissertation introduces novel hybrid-learning algorithms for improved mmWave imaging systems applicable to a host of problems in perception and sensing. Various problem spaces are explored, including static and dynamic gesture classification; precise hand localization for human computer interaction; high-resolution near-field mmWave imaging using forward synthetic aperture radar (SAR); SAR under irregular scanning geometries; mmWave image super-resolution using deep neural network (DNN) and Vision Transformer (ViT) architectures; and data-level multiband radar fusion using a novel hybrid-learning architecture. Furthermore, we introduce several novel approaches for deep learning model training and dataset synthesis.Comment: PhD Dissertation Submitted to UTD ECE Departmen

Stereoscopic Depth Perception Through Foliage

Both humans and computational methods struggle to discriminate the depths of objects hidden beneath foliage. However, such discrimination becomes feasible when we combine computational optical synthetic aperture sensing with the human ability to fuse stereoscopic images. For object identification tasks, as required in search and rescue, wildlife observation, surveillance, and early wildfire detection, depth assists in differentiating true from false findings, such as people, animals, or vehicles vs. sun-heated patches at the ground level or in the tree crowns, or ground fires vs. tree trunks. We used video captured by a drone above dense woodland to test users' ability to discriminate depth. We found that this is impossible when viewing monoscopic video and relying on motion parallax. The same was true with stereoscopic video because of the occlusions caused by foliage. However, when synthetic aperture sensing was used to reduce occlusions and disparity-scaled stereoscopic video was presented, whereas computational (stereoscopic matching) methods were unsuccessful, human observers successfully discriminated depth. This shows the potential of systems which exploit the synergy between computational methods and human vision to perform tasks that neither can perform alone

Emerging Approaches for THz Array Imaging: A Tutorial Review and Software Tool

Accelerated by the increasing attention drawn by 5G, 6G, and Internet of Things applications, communication and sensing technologies have rapidly evolved from millimeter-wave (mmWave) to terahertz (THz) in recent years. Enabled by significant advancements in electromagnetic (EM) hardware, mmWave and THz frequency regimes spanning 30 GHz to 300 GHz and 300 GHz to 3000 GHz, respectively, can be employed for a host of applications. The main feature of THz systems is high-bandwidth transmission, enabling ultra-high-resolution imaging and high-throughput communications; however, challenges in both the hardware and algorithmic arenas remain for the ubiquitous adoption of THz technology. Spectra comprising mmWave and THz frequencies are well-suited for synthetic aperture radar (SAR) imaging at sub-millimeter resolutions for a wide spectrum of tasks like material characterization and nondestructive testing (NDT). This article provides a tutorial review of systems and algorithms for THz SAR in the near-field with an emphasis on emerging algorithms that combine signal processing and machine learning techniques. As part of this study, an overview of classical and data-driven THz SAR algorithms is provided, focusing on object detection for security applications and SAR image super-resolution. We also discuss relevant issues, challenges, and future research directions for emerging algorithms and THz SAR, including standardization of system and algorithm benchmarking, adoption of state-of-the-art deep learning techniques, signal processing-optimized machine learning, and hybrid data-driven signal processing algorithms...Comment: Submitted to Proceedings of IEE

Millimetre wave imaging for concealed target detection

PhDConcealed weapon detection (CWD) has been a hot topic as the concern about pub- lic safety increases. A variety of approaches for the detection of concealed objects on the human body based on earth magnetic ¯eld distortion, inductive magnetic ¯eld, acoustic and ultrasonic, electromagnetic resonance, MMW (millimetre wave), THz, Infrared, x-ray technologies have been suggested and developed. Among all of them, MMW holographic imaging is considered as a promising approach due to the relatively high penetration and high resolution that it can o®er. Typical concealed target detection methods are classi¯ed into 2 categories, the ¯rst one is a resonance based target identi¯cation technique, and the second one is an imaging based system. For the former, the complex natural resonance (CNR) frequencies associated with a certain target are extracted and used for identi¯cation, but this technique has an issue of high false alarm rate. The microwave/millimetre wave imaging systems can be categorized into two types: passive systems and active sys- tems. For the active microwave/millimetre wave imaging systems, the microwave holographic imaging approach was adopted in this thesis. Such a system can oper- ate at either a single frequency or multiple frequencies (wide band). An active, coherent, single frequency operation millimetre wave imaging system based on the theory of microwave holography was developed. Based on literature surveys and ¯rst hand experimental results, this thesis aims to provide system level parame- ter determination to aid the development of a target detection imager. The goal is approached step by step in 7 chapters, with topics and issues addressed rang- ing from reviewing the past work, ¯nding out the best candidate technology, i.e. the MMW holographic imaging combined with the resonance based target recog- i nition technique, the construction of the 94 GHz MMW holographic prototype imager, experimental trade-o® investigation of system parameters, imager per- formance evaluation, low pro¯le components and image enhancement techniques, feasibility investigation of resonance based technique, to system implementation based on the parameters and results achieved. The task set forth in the beginning is completed by coming up with an entire system design in the end.

Towards joint communication and sensing (Chapter 4)

Localization of user equipment (UE) in mobile communication networks has been supported from the early stages of 3rd generation partnership project (3GPP). With 5th Generation (5G) and its target use cases, localization is increasingly gaining importance. Integrated sensing and localization in 6th Generation (6G) networks promise the introduction of more efficient networks and compelling applications to be developed

OCM 2023 - Optical Characterization of Materials : Conference Proceedings

The state of the art in the optical characterization of materials is advancing rapidly. New insights have been gained into the theoretical foundations of this research and exciting developments have been made in practice, driven by new applications and innovative sensor technologies that are constantly evolving. The great success of past conferences proves the necessity of a platform for presentation, discussion and evaluation of the latest research results in this interdisciplinary field

Learning to Detect Open Carry and Concealed Object with 77GHz Radar

Detecting harmful carried objects plays a key role in intelligent surveillance systems and has widespread applications, for example, in airport security. In this paper, we focus on the relatively unexplored area of using low-cost 77GHz mmWave radar for the carried objects detection problem. The proposed system is capable of real-time detecting three classes of objects - laptop, phone, and knife - under open carry and concealed cases where objects are hidden with clothes or bags. This capability is achieved by the initial signal processing for localization and generating range-azimuth-elevation image cubes, followed by a deep learning-based prediction network and a multi-shot post-processing module for detecting objects. Extensive experiments for validating the system performance on detecting open carry and concealed objects have been presented with a self-built radar-camera testbed and collected dataset. Additionally, the influence of different input formats, factors, and parameters on system performance is analyzed, providing an intuitive understanding of the system. This system would be the very first baseline for other future works aiming to detect carried objects using 77GHz radar.Comment: 12 page

Photometric Depth Super-Resolution

This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2019. First three authors contribute equall