Novel Hybrid-Learning Algorithms for Improved Millimeter-Wave Imaging Systems

Increasing attention is being paid to millimeter-wave (mmWave), 30 GHz to 300 GHz, and terahertz (THz), 300 GHz to 10 THz, sensing applications including security sensing, industrial packaging, medical imaging, and non-destructive testing. Traditional methods for perception and imaging are challenged by novel data-driven algorithms that offer improved resolution, localization, and detection rates. Over the past decade, deep learning technology has garnered substantial popularity, particularly in perception and computer vision applications. Whereas conventional signal processing techniques are more easily generalized to various applications, hybrid approaches where signal processing and learning-based algorithms are interleaved pose a promising compromise between performance and generalizability. Furthermore, such hybrid algorithms improve model training by leveraging the known characteristics of radio frequency (RF) waveforms, thus yielding more efficiently trained deep learning algorithms and offering higher performance than conventional methods. This dissertation introduces novel hybrid-learning algorithms for improved mmWave imaging systems applicable to a host of problems in perception and sensing. Various problem spaces are explored, including static and dynamic gesture classification; precise hand localization for human computer interaction; high-resolution near-field mmWave imaging using forward synthetic aperture radar (SAR); SAR under irregular scanning geometries; mmWave image super-resolution using deep neural network (DNN) and Vision Transformer (ViT) architectures; and data-level multiband radar fusion using a novel hybrid-learning architecture. Furthermore, we introduce several novel approaches for deep learning model training and dataset synthesis.Comment: PhD Dissertation Submitted to UTD ECE Departmen

Emerging Approaches for THz Array Imaging: A Tutorial Review and Software Tool

Accelerated by the increasing attention drawn by 5G, 6G, and Internet of Things applications, communication and sensing technologies have rapidly evolved from millimeter-wave (mmWave) to terahertz (THz) in recent years. Enabled by significant advancements in electromagnetic (EM) hardware, mmWave and THz frequency regimes spanning 30 GHz to 300 GHz and 300 GHz to 3000 GHz, respectively, can be employed for a host of applications. The main feature of THz systems is high-bandwidth transmission, enabling ultra-high-resolution imaging and high-throughput communications; however, challenges in both the hardware and algorithmic arenas remain for the ubiquitous adoption of THz technology. Spectra comprising mmWave and THz frequencies are well-suited for synthetic aperture radar (SAR) imaging at sub-millimeter resolutions for a wide spectrum of tasks like material characterization and nondestructive testing (NDT). This article provides a tutorial review of systems and algorithms for THz SAR in the near-field with an emphasis on emerging algorithms that combine signal processing and machine learning techniques. As part of this study, an overview of classical and data-driven THz SAR algorithms is provided, focusing on object detection for security applications and SAR image super-resolution. We also discuss relevant issues, challenges, and future research directions for emerging algorithms and THz SAR, including standardization of system and algorithm benchmarking, adoption of state-of-the-art deep learning techniques, signal processing-optimized machine learning, and hybrid data-driven signal processing algorithms...Comment: Submitted to Proceedings of IEE

Radar Imaging in Challenging Scenarios from Smart and Flexible Platforms

undefine

Investigating Key Techniques to Leverage the Functionality of Ground/Wall Penetrating Radar

Ground penetrating radar (GPR) has been extensively utilized as a highly efficient and non-destructive testing method for infrastructure evaluation, such as highway rebar detection, bridge decks inspection, asphalt pavement monitoring, underground pipe leakage detection, railroad ballast assessment, etc. The focus of this dissertation is to investigate the key techniques to tackle with GPR signal processing from three perspectives: (1) Removing or suppressing the radar clutter signal; (2) Detecting the underground target or the region of interest (RoI) in the GPR image; (3) Imaging the underground target to eliminate or alleviate the feature distortion and reconstructing the shape of the target with good fidelity. In the first part of this dissertation, a low-rank and sparse representation based approach is designed to remove the clutter produced by rough ground surface reflection for impulse radar. In the second part, Hilbert Transform and 2-D Renyi entropy based statistical analysis is explored to improve RoI detection efficiency and to reduce the computational cost for more sophisticated data post-processing. In the third part, a back-projection imaging algorithm is designed for both ground-coupled and air-coupled multistatic GPR configurations. Since the refraction phenomenon at the air-ground interface is considered and the spatial offsets between the transceiver antennas are compensated in this algorithm, the data points collected by receiver antennas in time domain can be accurately mapped back to the spatial domain and the targets can be imaged in the scene space under testing. Experimental results validate that the proposed three-stage cascade signal processing methodologies can improve the performance of GPR system

高精度3次元近距離レーダに関する研究

Tohoku University佐藤源之課

실시간 근거리 영상화를 위한 MIMO 역합성 개구 레이더 시스템

학위논문(박사) -- 서울대학교대학원 : 공과대학 전기·정보공학부, 2022. 8. 남상욱.Microwave and millimeter wave (micro/mmW) imaging systems have advantages over other imaging systems in that they have penetration properties over non-metallic structures and non-ionization. However, these systems are commercially applicable in limited areas. Depending on the quality and size of the images, a system can be expensive and images cannot be provided in real-time. To overcome the challenges of the current micro/mmW imaging system, it is critical to suggest a new system concept and prove its potential benefits and hazards by demonstrating the testbed. This dissertation presents Ku1DMIC, a wide-band micro/mmW imaging system using Ku-band and 1D-MIMO array, which can overcome the challenges above. For cost-effective 3D imaging capabilities, Ku1DMIC uses 1D-MIMO array configuration and inverse synthetic aperture radar (ISAR) technique. At the same time, Ku1DMIC supports real-time data acquisition through a system-level design of a seamless interface with frequency modulated continuous wave (FMCW) radar. To show the feasibility of 3D imaging with Ku1DMIC and its real-time capabilities, an accelerated imaging algorithm, 1D-MIMO-ISAR RSA, is proposed and demonstrated. The detailed contributions of the dissertation are as follows. First, this dissertation presents Ku1DMIC – a Ku-band MIMO frequency-modulated continuous-wave (FMCW) radar experimental platform with real-time 2D near-field imaging capabilities. The proposed system uses Ku-band to cover the wider illumination area given the limited number of antennas and uses a fast ramp and wide-band FMCW waveform for rapid radar data acquisition while providing high-resolution images. The key design aspect behind the platform is stability, reconfigurability, and real-time capabilities, which allows investigating the exploration of the system’s strengths and weaknesses. To satisfy the design aspect, a digitally assisted platform is proposed and realized based on an AMD-Xilinx UltraScale+ Radio Frequency System on Chip (RFSoC). The experimental investigation for real-time 2D imaging has proved the ability of video-rate imaging at around 60 frames per second. Second, a waveform digital pre-distortion (DPD) method and calibration method are proposed to enhance the image quality. Even if a clean FMCW waveform is generated with the aid of the optimized waveform generator, the signal will inevitably suffer from distortion, especially in the RF subsystem of the platform. In near-field imaging applications, the waveform DPD is not effective at suppressing distortion in wide-band FMCW radar systems. To solve this issue, the LO-DPD architecture and binary search based DPD algorithm are proposed to make the waveform DPD effective in Ku1DMIC. Furthermore, an image-domain optimization correction method is proposed to compensate for the remaining errors that cannot be eliminated by the waveform DPD. For robustness to various unwanted signals such as noise and clutter signals, two regularized least squares problems are applied and compared: the generalized Tikhonov regularization and the total variation (TV) regularization. Through various 2D imaging experiments, it is confirmed that both methods can enhance the image quality by reducing the sidelobe level. Lastly, the research is conducted to realize real-time 3D imaging by applying the ISAR technique to Ku1DMIC. The realization of real-time 3D imaging using 1D-MIMO array configuration is impactful in that this configuration can significantly reduce the costs of the 3D imaging system and enable imaging of moving objects. To this end, the signal model for the 1D-MIMO-ISAR configuration is presented, and then the 1D-MIMO-ISAR range stacking algorithm (RSA) is proposed to accelerate the imaging reconstruction process. The proposed 1D-MIMO-ISAR RSA can reconstruct images within hundreds of milliseconds while maintaining almost the same image quality as the back-projection algorithm, bringing potential use for real-time 3D imaging. It also describes strategies for setting ROI, considering the real-world situations in which objects enter and exit the field of view, and allocating GPU memory. Extensive simulations and experiments have demonstrated the feasibility and potential benefits of 1D-MIMO-IASR configuration and 1D-MIMO-ISAR RSA.마이크로파 및 밀리미터파(micro/mmW) 영상화 시스템은 비금속 구조 및 비이온화에 비해 침투 특성이 있다는 점에서 다른 이미징 시스템에 비해 장점이 있다. 그러나 이러한 시스템은 제한된 영역에서만 상업적으로 적용되고 있다. 이미지의 품질과 크기에 따라 시스템이 매우 고가일 수 있으며 이미지를 실시간으로 제공할 수 없는 현황이다. 현재의 micro/mmW 이미징 시스템의 문제를 극복하려면 새로운 시스템 개념을 제안하고 테스트베드를 시연하여 잠재적인 이점과 위험을 입증하는 것이 중요하다. 본 논문에서는 Ku-band와 1D-MIMO 어레이를 이용한 광대역 micro/mmW 이미징 시스템인 Ku1DMIC를 제안하여 위와 같은 문제점을 극복할 수 있다. 비용 효율적인 3차원 영상화 기능을 위해 Ku1DMIC는 1D-MIMO 배열 기술과 ISAR(Inverse Synthetic Aperture Radar) 기술을 사용한다. 동시에 Ku1DMIC는 주파수 변조 연속파 (FMCW) 레이더와의 원활한 인터페이스의 시스템 수준 설계를 통해 실시간 데이터 수집을 지원한다. Ku1DMIC를 사용한 3차원 영상화의 구현 및 실시간 기능의 가능성을 보여주기 위해, 2차원 영상화를 위한 1D-MIMO RSA과 3차원 영상화를 위한 1D-MIMO-ISAR RSA가 제안되고 Ku1DMIC에서 구현된다. 따라서, 본 학위 논문의 주요 기여는 Ku-band 1D-MIMO 배열 기반 영상화 시스템 프로토타입을 개발 및 테스트하고, ISAR 기반 3차원 영상화 기능을 검사하고, 실시간 3차원 영상화 가능성을 조사하는 것이다. 이에 대한 세부적인 기여 항목은 다음과 같다. 첫째, 실시간 2D 근거리장 이미징 기능을 갖춘 Ku 대역 MIMO 주파수 변조 연속파(FMCW) 레이더 실험 플랫폼인 Ku1DMIC를 제시한다. 제안하는 시스템은 제한된 수의 안테나에서 더 넓은 조명 영역을 커버하기 위해 Ku 대역을 사용하고 고해상도 이미지를 제공하면서 빠른 레이더 데이터 수집을 위해 고속 램프 및 광대역 FMCW 파형을 사용한다. 플랫폼의 핵심 설계 원칙은 안정성, 재구성 가능성 및 실시간 기능으로 시스템의 강점과 약점을 광범위하게 탐색한다. 설계 원칙을 만족시키기 위해 AMD-Xilinx UltraScale+ RFSoC(Radio Frequency System on Chip)를 기반으로 디지털 지원 플랫폼을 제안하고 구현한다. 실시간 2D 이미징에 대한 실험적 조사는 초당 약 60프레임에서 비디오 속도 이미징의 능력을 입증했다. 둘째, 영상 품질 향상을 위한 파형 디지털 전치왜곡(DPD) 방법과 보정 방법을 제안한다. 최적화된 파형 발생기의 도움으로 깨끗한 FMCW 파형이 생성되더라도 특히 플랫폼의 RF 하위 시스템에서 신호는 필연적으로 왜곡을 겪게된다. 근거리 영상화 응용 분야에서는 파형 DPD는 광대역 FMCW 레이더 시스템의 왜곡을 억제하는 데 효과적이지 않다. 이 문제를 해결하기 위해 Ku1DMIC에서 파형 DPD가 유효하도록 LO-DPD 아키텍처와 이진 탐색 기반 DPD 알고리즘을 제안한다. 또한, 파형 DPD로 제거할 수 없는 나머지 오류를 보상하기 위해 이미지 영역 최적화 보정 방법을 제안한다. 노이즈 및 클러터 신호와 같은 다양한 원치 않는 신호에 대한 견고성을 위해 일반화된 Tikhonov 정규화 및 전체 변동(TV) 정규화라는 두 가지 정규화된 최소 자승 문제를 적용 후 비교한다. 다양한 2차원 영상화 실험을 통해 두 방법 모두 부엽 레벨을 줄여 화질을 향상시킬 수 있음을 확인한다. 마지막으로, ISAR 기법을 2차원 영상 플랫폼에 적용하여 실시간 3차원 영상을 구현하기 위한 연구를 진행한다. 1D-MIMO-ISAR 구성에서 실시간 3D 이미징의 구현은 이러한 구성이 3D 이미징 시스템의 비용을 크게 줄일 수 있다는 점에서 영향력이 있다. 따라서 이 논문에서는 1D-MIMO-ISAR 구성에 대한 이미징 재구성을 가속화하기 위해 1D-MIMO-ISAR 범위 스태킹 알고리즘(RSA)을 제안한다. 제안된 1D-MIMO-ISAR RSA는 널리 알려진 Back-Projection 알고리즘과 거의 동일한 이미지 품질을 유지하면서도 수백 밀리초 이내에 이미지를 재구성함으로써 실시간 영상화에 대한 가능성을 보여준다. 또한 물체가 시야에 들어오고 나가는 실제 상황을 고려하기 위한 ROI 설정, 그리고 메모리 할당에 대한 전략을 설명한다. 광범위한 시뮬레이션과 실험을 통해 1D-MIMO-IASR 구성 및 1D-MIMO-ISAR RSA의 가능성과 잠재적 이점을 확인한다.1 INTRODUCTION 1 1.1 Microwave and millimeter-wave imaging 1 1.2 Imaging with radar system 2 1.3 Challenges and motivation 5 1.4 Outline of the dissertation 8 2 FUNDAMENTAL OF TWO-DIMENSIONAL IMAGING USING A MIMO RADAR 9 2.1 Signal model 9 2.2 Consideration of waveform 12 2.3 Image reconstruction algorithm 16 2.3.1 Back-projection algorithm 16 2.3.2 1D-MIMO range-migration algorithm 20 2.3.3 1D-MIMO range stacking algorithm 27 2.4 Sampling criteria and resolution 31 2.5 Simulation results 36 3 MIMO-FMCW RADAR IMPLEMENTATION WITH 16 TX - 16 RX ONE- DIMENSIONAL ARRAYS 46 3.1 Wide-band FMCW waveform generator architecture 46 3.2 Overall system architecture 48 3.3 Antenna and RF transceiver module 53 3.4 Wide-band FMCW waveform generator 55 3.5 FPGA-based digital hardware design 63 3.6 System integration and software design 71 3.7 Testing and measurement 75 3.7.1 Chirp waveform measurement 75 3.7.2 Range profile measurement 77 3.7.3 2-D imaging test 79 4 METHODS OF IMAGE QUALITY ENHANCEMENT 84 4.1 Signal model 84 4.2 Digital pre-distortion of chirp signal 86 4.2.1 Proposed DPD hardware system 86 4.2.2 Proposed DPD algorithm 88 4.2.3 Measurement results 90 4.3 Robust calibration method for signal distortion 97 4.3.1 Signal model 98 4.3.2 Problem formulation 99 4.3.3 Measurement results 105 5 THREE-DIMENSIONAL IMAGING USING 1-D ARRAY SYSTEM AND ISAR TECHNIQUE 110 5.1 Formulation for 1D-MIMO-ISAR RSA 111 5.2 Algorithm implementation 114 5.3 Simulation results 120 5.4 Experimental results 122 6 CONCLUSIONS AND FUTURE WORK 127 6.1 Conclusions 127 6.2 Future work 129 6.2.1 Effects of antenna polarization in the Ku-band 129 6.2.2 Forward-looking near-field ISAR configuration 130 6.2.3 Estimation of the movement errors in ISAR configuration 131 Abstract (In Korean) 145 Acknowlegement 148박