Depth Fields: Extending Light Field Techniques to Time-of-Flight Imaging

A variety of techniques such as light field, structured illumination, and time-of-flight (TOF) are commonly used for depth acquisition in consumer imaging, robotics and many other applications. Unfortunately, each technique suffers from its individual limitations preventing robust depth sensing. In this paper, we explore the strengths and weaknesses of combining light field and time-of-flight imaging, particularly the feasibility of an on-chip implementation as a single hybrid depth sensor. We refer to this combination as depth field imaging. Depth fields combine light field advantages such as synthetic aperture refocusing with TOF imaging advantages such as high depth resolution and coded signal processing to resolve multipath interference. We show applications including synthesizing virtual apertures for TOF imaging, improved depth mapping through partial and scattering occluders, and single frequency TOF phase unwrapping. Utilizing space, angle, and temporal coding, depth fields can improve depth sensing in the wild and generate new insights into the dimensions of light's plenoptic function.Comment: 9 pages, 8 figures, Accepted to 3DV 201

Micro Fourier Transform Profilometry (μ\muFTP): 3D shape measurement at 10,000 frames per second

Recent advances in imaging sensors and digital light projection technology have facilitated a rapid progress in 3D optical sensing, enabling 3D surfaces of complex-shaped objects to be captured with improved resolution and accuracy. However, due to the large number of projection patterns required for phase recovery and disambiguation, the maximum fame rates of current 3D shape measurement techniques are still limited to the range of hundreds of frames per second (fps). Here, we demonstrate a new 3D dynamic imaging technique, Micro Fourier Transform Profilometry ($\mu$FTP), which can capture 3D surfaces of transient events at up to 10,000 fps based on our newly developed high-speed fringe projection system. Compared with existing techniques, $\mu$FTP has the prominent advantage of recovering an accurate, unambiguous, and dense 3D point cloud with only two projected patterns. Furthermore, the phase information is encoded within a single high-frequency fringe image, thereby allowing motion-artifact-free reconstruction of transient events with temporal resolution of 50 microseconds. To show $\mu$FTP's broad utility, we use it to reconstruct 3D videos of 4 transient scenes: vibrating cantilevers, rotating fan blades, bullet fired from a toy gun, and balloon's explosion triggered by a flying dart, which were previously difficult or even unable to be captured with conventional approaches.Comment: This manuscript was originally submitted on 30th January 1

FPGA-Based Portable Ultrasound Scanning System with Automatic Kidney Detection

Bedsides diagnosis using portable ultrasound scanning (PUS) offering comfortable diagnosis with various clinical advantages, in general, ultrasound scanners suffer from a poor signal-to-noise ratio, and physicians who operate the device at point-of-care may not be adequately trained to perform high level diagnosis. Such scenarios can be eradicated by incorporating ambient intelligence in PUS. In this paper, we propose an architecture for a PUS system, whose abilities include automated kidney detection in real time. Automated kidney detection is performed by training the Viola–Jones algorithm with a good set of kidney data consisting of diversified shapes and sizes. It is observed that the kidney detection algorithm delivers very good performance in terms of detection accuracy. The proposed PUS with kidney detection algorithm is implemented on a single Xilinx Kintex-7 FPGA, integrated with a Raspberry Pi ARM processor running at 900 MHz

Front-end receiver for miniaturised ultrasound imaging

Point of care ultrasonography has been the focus of extensive research over the past few decades. Miniaturised, wireless systems have been envisaged for new application areas, such as capsule endoscopy, implantable ultrasound and wearable ultrasound. The hardware constraints of such small-scale systems are severe, and tradeoffs between power consumption, size, data bandwidth and cost must be carefully balanced. To address these challenges, two synthetic aperture receiver architectures are proposed and compared. The architectures target highly miniaturised, low cost, B-mode ultrasound imaging systems. The first architecture utilises quadrature (I/Q) sampling to minimise the signal bandwidth and computational load. Synthetic aperture beamforming is carried out using a single-channel, pipelined protocol in order to minimise system complexity and power consumption. A digital beamformer dynamically apodises and focuses the data by interpolating and applying complex phase rotations to the I/Q samples. The beamformer is implemented on a Spartan-6 FPGA and consumes 296mW for a frame rate of 7Hz. The second architecture employs compressive sensing within the finite rate of innovation (FRI) framework to further reduce the data bandwidth. Signals are sampled below the Nyquist frequency, and then transmitted to a digital back-end processor, which reconstructs I/Q components non-linearly, and then carries out synthetic aperture beamforming. Both architectures were tested in hardware using a single-channel analogue front-end (AFE) that was designed and fabricated in AMS 0.35μm CMOS. The AFE demodulates RF ultrasound signals sequentially into I/Q components, and comprises a low-noise preamplifier, mixer, programmable gain amplifier (PGA) and lowpass filter. A variable gain low noise preamplifier topology is used to enable quasi-exponential time-gain control (TGC). The PGA enables digital selection of three gain values (15dB, 22dB and 25.5dB). The bandwidth of the lowpass filter is also selectable between 1.85MHz, 510kHz and 195kHz to allow for testing of both architectural frameworks. The entire AFE consumes 7.8 mW and occupies an area of 1.5×1.5 mm. In addition to the AFE, this thesis also presents the design of a pseudodifferential, log-domain multiplier-filter or “multer” which demodulates low-RF signals in the current-domain. This circuit targets high impedance transducers such as capacitive micromachined ultrasound transducers (CMUTs) and offers a 20dB improvement in dynamic range over the voltage-mode AFE. The bandwidth is also electronically tunable. The circuit was implemented in 0.35μm BiCMOS and was simulated in Cadence; however, no fabrication results were obtained for this circuit. B-mode images were obtained for both architectures. The quadrature SAB method yields a higher image SNR and 9% lower root mean squared error with respect to the RF-beamformed reference image than the compressive SAB method. Thus, while both architectures achieve a significant reduction in sampling rate, system complexity and area, the quadrature SAB method achieves better image quality. Future work may involve the addition of multiple receiver channels and the development of an integrated system-on-chip.Open Acces

Remote Sensing Data Compression

A huge amount of data is acquired nowadays by different remote sensing systems installed on satellites, aircrafts, and UAV. The acquired data then have to be transferred to image processing centres, stored and/or delivered to customers. In restricted scenarios, data compression is strongly desired or necessary. A wide diversity of coding methods can be used, depending on the requirements and their priority. In addition, the types and properties of images differ a lot, thus, practical implementation aspects have to be taken into account. The Special Issue paper collection taken as basis of this book touches on all of the aforementioned items to some degree, giving the reader an opportunity to learn about recent developments and research directions in the field of image compression. In particular, lossless and near-lossless compression of multi- and hyperspectral images still remains current, since such images constitute data arrays that are of extremely large size with rich information that can be retrieved from them for various applications. Another important aspect is the impact of lossless compression on image classification and segmentation, where a reasonable compromise between the characteristics of compression and the final tasks of data processing has to be achieved. The problems of data transition from UAV-based acquisition platforms, as well as the use of FPGA and neural networks, have become very important. Finally, attempts to apply compressive sensing approaches in remote sensing image processing with positive outcomes are observed. We hope that readers will find our book useful and interestin

Towards Large-scale Single-shot Millimeter-wave Imaging for Low-cost Security Inspection

Millimeter-wave (MMW) imaging is emerging as a promising technique for safe security inspection. It achieves a delicate balance between imaging resolution, penetrability and human safety, resulting in higher resolution compared to low-frequency microwave, stronger penetrability compared to visible light, and stronger safety compared to X ray. Despite of recent advance in the last decades, the high cost of requisite large-scale antenna array hinders widespread adoption of MMW imaging in practice. To tackle this challenge, we report a large-scale single-shot MMW imaging framework using sparse antenna array, achieving low-cost but high-fidelity security inspection under an interpretable learning scheme. We first collected extensive full-sampled MMW echoes to study the statistical ranking of each element in the large-scale array. These elements are then sampled based on the ranking, building the experimentally optimal sparse sampling strategy that reduces the cost of antenna array by up to one order of magnitude. Additionally, we derived an untrained interpretable learning scheme, which realizes robust and accurate image reconstruction from sparsely sampled echoes. Last, we developed a neural network for automatic object detection, and experimentally demonstrated successful detection of concealed centimeter-sized targets using 10% sparse array, whereas all the other contemporary approaches failed at the same sample sampling ratio. The performance of the reported technique presents higher than 50% superiority over the existing MMW imaging schemes on various metrics including precision, recall, and mAP50. With such strong detection ability and order-of-magnitude cost reduction, we anticipate that this technique provides a practical way for large-scale single-shot MMW imaging, and could advocate its further practical applications

Architectures and Algorithms for the Signal Processing of Advanced MIMO Radar Systems

This thesis focuses on the research, development and implementation of novel concepts, architectures, demonstrator systems and algorithms for the signal processing of advanced Multiple Input Multiple Output (MIMO) radar systems. The key concept is to address compact system, which have high resolutions and are able to perform a fast radar signal processing, three-dimensional (3D), and four-dimensional (4D) beamforming for radar image generation and target estimation. The idea is to obtain a complete sensing of range, Azimuth and elevation (additionally Doppler as the fourth dimension) from the targets in the radar captures. The radar technology investigated, aims at addressing sev- eral civil and military applications, such as surveillance and detection of targets, both air and ground based, and situational awareness, both in cars and in flying platforms, from helicopters, to Unmanned Aerial Vehicles (UAV) and air-taxis. Several major topics have been targeted. The development of complete systems and innovative FPGA, ARM and software based digital architectures for 3D imaging MIMO radars, which operate in both Time Division Multiplexing (TDM) and Frequency Divi- sion Multiplexing (FDM) modes, with Frequency Modulated Continuous Wave (FMCW) and Orthogonal Frequency Division Multiplexing (OFDM) signals, respectively. The de- velopment of real-time radar signal processing, beamforming and Direction-Of-Arrival (DOA) algorithms for target detection, with particular focus on FFT based, hardware implementable techniques. The study and implementation of advanced system concepts, parametrisation and simulation of next generation real-time digital radars (e.g. OFDM based). The design and development of novel constant envelope orthogonal waveforms for real-time 3D OFDM MIMO radar systems. The MIMO architectures presented in this thesis are a collection of system concepts, de- sign and simulations, as well as complete radar demonstrators systems, with indoor and outdoor measurements. Several of the results shown, come in the form of radar images which have been captured in field-test, in different scenarios, which aid in showing the proper functionality of the systems. The research activities for this thesis, have been carried out on the premises of Air- bus, based in Munich (Germany), as part of a Ph.D. candidate joint program between Airbus and the Polytechnic Department of Engineering and Architecture (Dipartimento Politecnico di Ingegneria e Architettura), of the University of Udine, based in Udine (Italy).Questa tesi si concentra sulla ricerca, lo sviluppo e l\u2019implementazione di nuovi concetti, architetture, sistemi dimostrativi e algoritmi per l\u2019elaborazione dei segnali in sistemi radar avanzati, basati su tecnologia Multiple Input Multiple Output (MIMO). Il con- cetto chiave `e quello di ottenere sistemi compatti, dalle elevate risoluzioni e in grado di eseguire un\u2019elaborazione del segnale radar veloce, un beam-forming tri-dimensionale (3D) e quadri-dimensionale (4D) per la generazione di immagini radar e la stima delle informazioni dei bersagli, detti target. L\u2019idea `e di ottenere una stima completa, che includa la distanza, l\u2019Azimuth e l\u2019elevazione (addizionalmente Doppler come quarta di- mensione) dai target nelle acquisizioni radar. La tecnologia radar indagata ha lo scopo di affrontare diverse applicazioni civili e militari, come la sorveglianza e la rilevazione di targets, sia a livello aereo che a terra, e la consapevolezza situazionale, sia nelle auto che nelle piattaforme di volo, dagli elicotteri, ai Unmanned Aerial Vehicels (UAV) e taxi volanti (air-taxis). Le tematiche affrontante sono molte. Lo sviluppo di sistemi completi e di architetture digitali innovative, basate su tecnologia FPGA, ARM e software, per radar 3D MIMO, che operano in modalit`a Multiplexing Time Division Multiplexing (TDM) e Multiplexing Frequency Diversion (FDM), con segnali di tipo FMCW (Frequency Modulated Contin- uous Wave) e Orthogonal Frequency Division Multiplexing (OFDM), rispettivamente. Lo sviluppo di tecniche di elaborazione del segnale radar in tempo reale, algoritmi di beam-forming e di stima della direzione di arrivo, Direction-Of-Arrival (DOA), dei seg- nali radar, per il rilevamento dei target, con particolare attenzione a processi basati su trasformate di Fourier (FFT). Lo studio e l\u2019implementazione di concetti di sistema avan- zati, parametrizzazione e simulazione di radar digitali di prossima generazione, capaci di operare in tempo reale (ad esempio basati su architetture OFDM). Progettazione e sviluppo di nuove forme d\u2019onda ortogonali ad inviluppo costante per sistemi radar 3D di tipo OFDM MIMO, operanti in tempo reale. Le attivit`a di ricerca di questa tesi sono state svolte presso la compagnia Airbus, con sede a Monaco di Baviera (Germania), nell\u2019ambito di un programma di dottorato, svoltosi in maniera congiunta tra Airbus ed il Dipartimento Politecnico di Ingegneria e Architettura dell\u2019Universit`a di Udine, con sede a Udine

Ultrasound Imaging

In this book, we present a dozen state of the art developments for ultrasound imaging, for example, hardware implementation, transducer, beamforming, signal processing, measurement of elasticity and diagnosis. The editors would like to thank all the chapter authors, who focused on the publication of this book

Hydrogen Epoch of Reionization Array (HERA)

The Hydrogen Epoch of Reionization Array (HERA) is a staged experiment to measure 21 cm emission from the primordial intergalactic medium (IGM) throughout cosmic reionization ($z=6-12$), and to explore earlier epochs of our Cosmic Dawn ($z\sim30$). During these epochs, early stars and black holes heated and ionized the IGM, introducing fluctuations in 21 cm emission. HERA is designed to characterize the evolution of the 21 cm power spectrum to constrain the timing and morphology of reionization, the properties of the first galaxies, the evolution of large-scale structure, and the early sources of heating. The full HERA instrument will be a 350-element interferometer in South Africa consisting of 14-m parabolic dishes observing from 50 to 250 MHz. Currently, 19 dishes have been deployed on site and the next 18 are under construction. HERA has been designated as an SKA Precursor instrument. In this paper, we summarize HERA's scientific context and provide forecasts for its key science results. After reviewing the current state of the art in foreground mitigation, we use the delay-spectrum technique to motivate high-level performance requirements for the HERA instrument. Next, we present the HERA instrument design, along with the subsystem specifications that ensure that HERA meets its performance requirements. Finally, we summarize the schedule and status of the project. We conclude by suggesting that, given the realities of foreground contamination, current-generation 21 cm instruments are approaching their sensitivity limits. HERA is designed to bring both the sensitivity and the precision to deliver its primary science on the basis of proven foreground filtering techniques, while developing new subtraction techniques to unlock new capabilities. The result will be a major step toward realizing the widely recognized scientific potential of 21 cm cosmology.Comment: 26 pages, 24 figures, 2 table