Micro Fourier Transform Profilometry (μ\muFTP): 3D shape measurement at 10,000 frames per second

Recent advances in imaging sensors and digital light projection technology have facilitated a rapid progress in 3D optical sensing, enabling 3D surfaces of complex-shaped objects to be captured with improved resolution and accuracy. However, due to the large number of projection patterns required for phase recovery and disambiguation, the maximum fame rates of current 3D shape measurement techniques are still limited to the range of hundreds of frames per second (fps). Here, we demonstrate a new 3D dynamic imaging technique, Micro Fourier Transform Profilometry ($\mu$FTP), which can capture 3D surfaces of transient events at up to 10,000 fps based on our newly developed high-speed fringe projection system. Compared with existing techniques, $\mu$FTP has the prominent advantage of recovering an accurate, unambiguous, and dense 3D point cloud with only two projected patterns. Furthermore, the phase information is encoded within a single high-frequency fringe image, thereby allowing motion-artifact-free reconstruction of transient events with temporal resolution of 50 microseconds. To show $\mu$FTP's broad utility, we use it to reconstruct 3D videos of 4 transient scenes: vibrating cantilevers, rotating fan blades, bullet fired from a toy gun, and balloon's explosion triggered by a flying dart, which were previously difficult or even unable to be captured with conventional approaches.Comment: This manuscript was originally submitted on 30th January 1

Temporal phase unwrapping using deep learning

The multi-frequency temporal phase unwrapping (MF-TPU) method, as a classical phase unwrapping algorithm for fringe projection profilometry (FPP), is capable of eliminating the phase ambiguities even in the presence of surface discontinuities or spatially isolated objects. For the simplest and most efficient case, two sets of 3-step phase-shifting fringe patterns are used: the high-frequency one is for 3D measurement and the unit-frequency one is for unwrapping the phase obtained from the high-frequency pattern set. The final measurement precision or sensitivity is determined by the number of fringes used within the high-frequency pattern, under the precondition that the phase can be successfully unwrapped without triggering the fringe order error. Consequently, in order to guarantee a reasonable unwrapping success rate, the fringe number (or period number) of the high-frequency fringe patterns is generally restricted to about 16, resulting in limited measurement accuracy. On the other hand, using additional intermediate sets of fringe patterns can unwrap the phase with higher frequency, but at the expense of a prolonged pattern sequence. Inspired by recent successes of deep learning techniques for computer vision and computational imaging, in this work, we report that the deep neural networks can learn to perform TPU after appropriate training, as called deep-learning based temporal phase unwrapping (DL-TPU), which can substantially improve the unwrapping reliability compared with MF-TPU even in the presence of different types of error sources, e.g., intensity noise, low fringe modulation, and projector nonlinearity. We further experimentally demonstrate for the first time, to our knowledge, that the high-frequency phase obtained from 64-period 3-step phase-shifting fringe patterns can be directly and reliably unwrapped from one unit-frequency phase using DL-TPU

Single-shot compressed ultrafast photography: a review

Compressed ultrafast photography (CUP) is a burgeoning single-shot computational imaging technique that provides an imaging speed as high as 10 trillion frames per second and a sequence depth of up to a few hundred frames. This technique synergizes compressed sensing and the streak camera technique to capture nonrepeatable ultrafast transient events with a single shot. With recent unprecedented technical developments and extensions of this methodology, it has been widely used in ultrafast optical imaging and metrology, ultrafast electron diffraction and microscopy, and information security protection. We review the basic principles of CUP, its recent advances in data acquisition and image reconstruction, its fusions with other modalities, and its unique applications in multiple research fields

Radar and RGB-depth sensors for fall detection: a review

This paper reviews recent works in the literature on the use of systems based on radar and RGB-Depth (RGB-D) sensors for fall detection, and discusses outstanding research challenges and trends related to this research field. Systems to detect reliably fall events and promptly alert carers and first responders have gained significant interest in the past few years in order to address the societal issue of an increasing number of elderly people living alone, with the associated risk of them falling and the consequences in terms of health treatments, reduced well-being, and costs. The interest in radar and RGB-D sensors is related to their capability to enable contactless and non-intrusive monitoring, which is an advantage for practical deployment and users’ acceptance and compliance, compared with other sensor technologies, such as video-cameras, or wearables. Furthermore, the possibility of combining and fusing information from The heterogeneous types of sensors is expected to improve the overall performance of practical fall detection systems. Researchers from different fields can benefit from multidisciplinary knowledge and awareness of the latest developments in radar and RGB-D sensors that this paper is discussing

On the use of deep learning for phase recovery

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and outlook on how to better use DL to improve the reliability and efficiency in PR. Furthermore, we present a live-updating resource (https://github.com/kqwang/phase-recovery) for readers to learn more about PR.Comment: 82 pages, 32 figure

3D Micromachining of Optical Devices on Transparent Material by Ultrafast Laser

Ultrafast lasers, also referred to as ultrashort pulse lasers, have played an important role in the development of next generation manufacturing technologies in recent years. Their broad range of applications has been investigated in the field of microstructure processing for the biomedical, optical, and many other laboratory and industrial fields. Ultrafast laser machining has numerous unique advantages, including high precision, a small heat affected area, high peak intensity, 3D direct-writing, and other flexible capabilities When integrated with optical delivery, motion devices and control systems, one-step fabrication of assemble-free micro-devices can be realized. In particular, ultrafast lasers enable the creation of various three-dimensional, laser-induced modifications using an extremely high peak intensity over a short time frame, producing precise ablation of material and a small heat affected area in transparent materials. In contrast, lasers with longer pulse durations are based on a thermal effect, which results in significant melting in the heat affected area. In general, ultrafast laser micromachining can be used either to subtract material from or to change the material properties of both absorptive and transparent substances. Recently, integrated micro-devices including optical fiber sensors, microfluidic devices, and lab-on-chips (LOC) have gained worldwide recognition because of their unique characteristics. These micro-devices have been widely used for a broad range of applications, from fundamental research to industry. The development of integrated glass micro-devices introduced new possibilities for biomedical, environmental, civil and other industries and research areas. Of these devices, optical fiber sensors are recognized for their small size, accuracy, resistance to corrosion, fast response and high integration. They have demonstrated their excellent performance in sensing temperature, strain, refractive index and many other physical quantities. In addition to the all-in-fiber device, the LOC is another attractive candidate for use in micro-electro-mechanical systems (MEMS) because it includes several laboratory functions on a single integrated circuit. LOCs provide such advantages as low fluid volume consumption, improved analysis and response times due to short diffusion distances, and better process control, all of which are specific to their application. Combining ultrafast laser micromachining techniques with integrated micro-devices has resulted in research on a variety of fabrication methods targeted for particular purposes. In this dissertation, the direct creation of three-dimensional (3D) structures using an ultra-fast laser was investigated for use in optical devices. This research was motivated by the desire to understand more fully the relationship among laser parameters, material properties and 3D optical structures. Various all-in-fiber sensors in conjunction with femtosecond laser ablation and irradiation were investigated based on magnetic field, temperature and strain application. An incoherent optical carrier based microwave interferometry technique was used for in-situ weak reflector fabrication and a picosecond laser micromachining technique was introduced for developing LOCs with unlimited utilization potential

Acquisition of 3D shapes of moving objects using fringe projection profilometry

Three-dimensional (3D) shape measurement for object surface reconstruction has potential applications in many areas, such as security, manufacturing and entertainment. As an effective non-contact technique for 3D shape measurements, fringe projection profilometry (FPP) has attracted significant research interests because of its high measurement speed, high measurement accuracy and ease to implement. Conventional FPP analysis approaches are applicable to the calculation of phase differences for static objects. However, 3D shape measurement for dynamic objects remains a challenging task, although they are highly demanded in many applications. The study of this thesis work aims to enhance the measurement accuracy of the FPP techniques for the 3D shape of objects subject to movement in the 3D space. The 3D movement of objects changes not only the position of the object but also the height information with respect to the measurement system, resulting in motion-induced errors with the use of existing FPP technology. The thesis presents the work conducted for solutions of this challenging problem

Accelerating Quantitative Susceptibility Mapping using Compressed Sensing and Deep Neural Network

Quantitative susceptibility mapping (QSM) is an MRI phase-based post-processing method that quantifies tissue magnetic susceptibility distributions. However, QSM acquisitions are relatively slow, even with parallel imaging. Incoherent undersampling and compressed sensing reconstruction techniques have been used to accelerate traditional magnitude-based MRI acquisitions; however, most do not recover the full phase signal due to its non-convex nature. In this study, a learning-based Deep Complex Residual Network (DCRNet) is proposed to recover both the magnitude and phase images from incoherently undersampled data, enabling high acceleration of QSM acquisition. Magnitude, phase, and QSM results from DCRNet were compared with two iterative and one deep learning methods on retrospectively undersampled acquisitions from six healthy volunteers, one intracranial hemorrhage and one multiple sclerosis patients, as well as one prospectively undersampled healthy subject using a 7T scanner. Peak signal to noise ratio (PSNR), structural similarity (SSIM) and region-of-interest susceptibility measurements are reported for numerical comparisons. The proposed DCRNet method substantially reduced artifacts and blurring compared to the other methods and resulted in the highest PSNR and SSIM on the magnitude, phase, local field, and susceptibility maps. It led to 4.0% to 8.8% accuracy improvements in deep grey matter susceptibility than some existing methods, when the acquisition was accelerated four times. The proposed DCRNet also dramatically shortened the reconstruction time by nearly 10 thousand times for each scan, from around 80 hours using conventional approaches to only 30 seconds.Comment: 10 figure

Holographic MIMO Communications: Theoretical Foundations, Enabling Technologies, and Future Directions

Future wireless systems are envisioned to create an endogenously holography-capable, intelligent, and programmable radio propagation environment, that will offer unprecedented capabilities for high spectral and energy efficiency, low latency, and massive connectivity. A potential and promising technology for supporting the expected extreme requirements of the sixth-generation (6G) communication systems is the concept of the holographic multiple-input multiple-output (HMIMO), which will actualize holographic radios with reasonable power consumption and fabrication cost. The HMIMO is facilitated by ultra-thin, extremely large, and nearly continuous surfaces that incorporate reconfigurable and sub-wavelength-spaced antennas and/or metamaterials. Such surfaces comprising dense electromagnetic (EM) excited elements are capable of recording and manipulating impinging fields with utmost flexibility and precision, as well as with reduced cost and power consumption, thereby shaping arbitrary-intended EM waves with high energy efficiency. The powerful EM processing capability of HMIMO opens up the possibility of wireless communications of holographic imaging level, paving the way for signal processing techniques realized in the EM-domain, possibly in conjunction with their digital-domain counterparts. However, in spite of the significant potential, the studies on HMIMO communications are still at an initial stage, its fundamental limits remain to be unveiled, and a certain number of critical technical challenges need to be addressed. In this survey, we present a comprehensive overview of the latest advances in the HMIMO communications paradigm, with a special focus on their physical aspects, their theoretical foundations, as well as the enabling technologies for HMIMO systems. We also compare the HMIMO with existing multi-antenna technologies, especially the massive MIMO, present various...Comment: double column, 58 page