Micro Fourier Transform Profilometry (μ\muFTP): 3D shape measurement at 10,000 frames per second

Recent advances in imaging sensors and digital light projection technology have facilitated a rapid progress in 3D optical sensing, enabling 3D surfaces of complex-shaped objects to be captured with improved resolution and accuracy. However, due to the large number of projection patterns required for phase recovery and disambiguation, the maximum fame rates of current 3D shape measurement techniques are still limited to the range of hundreds of frames per second (fps). Here, we demonstrate a new 3D dynamic imaging technique, Micro Fourier Transform Profilometry ($\mu$FTP), which can capture 3D surfaces of transient events at up to 10,000 fps based on our newly developed high-speed fringe projection system. Compared with existing techniques, $\mu$FTP has the prominent advantage of recovering an accurate, unambiguous, and dense 3D point cloud with only two projected patterns. Furthermore, the phase information is encoded within a single high-frequency fringe image, thereby allowing motion-artifact-free reconstruction of transient events with temporal resolution of 50 microseconds. To show $\mu$FTP's broad utility, we use it to reconstruct 3D videos of 4 transient scenes: vibrating cantilevers, rotating fan blades, bullet fired from a toy gun, and balloon's explosion triggered by a flying dart, which were previously difficult or even unable to be captured with conventional approaches.Comment: This manuscript was originally submitted on 30th January 1

Temporal phase unwrapping using deep learning

The multi-frequency temporal phase unwrapping (MF-TPU) method, as a classical phase unwrapping algorithm for fringe projection profilometry (FPP), is capable of eliminating the phase ambiguities even in the presence of surface discontinuities or spatially isolated objects. For the simplest and most efficient case, two sets of 3-step phase-shifting fringe patterns are used: the high-frequency one is for 3D measurement and the unit-frequency one is for unwrapping the phase obtained from the high-frequency pattern set. The final measurement precision or sensitivity is determined by the number of fringes used within the high-frequency pattern, under the precondition that the phase can be successfully unwrapped without triggering the fringe order error. Consequently, in order to guarantee a reasonable unwrapping success rate, the fringe number (or period number) of the high-frequency fringe patterns is generally restricted to about 16, resulting in limited measurement accuracy. On the other hand, using additional intermediate sets of fringe patterns can unwrap the phase with higher frequency, but at the expense of a prolonged pattern sequence. Inspired by recent successes of deep learning techniques for computer vision and computational imaging, in this work, we report that the deep neural networks can learn to perform TPU after appropriate training, as called deep-learning based temporal phase unwrapping (DL-TPU), which can substantially improve the unwrapping reliability compared with MF-TPU even in the presence of different types of error sources, e.g., intensity noise, low fringe modulation, and projector nonlinearity. We further experimentally demonstrate for the first time, to our knowledge, that the high-frequency phase obtained from 64-period 3-step phase-shifting fringe patterns can be directly and reliably unwrapped from one unit-frequency phase using DL-TPU

A three-dimensional imaging system for surface profilometry of moving objects

Non-contact optical imaging system design and the corresponding surface profilometry algorithm are critical components in various metrology applications, such as surface inspection of semiconductor components on the production line. For such challenging industrial applications, the most important considerations are often automation, precision and speed of the inspection. In this work, we propose a mathematical framework and a dynamic phase-shift algorithm (D-PSA) for a dense surface profilometry of moving objects. We also present a fringe pattern projection system with projector and camera arrays, with an aim to reduce the undesirable effects such as the uneven illumination and the perspective geometry effect on the reconstructed surface using a large field-of-view inspection system. This system is then applied to the inspection of the surface of moving printed circuit boards along a conveyor belt. Experimental results show that our approach can reconstruct the object surface effectively and efficiently. © 2013 IEEE.published_or_final_versio

3D Shape Measurement of Objects in Motion and Objects with Complex Surfaces

This thesis aims to address the issues caused by high reflective surface and object with motion in the three dimensional (3D) shape measurement based on phase shifting profilometry (PSP). Firstly, the influence of the reflectivity of the object surface on the fringe patterns is analysed. One of the essential factors related to phase precision is modulation index, which has a direct relationship with the surface reflectivity. A comparative study focusing on the modulation index of different materials is presented. The distribution of modulation index for different material samples is statistically analysed, which leads to the conclusion that the modulation index is determined by the diffuse reflectivity. Then the method based on optimized combination of multiple reflected image patterns is proposed to address the saturation issue and improve the accuracy for the reconstruction of object with high reflectivity.A set of phase shifted sinusoidal fringe patterns with different exposure time are projected to the object and then captured by camera. Then a set of masks are generated to select the data for the compositing. Maximalsignal-to-noise ratio combining model is employed to form the composite images pattern. The composite images are then used to phase mapping.Comparing to the method only using the highest intensity of pixels for compositing image, the signal noise ratio (SNR) of composite image is increased due to more efficient use of information carried by the images

An illumination-invariant phase-shifting algorithm for three-dimensional profilometry

Image Processing: Machine Vision Applications V, Burlingame, California, USA, 22 January, 2012Uneven illumination is a common problem in real optical systems for machine vision applications, and it contributes significant errors when using phase-shifting algorithms (PSA) to reconstruct the surface of a moving object. Here, we propose an illumination-reflectivity-focus (IRF) model to characterize this uneven illumination effect on phase-measuring profilometry. With this model, we separate the illumination factor effectively, and then formulate the phase reconstruction as an optimization problem. To simplify the optimization process, we calibrate the uneven illumination distribution beforehand, and then use the calibrated illumination information during surface profilometry. After calibration, the degrees of freedom are reduced. Accordingly, we develop a novel illumination-invariant phase-shifting algorithm (II-PSA) to reconstruct the surface of a moving object under an uneven illumination environment. Experimental results show that the proposed algorithm can improve the reconstruction quality both visually and numerically. Therefore, using this IRF model and the corresponding II-PSA, not only can we handle uneven illumination in a real optical system with a large field of view (FOV), but we also develop a robust and efficient method for reconstructing the surface of a moving object. © 2012 Copyright Society of Photo-Optical Instrumentation Engineers (SPIE).link_to_subscribed_fulltextpublished_or_final_versio

Acquisition of 3D shapes of moving objects using fringe projection profilometry

Three-dimensional (3D) shape measurement for object surface reconstruction has potential applications in many areas, such as security, manufacturing and entertainment. As an effective non-contact technique for 3D shape measurements, fringe projection profilometry (FPP) has attracted significant research interests because of its high measurement speed, high measurement accuracy and ease to implement. Conventional FPP analysis approaches are applicable to the calculation of phase differences for static objects. However, 3D shape measurement for dynamic objects remains a challenging task, although they are highly demanded in many applications. The study of this thesis work aims to enhance the measurement accuracy of the FPP techniques for the 3D shape of objects subject to movement in the 3D space. The 3D movement of objects changes not only the position of the object but also the height information with respect to the measurement system, resulting in motion-induced errors with the use of existing FPP technology. The thesis presents the work conducted for solutions of this challenging problem