Real-time 3-D Reconstruction by Means of Structured Light Illumination

Structured light illumination (SLI) is the process of projecting a series of light striped patterns such that, when viewed at an angle, a digital camera can reconstruct a 3-D model of a target object\u27s surface. But by relying on a series of time multiplexed patterns, SLI is not typically associated with video applications. For this purpose of acquiring 3-D video, a common SLI technique is to drive the projector/camera pair at very high frame rates such that any object\u27s motion is small over the pattern set. But at these high frame rates, the speed at which the incoming video can be processed becomes an issue. So much so that many video-based SLI systems record camera frames to memory and then apply off-line processing. In order to overcome this processing bottleneck and produce 3-D point clouds in real-time, we present a lookup-table (LUT) based solution that in our experiments, using a 640 by 480 video stream, can generate intermediate phase data at 1063.8 frames per second and full 3-D coordinate point clouds at 228.3 frames per second. These achievements are 25 and 10 times faster than previously reported studies. At the same time, a novel dual-frequency pattern is developed which combines a high-frequency sinusoid component with a unit-frequency sinusoid component, where the high-frequency component is used to generate robust phase information and the unit-frequency component is used to reduce phase unwrapping ambiguities. Finally, we developed a gamma model for SLI, which can correct the non-linear distortion caused by the optical devices. For three-step phase measuring profilometry (PMP), analysis of the root mean squared error of the corrected phase showed a 60х reduction in phase error when the gamma calibration is performed versus 33х reduction without calibration

Free-body Modeling of the Dynamics of a Fin-Stabilized Ballistics Missile in Nonspinnning Vertical Trajectories

Part 1 of an experimental and analytical investigation of the behavior in free flight of inertia-propelled bodies that are fin-stabilized is presented. The dynamic response in vertical trajectories starting from rest has been found for two sizes of a single configuration of the Basic Finner Research Missile with a variety of model parameters and initial conditions. A relatively simple formulation of the drag function for longitudinally accelerated motion in the laminar flow regime is found to give excellent agreement between predicted and experimental behavior. Time-position histories of test trajectories are presented in tabular form

Tri-dimensional optical inspection based on flexible image guide: first step toward 3D industrial endoscopy

International audience3D optical endoscopy is now a major challenge to allow the high resolution inspection of industrial equipments. The proposed instrument is based on a flexible image guide (70 000 fibres) and a Digital Micro mirror Device (DMD, 1024 x 768 "on-off" micro mirrors). The optical design is as follows: the light emitted by a 532 nm laser diode is dynamically structured by the DMD chip as a fringes pattern which is phase-shifted due to the active control of the DMD chip and projected onto an object on a circular field of 6 mm in diameter. Due to a telecentric and binocular arrangement that creates a stereoscopic angle, it is possible to get a depth of field of 2 mm along the optical axis without keystone distortions and few disturbances created by defocus and coma aberrations. Then, images are captured by a 1024 x 768 digital camera (not yet moved away by fibres) at 15 fps and directly used in the reconstruction algorithm to access the tri-dimensional shape of the unpainted object. The results are compared to incoherent white light results obtained with white painted mechanical objects. The lateral resolution is 31.3 µm and the RMS axial resolution is 10 µm for the laser-based design after speckle attenuation

Novel Approaches in Structured Light Illumination

Among the various approaches to 3-D imaging, structured light illumination (SLI) is widely spread. SLI employs a pair of digital projector and digital camera such that the correspondences can be found based upon the projecting and capturing of a group of designed light patterns. As an active sensing method, SLI is known for its robustness and high accuracy. In this dissertation, I study the phase shifting method (PSM), which is one of the most employed strategy in SLI. And, three novel approaches in PSM have been proposed in this dissertation. First, by regarding the design of patterns as placing points in an N-dimensional space, I take the phase measuring profilometry (PMP) as an example and propose the edge-pattern strategy which achieves maximum signal to noise ratio (SNR) for the projected patterns. Second, I develop a novel period information embedded pattern strategy for fast, reliable 3-D data acquisition and reconstruction. The proposed period coded phase shifting strategy removes the depth ambiguity associated with traditional phase shifting patterns without reducing phase accuracy or increasing the number of projected patterns. Thus, it can be employed for high accuracy realtime 3-D system. Then, I propose a hybrid approach for high quality 3-D reconstructions with only a small number of illumination patterns by maximizing the use of correspondence information from the phase, texture, and modulation data derived from multi-view, PMP-based, SLI images, without rigorously synchronizing the cameras and projectors and calibrating the device gammas. Experimental results demonstrate the advantages of the proposed novel strategies for 3-D SLI systems

Snapshot Three-Dimensional Surface Imaging With Multispectral Fringe Projection Profilometry

Fringe Projection Profilometry (FPP) is a popular method for non-contact optical surface measurements, including motion tracking. The technique derives 3D surface maps from phase maps estimated from the distortions of fringe patterns projected onto the surface of an object. Estimation of phase maps is commonly performed with spatial phase retrieval algorithms that use a series of complex data processing stages. Researchers must have advanced data analysis skills to process FPP data due to a lack of availability of simple research-oriented software tools. Chapter 2 describes a comprehensive FPP software tool called PhaseWareTM that allows novice to experienced users to perform pre-processing of fringe patterns, phase retrieval, phase unwrapping, and finally post-processing. The sequential process of acquiring fringe patterns from an object is necessary to sample the surface densely enough to accurately estimate surface profiles. Sequential fringe acquisition performs poorly if the object is in motion between fringe projections. To overcome this limitation, we developed a novel method of FPP called multispectral fringe projection profilometry (MFPP), where multiple fringe patterns are composited into a multispectral illumination pattern and a single multispectral camera is used to capture the frame. Chapter 3 introduces this new technique and shows how it can be used to perform 3D profilometry at video frame rates. Although the first attempt at MFPP significantly improved acquisition speed, it did not fully satisfy the condition for temporal phase retrieval, which requires at least three phase-shifted fringe patterns to characterize a surface. To overcome this limitation, Chapter 4 introduces an enhanced version of MFPP that utilized a specially designed multispectral illuminator to simultaneously project four p/2 phase-shifted fringe patterns onto an object. Combined with spectrally matched multispectral imaging, the refined MFPP method resulted in complete data for temporal phase retrieval using only a single camera exposure, thereby maintaining the high sampling speed for profilometry of moving objects. In conclusion, MFPP overcomes the limitations of sequential sampling imposed by FPP with temporal phase extraction without sacrificing data quality or accuracy of the reconstructed surface profiles. Since MFPP utilizes no moving parts and is based on MEMS technology, it is applicable to miniaturization for use in mobile devices and may be useful for space-constrained applications such as robotic surgery. Fringe Projection Profilometry (FPP) is a popular method for non-contact optical surface measurements such as motion tracking. The technique derives 3D surface maps from phase maps estimated from the distortions of fringe patterns projected onto the surface of the object. To estimate surface profiles accurately, sequential acquisition of fringe patterns is required; however, sequential fringe projection and acquisition perform poorly if the object is in motion during the projection. To overcome this limitation, we developed a novel method of FPP maned multispectral fringe projection profilometry (MFPP). The proposed method provides multispectral illumination patterns using a multispectral filter array (MFA) to generate multiple fringe patterns from a single illumination and capture the composite pattern using a single multispectral camera. Therefore, a single camera acquisition can provide multiple fringe patterns, and this directly increases the speed of imaging by a factor equal to the number of fringe patterns included in the composite pattern. Chapter 3 introduces this new technique and shows how it can be used to perform 3D profilometry at video frame rates. The first attempt at MFPP significantly improved acquisition speed by a factor of eight by providing eight different fringe patterns in four different directions, which permits the system to detect more morphological details. However, the phase retrieval algorithm used in this method was based on the spatial phase stepping process that had a few limitations, including high sensitive to the quality of the fringe patterns and being a global process, as it spreads the effect of the noisy pixels across the entire result. To overcome this limitation, Chapter 4 introduces an enhanced version of MFPP that utilized a specially designed multispectral illuminator to simultaneously project four p/2 phase-shifted fringe patterns onto an object. Combined with a spectrally matched multispectral camera, the refined MFPP method provided the needed data for the temporal phase retrieval algorithm using only a single camera exposure. Thus, it delivers high accuracy and pixel-wise measurement (thanks to the temporal phase stepping algorithms) while maintaining a high sampling rate for profilometry of moving objects. In conclusion, MFPP overcomes the limitations of sequential sampling imposed by FPP with temporal phase extraction without sacrificing data quality or accuracy of the reconstructed surface profiles. Since MFPP utilizes no moving parts and is based on MEMS technology, it is applicable to miniaturization for use in mobile devices and may be useful for space-constrained applications such as robotic surgery

Experimental Analysis of Pose Tracking Performance of Visual and Infrared Stereo Cameras under Orbital Light Conditions

La telecamera stereo D435 e il LIDAR a stato solido HPS-3D160, insieme alla telecamera ZED precedentemente installata, sono stati integrati nel laboratorio RACOON-Lab di TUM. La struttura hardware risultante consente una rapida aggiunta di futuri nuovi sensori alla piattaforma, mentre il nuovo formato di memorizzazione consente di archiviare tutti i risultati in un formato aperto ed estensibile per l'uso futuro. Il sistema completoè stato poi testato in condizioni di luce orbitale.ope

Electric Vehicle Battery Module Dismantling "Analysis and Evaluation of Robotic Dismantling Techniques for Irre- versible Fasteners, including Object Detection of Components."

This thesis examines a study of The Litium-Ion Battery (LIB) from a electric vehicle, and it’s recycling processes. A Battery Module (BM) from the LIB is shredded when considered an End-Of-Life product, and motivates for automated dismantling concepts to separate the components to save raw materials. From State-of-the-art (SoA) research projects and background theory, automatic module dis- mantling concepts have been evaluated for a Volkswagen E-Golf 2019 battery module. The presence of irreversible fasteners make the use off destructive dismantling techniques neces- sary. This study evaluates two different concepts to disconnect laser welds holding together the compressive plates made of steel. A hydraulic actuated concept is first investigated to separate the welded compressive plates within the casing. A FEM analysis with different configurations is performed to evaluate the most effective hydraulic solution when analysing the Von Mises stress. This solution is further compared with another automatic dismantling concept, namely milling. For the purpose of an automated milling concept, manipulators from ABB are assessed and the feasibility is verified based on results from manual milling operation. The proposed dismantling operation is made possible by developing a system architecture combining robotic control and computer vision. Open source software based on Robot Op- erating System (ROS) and MoveIt connect and control an ABB IRB4400 industrial robot whereas the computer vision setup involves a cutting edge 3D camera, Zivid, and object detection algorithm YOLOv5 best suited for this task. Adjustable acquisition settings in services from Zivid’s ROS driver are tested to accomplish the optimal capture configuration. Two datasets generated with RoboFlow were exported in the YOLOv5 PyTorch format. Custom object detection models with annotated components from the BM was trained and tested with image captures. All in all, this study demonstrates that the automatic dismantling of battery modules can be achieved even though they include irreversible fasteners. The proposed methods are verified on a specific battery module (Egolf 2019) but are flexible enough to be easily extended to a large variety of EV battery modules

Analysis of Gothic Architectural Details by Spatial Object Reconstruction Techniques

The paper focuses on the 3D data acquisition technologiesthat support capturing the geometry of medieval architecturalfragmented stones. High-resolution models of such fragmentsenable the analysis of profile shapes as well as the markinglines and curves left by the instruments of stonemasons, andtherefore, indirectly, identifying connections between severalmaster builders could become possible. Considering therequirements of historical analysis and the fact that the investigatedstones are under monument protection, the authorsdecided to use remote sensing technologies, such as structuredlight scanning, terrestrial laser scanning, depth camera andimage-based reconstruction.The paper evaluates the discussed technologies based on theaccuracy and geometric resolution of the obtained 3D models.Besides technical parameters, time and cost requirements alsohave been investigated. The paper gives an overview on theadvantages and shortcomings of the applied data acquisitiontechnologies and of the provided end-products