Cavlectometry: Towards Holistic Reconstruction of Large Mirror Objects

We introduce a method based on the deflectometry principle for the reconstruction of specular objects exhibiting significant size and geometric complexity. A key feature of our approach is the deployment of an Automatic Virtual Environment (CAVE) as pattern generator. To unfold the full power of this extraordinary experimental setup, an optical encoding scheme is developed which accounts for the distinctive topology of the CAVE. Furthermore, we devise an algorithm for detecting the object of interest in raw deflectometric images. The segmented foreground is used for single-view reconstruction, the background for estimation of the camera pose, necessary for calibrating the sensor system. Experiments suggest a significant gain of coverage in single measurements compared to previous methods. To facilitate research on specular surface reconstruction, we will make our data set publicly available

Shape reconstruction from gradient data

We present a novel method for reconstructing the shape of an object from measured gradient data. A certain class of optical sensors does not measure the shape of an object, but its local slope. These sensors display several advantages, including high information efficiency, sensitivity, and robustness. For many applications, however, it is necessary to acquire the shape, which must be calculated from the slopes by numerical integration. Existing integration techniques show drawbacks that render them unusable in many cases. Our method is based on approximation employing radial basis functions. It can be applied to irregularly sampled, noisy, and incomplete data, and it reconstructs surfaces both locally and globally with high accuracy.Comment: 16 pages, 5 figures, zip-file, submitted to Applied Optic

3D inspection methods for specular or partially specular surfaces

Deflectometric techniques are a powerful tool for the automated quality control of specular or shiny surfaces. These techniques are based on using a camera to observe a reference pattern reflected on the surface under inspection, exploiting the dependence of specular reflections on surface normals to recover shape information from the acquired images. Although deflectometry is already used in industrial environments such as the quality control of lenses or car bodies, there are still some open problems. On the one hand, using quantitative deflectometry, the normal vector field and the 3D shape of a surface can be obtained, but these techniques do not yet take full advantage of their local sensitivity because the achieved global accuracies are affected by calibration errors. On the other hand, qualitative deflectometry is used to detect surface imperfections without absolute measurements, exploiting the local sensitivity of deflectometric recordings with reduced calibration requirements. However, this qualitative approach requires further processing that can involve a considerable engineering effort, particularly for aesthetic defects which are inherently subjective. The first part of this thesis aims to contribute to a better understanding of how deflectometric setups and their calibration errors affect quantitative measurements. Different error sources are considered including the camera calibration uncertainty and several non-ideal characteristics of LCD screens used to generate the light patterns. Experiments performed using real measurements and simulations show that the non-planarity of the LCD screen and the camera calibration are the dominant sources of error. The second part of the thesis investigates the use of deep learning to identify geometrical imperfections and texture defects based on deflectometric data. Two different approaches are explored to extract and combine photometric and geometric information using convolutional neural network architectures: one for automated classification of defective samples, and another one for automated segmentation of defective regions in a sample. The experimental results in a real industrial case study indicate that both architectures are able to learn relevant features from deflectometric data, enabling the classification and segmentation of defects based on a dataset of user-provided examples.Teknika deflektometrikoak tresna baliotsuak dira gainazal espekular edo distiratsuen kalitate kontrol automatikoa gauzatzeko. Teknika hauetan, kamera bat erabiltzen da ikuskatu beharreko gainazalean islatutako erreferentziazko patroi bat behatzeko, eta isladapen espekularrek gainazalen bektore normalengan duten menpekotasuna ustiatzen dute irudietatik informazio geometrikoa berreskuratzeko. Zenbait industria-aplikaziotan deflektometria jada erabiltzen bada ere –adibidez, betaurrekoen edo autoen karrozerien kalitate kontrolean-, oraindik badaude hobetu beharreko hainbat esparru. Batetik, deflektometria kuantitatiboak aukera ematen du gainazal baten bektore-eremu normala eta 3D forma lortzeko, baina gaur egun teknika hauek ez dute beren sentsibilitate lokal guztia aprobetxatzen kalibrazio-akatsek zehaztasun globalean duten eraginagatik. Bestetik, deflektometria kualitatiboa neurketa absoluturik egin gabe gainazal akatsak antzemateko erabili daiteke, kalibrazio-eskakizun murriztuekin sentsibilitate lokala ustiatuz. Hala ere, teknika horiek algoritmoen garapenean esfortzu handia ekar dezakeen prozesamendu bat eskatzen dute, bereziki bere baitan subjektiboak diren akats estetikoetarako. Hala ere, teknika horiek algoritmoen garapenean esfortzu handia ekar dezakeen prozesamendu bat eskatzen dute, bereziki bere baitan subjektiboak diren akats estetikoetarako. Tesi honen lehen zatiaren helburua adkizizio sistema osatzen duten gailuek eta horien kalibrazioek neurketa kuantitatiboei nola eragiten dieten hobeto ulertzen laguntzea da. Hainbat errore-iturri hartzen dira kontuan, besteak beste kameraren kalibrazioaren ziurgabetasuna, eta argi-patroiak sortzeko erabilitako LCD pantailen zenbait ezaugarri ez-ideal. Neurketa errealetan eta simulazioetan egindako esperimentuek erakusten dute LCD pantailaren deformazioak eta kameraren kalibrazioak eragindako erroreak direla neurketen akats eta ziurgabetasun iturri nagusiak. Tesiaren bigarren zatian, datu deflektometrikoetatik abiatuz, inperfekzio geometrikoak eta testura-akatsak identifikatzeko ikaskuntza sakoneko metodoen erabilera ikertzen da. Helburu honekin, irudietatik informazio fotometrikoa eta geometrikoa atera eta konbinatzen duten sare neuronal konboluzionaletan oinarritutako bi arkitektura proposatzen dira: bata, lagin akastunak automatikoki sailkatzeko; eta, bestea, laginetako eremu akastunak automatikoki segmentatzeko. Automobilgintza industriako kasu praktiko baten lortutako emaitzek erakusten dute erabilitako arkitekturek datu deflektometrikoetatik ezaugarri esanguratsuak ikas ditzaketela, erabiltzaileak emandako adibide multzo batean oinarrituta gainazal akatsak sailkatu eta segmentatzea ahalbidetuz.Las técnicas deflectométricas son una herramienta valiosa para automatizar el control de calidad de superficies especulares o reflectantes. Estas técnicas se basan en el uso de una cámara para observar un patrón de referencia reflejado en la superficie bajo inspección, explotando la dependencia de los reflejos especulares en la normal de la superficie para recuperar información geométrica a partir de las imágenes adquiridas. Aunque la deflectometría ya se usa en algunas aplicaciones industriales, tales como el control de calidad de lentes o carrocerías de coches, todavía hay algunos problemas abiertos. Por un lado, la deflectometría cuantitativa permite obtener el campo vectorial normal y la forma 3D de una superficie, pero a día de hoy no es capaz de aprovechar al máximo su sensibilidad local ya que la precisión global se ve afectada por errores de calibración. Por otro lado, la deflectometría cualitativa se utiliza para detectar imperfecciones de la superficie sin mediciones absolutas, explotando la sensibilidad local de la deflectometría con requisitos de calibración reducidos. Sin embargo, estos métodos requieren un procesamiento adicional que puede implicar un esfuerzo considerable en el desarrollo de algoritmos, particularmente para defectos estéticos que son inherentemente subjetivos. La primera parte de esta tesis tiene como objetivo contribuir a una mejor comprensión de cómo el sistema de adquisición y su calibración afectan a las mediciones cuantitativas. Se consideran dife-rentes fuentes de error, incluida la incertidumbre de calibración de la cámara y varias características no ideales de las pantallas LCD utilizadas para generar los patrones de luz. Los experimentos realizados con mediciones reales y simulaciones indican que los errores inducidos por la deformación de la pantalla LCD y la calibración de la cámara son las principales fuentes de error e incertidumbre. La segunda parte de la tesis investiga el uso del aprendizaje profundo para identificar imperfecciones geométricas y defectos de textura a partir de datos deflectométricos. Se adoptan dos enfoques diferentes para extraer y combinar información fotométrica y geométrica utilizando sendas arquitecturas basadas en redes neuronales convolucionales: una para la clasificación automatizada de muestras defectuosas y otra para la segmentación automatizada de regiones defectuosas en una muestra. Los resultados experimentales en un caso de estudio industrial real indican que ambas arquitecturas pueden aprender características relevantes de los datos deflectométricos, permitiendo la clasificación y segmentación de defectos en base a un conjunto de datos de ejemplos proporcionados por el usuario

The effects of pattern screen surface deformation on deflectometric measurements - A simulation study

Phase-measuring deflectometry (PMD) is an optical inspection technique for full-field topography measurements of reflective sample surfaces. The measurement principle relies on the analysis of specific patterns, reflected at the sample surface. Evaluation algorithms often model the respective pattern screen as a planar light source. However, the 32\u27\u27 pattern screen in our inspection setup exhibits a central bulge of its surface of about 2–3 mm. This paper presents a simulation framework for PMD to evaluate the effects of a deformed screen surface. The idea is to simulate image data acquired with screen surface deformations and to examine the effects on the PMD evaluation results. The simulated setup consists of a 32\u27\u27 pattern screen with an adjustable central bulge height of 0–3 mm and two cameras with a field of view (FOV) of approximately 225 mm by 172 mm on the sample surface. A first experiment examines the reconstruction errors for a planar sample surface if the reconstruction algorithm uses perfect calibration data (i.e. the same parameters used for the simulated image acquisition). The reconstructed surfaces exhibit a tilt with a maximum height difference of 174 μm across the FOV. A second experiment repeats the reconstruction process of the same sample surface, using camera parameters determined in a simulated calibration process. The resulting surfaces possess irregular, wave-like errors with amplitudes of up to 9 μm in the FOV. The presented simulation results reveal the accuracy limits if a deformation model of the pattern screen is not explicitly included in the reconstruction process

Reconstruction of Specular Reflective Surfaces using Auto-Calibrating Deflectometry

This thesis discusses deflectometry as a reconstruction method for highly reflecting surfaces. It focuses on deflectometry alone and does not use other reconstruction techniques to supplement with additional data. It explains the measurement process and principle and provides a crash course into an efficient mathematical representation of the principles involved. Using this, it reformulates existing three-dimensional reconstructing methods, expands upon them and develops new ones

Three-Dimensional Shape Measurements of Specular Objects Using Phase-Measuring Deflectometry

The fast development in the fields of integrated circuits, photovoltaics, the automobile industry, advanced manufacturing, and astronomy have led to the importance and necessity of quickly and accurately obtaining three-dimensional (3D) shape data of specular surfaces for quality control and function evaluation. Owing to the advantages of a large dynamic range, non-contact operation, full-field and fast acquisition, high accuracy, and automatic data processing, phase-measuring deflectometry (PMD, also called fringe reflection profilometry) has been widely studied and applied in many fields. Phase information coded in the reflected fringe patterns relates to the local slope and height of the measured specular objects. The 3D shape is obtained by integrating the local gradient data or directly calculating the depth data from the phase information. We present a review of the relevant techniques regarding classical PMD. The improved PMD technique is then used to measure specular objects having discontinuous and/or isolated surfaces. Some influential factors on the measured results are presented. The challenges and future research directions are discussed to further advance PMD techniques. Finally, the application fields of PMD are briefly introduce

Absolute height measurement of specular surfaces with modified active fringe reflection photogrammetry

Deflectometric methods have existed for more than a decade for slope measurement of specular freeform surfaces through utilization of the deformation of a sample pattern after reflection from a test surface. Usually, these approaches require two-directional fringe patterns to be projected on a LCD screen or ground glass and require slope integration, which leads to some complexity for the whole measuring process. This paper proposes a new mathematical measurement model for measuring topography information of freeform specular surfaces, which integrates a virtual reference specular surface into the method of active fringe reflection delfectometry and presents a straight-forward relation between height and phase. This method only requires one direction of horizontal or vertical sinusoidal fringe patterns to be projected on a LCD screen, resulting in a significant reduction in capture time over established method. Assuming the whole system has been pre-calibrated, during the measurement process, the fringe patterns are captured separately via the virtual reference and detected freeform surfaces by a CCD camera. The reference phase can be solved according to spatial geometrical relation between LCD screen and CCD camera. The captured phases can be unwrapped with a heterodyne technique and optimum frequency selection method. Based on this calculated unwrapped-phase and that proposed mathematical model, absolute height of the inspected surface can be computed. Simulated and experimental results show that this methodology can conveniently calculate topography information for freeform and structured specular surfaces without integration and reconstruction processes

Reproducibility of two calibration procedures for phase-measuring deflectometry

Phase-measuring deflectometry is an optical inspection technique for reflective surfaces. It enables absolute, quantitative surface measurements, given a calibrated measurement setup. Two general calibration approaches can be found in literature: First, the stepwise approach uses a calibration pattern and determines internal camera parameters and external geometrical parameters in separate, consecutive steps. Second, the holistic approach optimizes all parameters collectively, based on deflectometric measurements of a calibration mirror. Whereas both approaches have been compared regarding the accuracy of subsequent surface measurements, the present contribution focuses on experimental examination of their reproducibility. In experiment E1, we assess the parameter variability by repeating both calibration procedures ten times. In an additional experiment E2, we repeat all calibration measurements related to a mirror/pattern position ten times in a row before rearranging the mirror/pattern, in order to examine the purely noise-related parameter variability. Finally, we calculate the coordinate variability of a set of world points projected onto the image planes of the calibrated cameras. The measured variability is consistently higher in E1 than in E2 (average ratio: 3.2). Unexpectedly, in both experiments, the external parameter variability also turns out to be higher for the holistic approach compared to stepwise calibration (average ratio: 2.3). This is of importance, since the holistic approach is known from literature to be more accurate than the stepwise approach, regarding their respective application to surface measurements. The image coordinate variability is comparable for both calibration approaches with an average of 0.84 and 0.21 camera pixels for E1 and E2, respectively