A Variational Stereo Method for the Three-Dimensional Reconstruction of Ocean Waves

We develop a novel remote sensing technique for the observation of waves on the ocean surface. Our method infers the 3-D waveform and radiance of oceanic sea states via a variational stereo imagery formulation. In this setting, the shape and radiance of the wave surface are given by minimizers of a composite energy functional that combines a photometric matching term along with regularization terms involving the smoothness of the unknowns. The desired ocean surface shape and radiance are the solution of a system of coupled partial differential equations derived from the optimality conditions of the energy functional. The proposed method is naturally extended to study the spatiotemporal dynamics of ocean waves and applied to three sets of stereo video data. Statistical and spectral analysis are carried out. Our results provide evidence that the observed omnidirectional wavenumber spectrum S(k) decays as k-2.5 is in agreement with Zakharov's theory (1999). Furthermore, the 3-D spectrum of the reconstructed wave surface is exploited to estimate wave dispersion and currents

Trifocal Relative Pose from Lines at Points and its Efficient Solution

We present a new minimal problem for relative pose estimation mixing point features with lines incident at points observed in three views and its efficient homotopy continuation solver. We demonstrate the generality of the approach by analyzing and solving an additional problem with mixed point and line correspondences in three views. The minimal problems include correspondences of (i) three points and one line and (ii) three points and two lines through two of the points which is reported and analyzed here for the first time. These are difficult to solve, as they have 216 and - as shown here - 312 solutions, but cover important practical situations when line and point features appear together, e.g., in urban scenes or when observing curves. We demonstrate that even such difficult problems can be solved robustly using a suitable homotopy continuation technique and we provide an implementation optimized for minimal problems that can be integrated into engineering applications. Our simulated and real experiments demonstrate our solvers in the camera geometry computation task in structure from motion. We show that new solvers allow for reconstructing challenging scenes where the standard two-view initialization of structure from motion fails.Comment: This material is based upon work supported by the National Science Foundation under Grant No. DMS-1439786 while most authors were in residence at Brown University's Institute for Computational and Experimental Research in Mathematics -- ICERM, in Providence, R

Simultaneously Reconstructing Transparent and Opaque Surfaces from Texture Images

This paper addresses the problem of reconstructing non-overlapping transparent and opaque surfaces from multiple view images. The reconstruction is attained through progressive refinement of an initial 3D shape by minimizing the error between the images of the object and the initial 3D shape. The challenge is to simultaneously reconstruct both the transparent and opaque surfaces given only a limited number of images. Any refinement methods can theoretically be applied if analytic relation between pixel value in the training images and vertices position of the initial 3D shape is known. This paper investigates such analytic relations for reconstructing opaque and transparent surfaces. The analytic relation for opaque surface follows diffuse reflection model, whereas for transparent surface follows ray tracing model. However, both relations can be converged for reconstruction both surfaces into texture mapping model. To improve the reconstruction results several strategies including regularization, hierarchical learning, and simulated annealing are investigated

Variational Stereo Imaging of Oceanic Waves with Statistical Constraints

An image processing observational technique for the stereoscopic reconstruction of the wave form of oceanic sea states is developed. The technique incorporates the enforcement of any given statistical wave law modeling the quasi Gaussianity of oceanic waves observed in nature. The problem is posed in a variational optimization framework, where the desired wave form is obtained as the minimizer of a cost functional that combines image observations, smoothness priors and a weak statistical constraint. The minimizer is obtained combining gradient descent and multigrid methods on the necessary optimality equations of the cost functional. Robust photometric error criteria and a spatial intensity compensation model are also developed to improve the performance of the presented image matching strategy. The weak statistical constraint is thoroughly evaluated in combination with other elements presented to reconstruct and enforce constraints on experimental stereo data, demonstrating the improvement in the estimation of the observed ocean surface

Single View Modeling and View Synthesis

This thesis develops new algorithms to produce 3D content from a single camera. Today, amateurs can use hand-held camcorders to capture and display the 3D world in 2D, using mature technologies. However, there is always a strong desire to record and re-explore the 3D world in 3D. To achieve this goal, current approaches usually make use of a camera array, which suffers from tedious setup and calibration processes, as well as lack of portability, limiting its application to lab experiments. In this thesis, I try to produce the 3D contents using a single camera, making it as simple as shooting pictures. It requires a new front end capturing device rather than a regular camcorder, as well as more sophisticated algorithms. First, in order to capture the highly detailed object surfaces, I designed and developed a depth camera based on a novel technique called light fall-off stereo (LFS). The LFS depth camera outputs color+depth image sequences and achieves 30 fps, which is necessary for capturing dynamic scenes. Based on the output color+depth images, I developed a new approach that builds 3D models of dynamic and deformable objects. While the camera can only capture part of a whole object at any instance, partial surfaces are assembled together to form a complete 3D model by a novel warping algorithm. Inspired by the success of single view 3D modeling, I extended my exploration into 2D-3D video conversion that does not utilize a depth camera. I developed a semi-automatic system that converts monocular videos into stereoscopic videos, via view synthesis. It combines motion analysis with user interaction, aiming to transfer as much depth inferring work from the user to the computer. I developed two new methods that analyze the optical flow in order to provide additional qualitative depth constraints. The automatically extracted depth information is presented in the user interface to assist with user labeling work. In this thesis, I developed new algorithms to produce 3D contents from a single camera. Depending on the input data, my algorithm can build high fidelity 3D models for dynamic and deformable objects if depth maps are provided. Otherwise, it can turn the video clips into stereoscopic video

Non-Contact Height Estimation for Material Extrusion Additive Systems via Monocular Imagery

Additive manufacturing is a dynamic technology with a compelling potential to advance the manufacturing industry. Despite its capacity to produce intricate designs in an efficient manner, industry still has not widely adopted additive manufacturing since its commercialization as a result of its many challenges related to quality control. The Air Force Research Laboratory (AFRL), Materials and Manufacturing Directorate, Functional Materials Division, Soft Matter Materials Branch (RXAS) requires a practical and reliable method for maintaining quality control for the production of printed flexible electronics. Height estimation is a crucial component for maintaining quality control in Material Extrusion Additive Manufacturing (MEAM), as the fundamental process for constructing any structure relies on the consecutive layering of precise extrusions. This work presents a computer vision solution to the problem of height estimation using monocular imagery as applicable to MEAM

Macro Photogrammetry & Surface Features Extraction for Paleolithic Portable Art Documentation

[EN] In this article, we propose a methodology for the archaeological documentation of limestone plaquettes decorated with faint paintings and fine engravings. The plaquette number 16330 is presented, belonging to the portable art collection in Cova del Parpalló (Gandía, Spain), one of the most important Paleolithic sites in the UNESCO¿s Rock Art of the Mediterranean Basin on the Iberian Peninsula. Macro photogrammetry is used to generate a 3D model and basic treatments on raster images. The resulting 3D model has a spatial resolution of tens of microns and was used to generate a digital elevation model (DEM) and orthorectified macro photographs for documenting the engravings and paintings. All stages of the workflow are detailed in-depth, specifying the data collection parameters and the configuration used in the subsequent processing with HyperCube and DStretch software. The resulting documentation is accurate, reproducible, and objective and allows the reinterpretation of the available graphic documentation started in the 1990s.This research was funded by Generalitat Valenciana (PROMETEO/2017/060) and Ministerio de la Ciencia e Innovación (HAR2017-85153-P).Cabrelles, M.; Lerma, JL.; Villaverde, V. (2020). Macro Photogrammetry & Surface Features Extraction for Paleolithic Portable Art Documentation. Applied Sciences. 10:1-16. https://doi.org/10.3390/app10196908S11610Porter, S. T., Huber, N., Hoyer, C., & Floss, H. (2016). Portable and low-cost solutions to the imaging of Paleolithic art objects: A comparison of photogrammetry and reflectance transformation imaging. Journal of Archaeological Science: Reports, 10, 859-863. doi:10.1016/j.jasrep.2016.07.013Cassen, S., Lescop, L., Grimaud, V., & Robin, G. (2014). Complementarity of acquisition techniques for the documentation of Neolithic engravings: lasergrammetric and photographic recording in Gavrinis passage tomb (Brittany, France). Journal of Archaeological Science, 45, 126-140. doi:10.1016/j.jas.2014.02.019López-Menchero Bendicho, V. M., Marchante Ortega, Á., Vincent, M., Cárdenas Martín-Buitrago, Á. J., & Onrubia Pintado, J. (2017). Uso combinado de la fotografía digital nocturna y de la fotogrametría en los procesos de documentación de petroglifos: el caso de Alcázar de San Juan (Ciudad Real, España). Virtual Archaeology Review, 8(17), 64. doi:10.4995/var.2017.6820Jalandoni, A., & Kottermair, M. (2018). Rock art as microtopography. Geoarchaeology, 33(5), 579-593. doi:10.1002/gea.21677Defrasne, C. (2014). Digital image enhancement for recording rupestrian engravings: applications to an alpine rockshelter. Journal of Archaeological Science, 50, 31-38. doi:10.1016/j.jas.2014.06.010Le Quellec, J.-L., Duquesnoy, F., & Defrasne, C. (2015). Digital image enhancement with DStretch ® : Is complexity always necessary for efficiency? Digital Applications in Archaeology and Cultural Heritage, 2(2-3), 55-67. doi:10.1016/j.daach.2015.01.003Evans, L., & Mourad, A.-L. (2018). DStretch® and Egyptian tomb paintings: A case study from Beni Hassan. Journal of Archaeological Science: Reports, 18, 78-84. doi:10.1016/j.jasrep.2018.01.011Rodríguez González, E., Pastor, S. C., & Casals, J. R. (2019). Lost colours: Photogrammetry, image analysis using the DStretch plugin, and 3-D modelling of post-firing painted pottery from the south west Iberian Peninsula. Digital Applications in Archaeology and Cultural Heritage, 13, e00093. doi:10.1016/j.daach.2019.e00093Domingo, I., García-Argüelles, P., Nadal, J., Fullola, J. M., Lerma, J. L., & Cabrelles, M. (2019). Humanizing European Paleolithic art: A new visual evidence of human/bird interactions at L’Hort de la Boquera site (Margalef de Montsant, Tarragona, Spain). L’Anthropologie, 123(1), 1-18. doi:10.1016/j.anthro.2019.01.001Menna, F., Nocerino, E., Morabito, D., Farella, E. M., Perini, M., & Remondino, F. (2017). AN OPEN SOURCE LOW-COST AUTOMATIC SYSTEM FOR IMAGE-BASED 3D DIGITIZATION. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W8, 155-162. doi:10.5194/isprs-archives-xlii-2-w8-155-2017Porter, S. T., Roussel, M., & Soressi, M. (2016). A Simple Photogrammetry Rig for the Reliable Creation of 3D Artifact Models in the Field. Advances in Archaeological Practice, 4(1), 71-86. doi:10.7183/2326-3768.4.1.71Angheluță, L. M., & Rădvan, R. (2019). MACRO PHOTOGRAMMETRY FOR THE DAMAGE ASSESSMENT OF ARTWORK PAINTED SURFACES. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W15, 101-107. doi:10.5194/isprs-archives-xlii-2-w15-101-2019Menna, F., Nocerino, E., Remondino, F., Dellepiane, M., Callieri, M., & Scopigno, R. (2016). 3D DIGITIZATION OF AN HERITAGE MASTERPIECE - A CRITICAL ANALYSIS ON QUALITY ASSESSMENT. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLI-B5, 675-683. doi:10.5194/isprsarchives-xli-b5-675-2016Lerma, J. L., Navarro, S., Cabrelles, M., & Seguí, A. E. (2010). Camera Calibration with Baseline Distance Constraints. The Photogrammetric Record, 25(130), 140-158. doi:10.1111/j.1477-9730.2010.00579.xDíaz-Guardamino, M., García Sanjuán, L., Wheatley, D., & Rodríguez Zamora, V. (2015). RTI and the study of engraved rock art: A re-examination of the Iberian south-western stelae of Setefilla and Almadén de la Plata 2 (Seville, Spain). Digital Applications in Archaeology and Cultural Heritage, 2(2-3), 41-54. doi:10.1016/j.daach.2015.07.002Quesada, E., & Harman, J. (2019). A step further in rock art digital enhancements. DStretch on Gigapixel imaging. Digital Applications in Archaeology and Cultural Heritage, 13, e00098. doi:10.1016/j.daach.2019.e0009

Depth Recovery with Rectification using Single-Lens Prism based Stereovision System

Ph.DDOCTOR OF PHILOSOPH