Uncalibrated, Two Source Photo-Polarimetric Stereo

none5siAvailable online: 6 May 2021.In this paper we present methods for estimating shape from polarisation and shading information, i.e. photo-polarimetric shape estimation, under varying, but unknown, illumination, i.e. in an uncalibrated scenario. We propose several alternative photo-polarimetric constraints that depend upon the partial derivatives of the surface and show how to express them in a unified system of partial differential equations of which previous work is a special case. By careful combination and manipulation of the constraints, we show how to eliminate non-linearities such that a discrete version of the problem can be solved using linear least squares. We derive a minimal, combinatorial approach for two source illumination estimation which we use with RANSAC for robust light direction and intensity estimation. We also introduce a new method for estimating a polarisation image from multichannel data and provide methods for estimating albedo and refractive index. We evaluate lighting, shape, albedo and refractive index estimation methods on both synthetic and real-world data showing improvements over existing state-of-the-art.noneTozza, Silvia; Zhu, Dizhong; Smith, William; Ramamoorthi, Ravi; Hancock, EdwinTozza, Silvia; Zhu, Dizhong; Smith, William; Ramamoorthi, Ravi; Hancock, Edwi

Reconstructing mass-conserved water surfaces using shape from shading and optical flow

This paper introduces a method for reconstructing water from real video footage. Using a single input video, the proposed method produces a more informative reconstruction from a wider range of possible scenes than the current state of the art. The key is the combination of vision algorithms and physics laws. Shape from shading is used to capture the change of the water's surface, from which a vertical velocity gradient field is calculated. Such a gradient field is used to constrain the tracking of horizontal velocities by minimizing an energy function as a weighted combination of mass-conservation and intensity-conservation. Hence the final reconstruction contains a dense velocity field that is incompressible in 3D. The proposed method is efficient and performs consistently well across water of different types

3D shape reconstruction using a polarisation reflectance model in conjunction with shading and stereo

Reconstructing the 3D geometry of objects from images is a fundamental problem in computer vision. This thesis focuses on shape from polarisation where the goal is to reconstruct a dense depth map from a sequence of polarisation images. Firstly, we propose a linear differential constraints approach to depth estimation from polarisation images. We demonstrate that colour images can deliver more robust polarimetric measurements compared to monochrome images. Then we explore different constraints by taking the polarisation images under two different light conditions with fixed view and show that a dense depth map, albedo map and refractive index can be recovered. Secondly, we propose a nonlinear method to reconstruct depth by an end-to-end method. We re-parameterise a polarisation reflectance model with respect to the depth map, and predict an optimum depth map by minimising an energy cost function between the prediction from the reflectance model and observed data using nonlinear least squares. Thirdly, we propose to enhance the polarisation camera with an additional RGB camera in a second view. We construct a higher-order graphical model by utilising an initial rough depth map estimated from the stereo views. The graphical model will correct the surface normal ambiguity which arises from the polarisation reflectance model. We then build a linear system to combine the corrected surface normal, polarimetric information and rough depth map to produce an accurate and dense depth map. Lastly, we derive a mixed polarisation model that describes specular and diffuse polarisation as well as mixtures of the two. This model is more physically accurate and allows us to decompose specular and diffuse reflectance from multiview images

NOVEL DENSE STEREO ALGORITHMS FOR HIGH-QUALITY DEPTH ESTIMATION FROM IMAGES

This dissertation addresses the problem of inferring scene depth information from a collection of calibrated images taken from different viewpoints via stereo matching. Although it has been heavily investigated for decades, depth from stereo remains a long-standing challenge and popular research topic for several reasons. First of all, in order to be of practical use for many real-time applications such as autonomous driving, accurate depth estimation in real-time is of great importance and one of the core challenges in stereo. Second, for applications such as 3D reconstruction and view synthesis, high-quality depth estimation is crucial to achieve photo realistic results. However, due to the matching ambiguities, accurate dense depth estimates are difficult to achieve. Last but not least, most stereo algorithms rely on identification of corresponding points among images and only work effectively when scenes are Lambertian. For non-Lambertian surfaces, the brightness constancy assumption is no longer valid. This dissertation contributes three novel stereo algorithms that are motivated by the specific requirements and limitations imposed by different applications. In addressing high speed depth estimation from images, we present a stereo algorithm that achieves high quality results while maintaining real-time performance. We introduce an adaptive aggregation step in a dynamic-programming framework. Matching costs are aggregated in the vertical direction using a computationally expensive weighting scheme based on color and distance proximity. We utilize the vector processing capability and parallelism in commodity graphics hardware to speed up this process over two orders of magnitude. In addressing high accuracy depth estimation, we present a stereo model that makes use of constraints from points with known depths - the Ground Control Points (GCPs) as referred to in stereo literature. Our formulation explicitly models the influences of GCPs in a Markov Random Field. A novel regularization prior is naturally integrated into a global inference framework in a principled way using the Bayes rule. Our probabilistic framework allows GCPs to be obtained from various modalities and provides a natural way to integrate information from various sensors. In addressing non-Lambertian reflectance, we introduce a new invariant for stereo correspondence which allows completely arbitrary scene reflectance (bidirectional reflectance distribution functions - BRDFs). This invariant can be used to formulate a rank constraint on stereo matching when the scene is observed by several lighting configurations in which only the lighting intensity varies

High-speed imaging of short wind waves by shape from refraction

This paper introduces the first high-speed system for slope imaging of wind-induced short water waves. The imaging slope gauge method is used, which is based on the shape from refraction principle. The downward looking camera with a telecentric lens observes the refraction of light rays coming from a high power custom telecentric LED light source that is placed underneath the wind wave facility. The light source can be programmed to arbitrary intensity gradients in the x- and y-direction, so that the origin of a light ray is coded in intensity. Four gradient images (acquired at 6000 fps) are combined for one 2D slope image. By only using intensity ratios, the measurements become independent of lens effects from the curved water surface and inhomogeneities in the light source. Independence of wave height is guaranteed by using telecentric illumination and telecentric imaging. The system is capable to measure the slopes of a wind-driven water surface in the Heidelberg Aeolotron wind-wave facility on a footprint of 200 x 160 mm with a spatial resolution of 0.22 mm and a temporal resolution of more than 1500 fps. For the first time, it is now possible to investigate the structure of short wind-induced waves with sufficient spatial and temporal resolution to study their dynamic characteristics without aliasing effects. Example images and a video of a 3D reconstructed water surface are shown to illustrate the principle

Sparse variational regularization for visual motion estimation

The computation of visual motion is a key component in numerous computer vision tasks such as object detection, visual object tracking and activity recognition. Despite exten- sive research effort, efficient handling of motion discontinuities, occlusions and illumina- tion changes still remains elusive in visual motion estimation. The work presented in this thesis utilizes variational methods to handle the aforementioned problems because these methods allow the integration of various mathematical concepts into a single en- ergy minimization framework. This thesis applies the concepts from signal sparsity to the variational regularization for visual motion estimation. The regularization is designed in such a way that it handles motion discontinuities and can detect object occlusions

Fotonien kartoitus reaaliajassa: Epäsuoran valaistuksen soveltamista dynaamisille ympäristöille reaaliajassa

The focus of this thesis is to provide better methods to simulate the behaviour of light in synthesis of photo-realistic images for real-time applications. Improvements introduced in this work are related to indirect component of the illumination, also known as global illumination, in which the contributed light has already been reflected from surface at least once. While there are a number of effective global illumination techniques based on precomputation that work well with static scenes, including global illumination for scenes with dynamic lighting and dynamic geometry remains a challenging problem. In this thesis, we describe a real-time global illumination algorithm based on photon mapping that evaluates several bounces of indirect lighting without any precomputed data in scenes with both dynamic lighting and fully dynamic geometry. To make photon mapping possible within the performance limitations of the real-time rendering, we utilize and expand on several optimization methods, such as reflective shadow maps, stratified sampling and Russian Roulette. Furthermore, we introduce an improved distribution kernel for the screen space irradiance estimation of the photon mapping. Finally, we present a new filtering solution for photon mapping.Opinnäytetyön painopisteenä on tarjota parempia menetelmiä valon käyttäytymisen simuloimiseksi reaaliaikaisten sovelluksien realistisessa kuvasynteesissä. Tässä työssä esitetyt parannukset liittyvät valaistuksen epäsuoraan komponenttiin, (tunnetaan myös globaalina valaistuksena), jossa valo on kulkenut ainakin yhden pintaheijastuksen kautta. On olemassa tehokkaita globaaleja valaistustekniikoita, jotka perustuvat ennakkotietoon. Nämä tekniikat toimivat hyvin staattisten ympäristöjen kanssa, mutta dynaamisen valaistusta ja geometriaa ympäristöt ovat edelleen haastava ongelma. Tässä opinnäytetyössä kuvataan reaaliaikainen globaali valaistusalgoritmi, joka perustuu fotonikartoitukseen ja jossa arvioidaan useita epäsuoran valaistuksen askelmia ilman ennalta laskettua. Jotta fotonikartoitus olisi mahdollista reaaliaikaisen renderoinnin suorituskyvyn määrittämissä rajoitteissa, käytämme useita optimointimenetelmiä, kuten heijastavia varjo-karttoja, kerrostettuja näytteitä ja venäläistä rulettia. Lisäksi esitämme parannetun distribuutiokernelin fotonikartoituksen säteilytysvoimakkuuden estimoinnille. Lopuksi esitämme uuden suodatusratkaisun fotonikartoitukseen