What Is Around The Camera?

How much does a single image reveal about the environment it was taken in? In this paper, we investigate how much of that information can be retrieved from a foreground object, combined with the background (i.e. the visible part of the environment). Assuming it is not perfectly diffuse, the foreground object acts as a complexly shaped and far-from-perfect mirror. An additional challenge is that its appearance confounds the light coming from the environment with the unknown materials it is made of. We propose a learning-based approach to predict the environment from multiple reflectance maps that are computed from approximate surface normals. The proposed method allows us to jointly model the statistics of environments and material properties. We train our system from synthesized training data, but demonstrate its applicability to real-world data. Interestingly, our analysis shows that the information obtained from objects made out of multiple materials often is complementary and leads to better performance.Comment: Accepted to ICCV. Project: http://homes.esat.kuleuven.be/~sgeorgou/multinatillum

Fully-automatic inverse tone mapping algorithm based on dynamic mid-level tone mapping

High Dynamic Range (HDR) displays can show images with higher color contrast levels and peak luminosities than the common Low Dynamic Range (LDR) displays. However, most existing video content is recorded and/or graded in LDR format. To show LDR content on HDR displays, it needs to be up-scaled using a so-called inverse tone mapping algorithm. Several techniques for inverse tone mapping have been proposed in the last years, going from simple approaches based on global and local operators to more advanced algorithms such as neural networks. Some of the drawbacks of existing techniques for inverse tone mapping are the need for human intervention, the high computation time for more advanced algorithms, limited low peak brightness, and the lack of the preservation of the artistic intentions. In this paper, we propose a fully-automatic inverse tone mapping operator based on mid-level mapping capable of real-time video processing. Our proposed algorithm allows expanding LDR images into HDR images with peak brightness over 1000 nits, preserving the artistic intentions inherent to the HDR domain. We assessed our results using the full-reference objective quality metrics HDR-VDP-2.2 and DRIM, and carrying out a subjective pair-wise comparison experiment. We compared our results with those obtained with the most recent methods found in the literature. Experimental results demonstrate that our proposed method outperforms the current state-of-the-art of simple inverse tone mapping methods and its performance is similar to other more complex and time-consuming advanced techniques

Algorithms for compression of high dynamic range images and video

The recent advances in sensor and display technologies have brought upon the High Dynamic Range (HDR) imaging capability. The modern multiple exposure HDR sensors can achieve the dynamic range of 100-120 dB and LED and OLED display devices have contrast ratios of 10^5:1 to 10^6:1. Despite the above advances in technology the image/video compression algorithms and associated hardware are yet based on Standard Dynamic Range (SDR) technology, i.e. they operate within an effective dynamic range of up to 70 dB for 8 bit gamma corrected images. Further the existing infrastructure for content distribution is also designed for SDR, which creates interoperability problems with true HDR capture and display equipment. The current solutions for the above problem include tone mapping the HDR content to fit SDR. However this approach leads to image quality associated problems, when strong dynamic range compression is applied. Even though some HDR-only solutions have been proposed in literature, they are not interoperable with current SDR infrastructure and are thus typically used in closed systems. Given the above observations a research gap was identified in the need for efficient algorithms for the compression of still images and video, which are capable of storing full dynamic range and colour gamut of HDR images and at the same time backward compatible with existing SDR infrastructure. To improve the usability of SDR content it is vital that any such algorithms should accommodate different tone mapping operators, including those that are spatially non-uniform. In the course of the research presented in this thesis a novel two layer CODEC architecture is introduced for both HDR image and video coding. Further a universal and computationally efficient approximation of the tone mapping operator is developed and presented. It is shown that the use of perceptually uniform colourspaces for internal representation of pixel data enables improved compression efficiency of the algorithms. Further proposed novel approaches to the compression of metadata for the tone mapping operator is shown to improve compression performance for low bitrate video content. Multiple compression algorithms are designed, implemented and compared and quality-complexity trade-offs are identified. Finally practical aspects of implementing the developed algorithms are explored by automating the design space exploration flow and integrating the high level systems design framework with domain specific tools for synthesis and simulation of multiprocessor systems. The directions for further work are also presented

LAN-HDR: Luminance-based Alignment Network for High Dynamic Range Video Reconstruction

As demands for high-quality videos continue to rise, high-resolution and high-dynamic range (HDR) imaging techniques are drawing attention. To generate an HDR video from low dynamic range (LDR) images, one of the critical steps is the motion compensation between LDR frames, for which most existing works employed the optical flow algorithm. However, these methods suffer from flow estimation errors when saturation or complicated motions exist. In this paper, we propose an end-to-end HDR video composition framework, which aligns LDR frames in the feature space and then merges aligned features into an HDR frame, without relying on pixel-domain optical flow. Specifically, we propose a luminance-based alignment network for HDR (LAN-HDR) consisting of an alignment module and a hallucination module. The alignment module aligns a frame to the adjacent reference by evaluating luminance-based attention, excluding color information. The hallucination module generates sharp details, especially for washed-out areas due to saturation. The aligned and hallucinated features are then blended adaptively to complement each other. Finally, we merge the features to generate a final HDR frame. In training, we adopt a temporal loss, in addition to frame reconstruction losses, to enhance temporal consistency and thus reduce flickering. Extensive experiments demonstrate that our method performs better or comparable to state-of-the-art methods on several benchmarks.Comment: ICCV 202

Large Deployable Reflector (LDR) system concept and technology definition study. Volume 2: Technology assessment and technology development plan

A study was conducted to define reasonable and representative LDR system concepts for the purpose of defining a technology development program aimed at providing the requisite technological capability necessary to start LDR development by the end of 1991. This volume presents thirteen technology assessments and technology development plans, as well as an overview and summary of the LDR concepts. Twenty-two proposed augmentation projects are described (selected from more than 30 candidates). The five LDR technology areas most in need of supplementary support are: cryogenic cooling; astronaut assembly of the optically precise LDR in space; active segmented primary mirror; dynamic structural control; and primary mirror contamination control. Three broad, time-phased, five-year programs were synthesized from the 22 projects, scheduled, and funding requirements estimated

Does HDR pre-processing improve the accuracy of 3D models obtained by means of two conventional SfM-MVS software packages? the case of the corral del veleta rock glacier

La precisión de los diferentes flujos de trabajo usando la estructura del Movimiento y técnicas estéreo vista múltiple (SfM-MVS) está probado. Nubes de doce puntos del Corral del Veleta glaciar de roca, en España, se produjeron con dos diferentes paquetes de software (123D Catch y Agisoft Photoscan), utilizando imágenes de bajo rango dinámico y composiciones de alto rango dinámico (HDR) para tres años distintos (2011, 2012 y 2014). La exactitud de las nubes de puntos resultante se evaluó mediante modelos de referencia adquiridos cada año con un escáner láser terrestre. Tres parámetros se utilizaron para estimar la precisión de cada nube de puntos: el RMSE, la nube-a-nube distancia (C2C) y la Multiscale-Model a comparación de modelos (M3C2). El M3C2 el error promedio varió de 0,084 m (desviación estándar de 0,403 m) a 1.451 m (desviación estándar de 1.625 m). Agisoft Photoscan superar 123D Catch, produciendo más precisas y densas nubes de puntos en 11 de 12 casos, siendo este trabajo, la primera comparación entre ambos paquetes de software en la literatura. Ninguna mejora significativa fue observada a través de HDR de pre-procesamiento. A nuestro conocimiento, esta es la primera vez que la exactitud geométrica de modelos 3D obtenidos utilizando LDR y HDR composiciones son comparados. Estos hallazgos pueden ser de interés para los investigadores que deseen para estimar los cambios geomórficas con SfM-MVS enfoques.The accuracy of different workflows using Structure-from-Motion and Multi-View-Stereo techniques (SfM-MVS) is tested. Twelve point clouds of the Corral del Veleta rock glacier, in Spain, were produced with two different software packages (123D Catch and Agisoft Photoscan), using Low Dynamic Range images and High Dynamic Range compositions (HDR) for three different years (2011, 2012 and 2014). The accuracy of the resulting point clouds was assessed using benchmark models acquired every year with a Terrestrial Laser Scanner. Three parameters were used to estimate the accuracy of each point cloud: the RMSE, the Cloud-to-Cloud distance (C2C) and the Multiscale-Model-to-Model comparison (M3C2). The M3C2 mean error ranged from 0.084 m (standard deviation of 0.403 m) to 1.451 m (standard deviation of 1.625 m). Agisoft Photoscan overcome 123D Catch, producing more accurate and denser point clouds in 11 out 12 cases, being this work, the first available comparison between both software packages in the literature. No significant improvement was observed using HDR pre-processing. To our knowledge, this is the first time that the geometrical accuracy of 3D models obtained using LDR and HDR compositions are compared. These findings may be of interest for researchers who wish to estimate geomorphic changes using SfM-MVS approaches.peerReviewe