Multifocus HDR VIS/NIR hyperspectral imaging and its application to works of art

The authors would like to thank Mr. Francisco Fernández Fábregas, owner of the Transfiguration of Christ for allowing us the privilege of studying this painting. We also acknowledge the collaboration of Angela Tate.This paper presents a complete framework for capturing and processing hyperspectral reflectance images of artworks in situ, using a hyperspectral line scanner. These capturing systems are commonly used in laboratory conditions synchronized with scanning stages specifically designed for planar surfaces. However, when the intended application domain does not allow for image capture in these controlled conditions, achieving useful spectral reflectance image data can be a very challenging task (due to uncontrolled illumination, high-dynamic range (HDR) conditions in the scene, and the influence of chromatic aberration on the image quality, among other factors). We show, for the first time, all the necessary steps in the image capturing and post-processing in order to obtain high-quality HDR-based reflectance in the visible and near infrared, directly from the data captured by using a hyperspectral line scanner coupled to a rotating tripod. Our results show that the proposed method outperforms the normal capturing process in terms of dynamic range, color and spectral accuracy. To demonstrate the potential interest of this processing strategy for on-site analysis of artworks, we applied it to the study of a vintage copy of the famous painting “Transfiguration” by Raphael, as well as a facsimile of “The Golden Haggadah” from the British Library of London. The second piece has been studied for the identification of highly reflective gold-foil covered areas.Spanish Ministry of Economy and Competitiveness, DPI2015-64571-R, ECQM2018-004952-

Beyond the Pixel: a Photometrically Calibrated HDR Dataset for Luminance and Color Prediction

Light plays an important role in human well-being. However, most computer vision tasks treat pixels without considering their relationship to physical luminance. To address this shortcoming, we introduce the Laval Photometric Indoor HDR Dataset, the first large-scale photometrically calibrated dataset of high dynamic range 360{\deg} panoramas. Our key contribution is the calibration of an existing, uncalibrated HDR Dataset. We do so by accurately capturing RAW bracketed exposures simultaneously with a professional photometric measurement device (chroma meter) for multiple scenes across a variety of lighting conditions. Using the resulting measurements, we establish the calibration coefficients to be applied to the HDR images. The resulting dataset is a rich representation of indoor scenes which displays a wide range of illuminance and color, and varied types of light sources. We exploit the dataset to introduce three novel tasks, where: per-pixel luminance, per-pixel color and planar illuminance can be predicted from a single input image. Finally, we also capture another smaller photometric dataset with a commercial 360{\deg} camera, to experiment on generalization across cameras. We are optimistic that the release of our datasets and associated code will spark interest in physically accurate light estimation within the community. Dataset and code are available at https://lvsn.github.io/beyondthepixel/

High Dynamic Range Spectral Imaging Pipeline For Multispectral Filter Array Cameras

Spectral filter arrays imaging exhibits a strong similarity with color filter arrays. This permits us to embed this technology in practical vision systems with little adaptation of the existing solutions. In this communication, we define an imaging pipeline that permits high dynamic range (HDR)-spectral imaging, which is extended from color filter arrays. We propose an implementation of this pipeline on a prototype sensor and evaluate the quality of our implementation results on real data with objective metrics and visual examples. We demonstrate that we reduce noise, and, in particular we solve the problem of noise generated by the lack o

An interest point based illumination condition matching approach to photometric registration within augmented reality worlds

With recent and continued increases in computing power, and advances in the field of computer graphics, realistic augmented reality environments can now offer inexpensive and powerful solutions in a whole range of training, simulation and leisure applications. One key challenge to maintaining convincing augmentation, and therefore user immersion, is ensuring consistent illumination conditions between virtual and real environments, so that objects appear to be lit by the same light sources. This research demonstrates how real world lighting conditions can be determined from the two-dimensional view of the user. Virtual objects can then be illuminated and virtual shadows cast using these conditions. This new technique uses pairs of interest points from real objects and the shadows that they cast, viewed from a binocular perspective, to determine the position of the illuminant. This research has been initially focused on single point light sources in order to show the potential of the technique and has investigated the relationships between the many parameters of the vision system. Optimal conditions have been discovered by mapping the results of experimentally varying parameters such as FoV, camera angle and pose, image resolution, aspect ratio and illuminant distance. The technique is able to provide increased robustness where greater resolution imagery is used. Under optimal conditions it is possible to derive the position of a real world light source with low average error. An investigation of available literature has revealed that other techniques can be inflexible, slow, or disrupt scene realism. This technique is able to locate and track a moving illuminant within an unconstrained, dynamic world without the use of artificial calibration objects that would disrupt scene realism. The technique operates in real-time as the new algorithms are of low computational complexity. This allows high framerates to be maintained within augmented reality applications. Illuminant updates occur several times a second on an average to high end desktop computer. Future work will investigate the automatic identification and selection of pairs of interest points and the exploration of global illuminant conditions. The latter will include an analysis of more complex scenes and the consideration of multiple and varied light sources.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Variable illumination and invariant features for detecting and classifying varnish defects

This work presents a method to detect and classify varnish defects on wood surfaces. Since these defects are only partially visible under certain illumination directions, one image doesn\u27t provide enough information for a recognition task. A classification requires inspecting the surface under different illumination directions, which results in image series. The information is distributed along this series and can be extracted by merging the knowledge about the defect shape and light direction

Doctor of Philosophy

dissertationThree-dimensional (3D) models of industrial plant primitives are used extensively in modern asset design, management, and visualization systems. Such systems allow users to efficiently perform tasks in Computer Aided Design (CAD), life-cycle management, construction progress monitoring, virtual reality training, marketing walk-throughs, or other visualization. Thus, capturing industrial plant models has correspondingly become a rapidly growing industry. The purpose of this research was to demonstrate an efficient way to ascertain physical model parameters of reflectance properties of industrial plant primitives for use in CAD and 3D modeling visualization systems. The first part of this research outlines the sources of error corresponding to 3D models created from Light Detection and Ranging (LiDAR) point clouds. Fourier analysis exposes the error due to a LiDAR system's finite sampling rate. Taylor expansion illustrates the errors associated with linearization due to flat polygonal surfaces. Finally, a statistical analysis of the error associated with LiDar scanner hardware is presented. The second part of this research demonstrates a method for determining Phong specular and Oren-Nayar diffuse reflectance parameters for modeling and rendering pipes, the most ubiquitous form of industrial plant primitives. For specular reflectance, the Phong model is used. Estimates of specular and diffuse parameters of two ideal cylinders and one measured cylinder using brightness data acquired from a LiDAR scanner are presented. The estimated reflectance model of the measured cylinder has a mean relative error of 2.88% and a standard deviation of relative error of 4.0%. The final part of this research describes a method for determining specular, diffuse and color material properties and applies the method to seven pipes from an industrial plant. The colorless specular and diffuse properties were estimated by numerically inverting LiDAR brightness data. The color ambient and diffuse properties are estimated using k-means clustering. The colorless properties yielded estimated brightness values that are within an RMS of 3.4% with a maximum of 7.0% and a minimum of 1.6%. The estimated color properties effected an RMS residual of 13.2% with a maximum of 20.3% and a minimum of 9.1%

3D panoramic imaging for virtual environment construction

The project is concerned with the development of algorithms for the creation of photo-realistic 3D virtual environments, overcoming problems in mosaicing, colour and lighting changes, correspondence search speed and correspondence errors due to lack of surface texture. A number of related new algorithms have been investigated for image stitching, content based colour correction and efficient 3D surface reconstruction. All of the investigations were undertaken by using multiple views from normal digital cameras, web cameras and a ”one-shot” panoramic system. In the process of 3D reconstruction a new interest points based mosaicing method, a new interest points based colour correction method, a new hybrid feature and area based correspondence constraint and a new structured light based 3D reconstruction method have been investigated. The major contributions and results can be summarised as follows: • A new interest point based image stitching method has been proposed and investigated. The robustness of interest points has been tested and evaluated. Interest points have been proved robust to changes in lighting, viewpoint, rotation and scale. • A new interest point based method for colour correction has been proposed and investigated. The results of linear and linear plus affine colour transforms have proved more accurate than traditional diagonal transforms in accurately matching colours in panoramic images. • A new structured light based method for correspondence point based 3D reconstruction has been proposed and investigated. The method has been proved to increase the accuracy of the correspondence search for areas with low texture. Correspondence speed has also been increased with a new hybrid feature and area based correspondence search constraint. • Based on the investigation, a software framework has been developed for image based 3D virtual environment construction. The GUI includes abilities for importing images, colour correction, mosaicing, 3D surface reconstruction, texture recovery and visualisation. • 11 research papers have been published.EThOS - Electronic Theses Online ServiceGBUnited Kingdo