Adaptive exposure estimation for high dynamic range imaging applied to natural scenes and daylight skies

Digital imaging of natural scenes and optical phenomena present on them (such as shadows, twilights, and crepuscular rays) can be a very challenging task because of the range spanned by the radiances impinging on the capture system. We propose a novel method for estimating the set of exposure times (bracketing set) needed to capture the full dynamic range of a scene with high dynamic range (HDR) content. The proposed method is adaptive to scene content and to any camera response and configuration, and it works on-line since the exposure times are estimated as the capturing process is ongoing. Besides, it requires no a priori information about scene content or radiance values. The resulting bracketing sets are minimal in the default method settings, but the user can set a tolerance for the maximum percentage of pixel population that is underexposed or saturated, which allows for a higher number of shots if a better signal-to-noise ratio (SNR) in the HDR scene is desired. This method is based on the use of the camera response function that is needed for building the HDR radiance map by stitching together several differently exposed low dynamic range images of the scene. The use of HDR imaging techniques converts our digital camera into a tool for measuring the relative radiance outgoing from each point of the scene, and for each color channel. This is important for accurate characterization of optical phenomena present in the atmosphere while not suffering any loss of information due to its HDR.We have compared our method with the most similar one developed so far [IEEE Trans. Image Process. 17, 1864 (2008)]. Results of the experiments carried out for 30 natural scenes show that our proposed method equals or outperforms the previously developed best approach, with less shots and shorter exposure times, thereby asserting the advantage of being adaptive to scene content for exposure time estimation. As we can also tune the balance between capturing time and the SNR in our method, we have compared its SNR performance against that of Barakat’s method as well as against a ground-truth HDR image of maximum SNR. Results confirm the success of the proposed method in exploiting its tunability to achieve the desired balance of total Δt and SNR

Colorimetric and spectral data analysis of consolidants used for preservation of medieval plasterwork

The incorporation of new methods of heritage analysis belonging to other branches of science is currently providing very useful tools for the examination of preservation products. This paper outlines the use of spectral images as an alternative to traditional colorimetry measurements carried out with a spectrophotometer for the evaluation of color changes produced by consolidation treatments applied on polychromed plasterwork. Thus, for this investigation we used a total of 18 plaster test specimens which reproduce the techniques and materials present in plasterwork from the medieval era, on which a selection of six currently-used consolidants was applied. By doing this, we have proved that the use of this method over traditional colorimetry has several advantages, such as the analysis of spatial homogeneity or obtaining colorimetric data of the entire scanned surface.Ministry of Economy andCompetitiveness and by the European Regional Development Fund(MINECO/ERDF, EU) within the framework of the Research ProjectHAR2015-66139-P, as well as by the Excellence Project of theAndalusian Regional Government P12-HUM-1941

Evaluation of Cleaning Processes Using Colorimetric and Spectral Data for the Removal of Layers of Limewash from Medieval Plasterwork

In the cultural heritage preservation of medieval buildings, it is common to find plaster walls covered in lime, which previously were painted in polychromy. The conservation interventions usually try to remove the whitewash, whilst maintaining the original color of the painted wall as much as possible. However, there is no agreement on which cleaning technique best preserves the original appearance of the colored plaster. Different pigments found below the lime layer may behave differently depending on the cleaning technique used. Usually, colorimetric or photometric area-based measurements are carried out to study the color of the cleaned areas to compare with their original color, obtained from pre-made plaster probes. However, this methodology fails when the mean color difference is not enough to fully characterize the changes in texture and color appearance. This study presents a set of experiments carried out using two different pigments (cinnabar and malachite) covered with lime, and treated with nine different cleaning techniques on plaster probes prepared according to medieval techniques. We have studied the effect of the cleaning process on the color and the homogeneity of the samples using a hyperspectral imaging workflow. Four different analysis methods are presented and discussed. Our results show that the proposed analysis is able to provide a much more comprehensive and diversified characterization of the quality of the cleaning method compared to the commonly used colorimetric or photometric area-based measurements.Ministry of Economy and Competitiveness HAR2015-66139-PEuropean Union (EU) HAR2015-66139-

Eight-Channel Multispectral Image Database for Saliency Prediction

Saliency prediction is a very important and challenging task within the computer vision community. Many models exist that try to predict the salient regions on a scene from its RGB image values. Several new models are developed, and spectral imaging techniques may potentially overcome the limitations found when using RGB images. However, the experimental study of such models based on spectral images is difficult because of the lack of available data to work with. This article presents the first eight-channel multispectral image database of outdoor urban scenes together with their gaze data recorded using an eyetracker over several observers performing different visualization tasks. Besides, the information from this database is used to study whether the complexity of the images has an impact on the saliency maps retrieved from the observers. Results show that more complex images do not correlate with higher differences in the saliency maps obtained.Spanish Ministry of Science, Innovation, and Universities (MICINN) RTI2018-094738-B-I00European Commissio

Improving unsupervised saliency detection by migrating from RGB to multispectral images

Saliency detection has been an important topic during the last decade. The main goal of saliency detection models is to detect the most relevant objects in a given scene. Most of these models use RGB (Red, Green, Blue) images as an input because they mainly focus on applications where features (eg, faces, textures, colors, or human silhouettes) are extracted from color images, and there are many labeled databases available for RGB-based saliency data. Nevertheless, the use of RGB inputs clearly limits the amount of information from where to extract the salient regions as spectral information is lost during the color image recording. On the contrary, multispectral systems are able to capture more than three bands in a single capture and can retrieve information from the full spectrum at a pixel. The main aim of this study is to investigate the advantages of using multispectral images instead of RGB images for saliency detection within the framework of unsupervised models. We compare the performance of several unsupervised saliency models with both RGB and multispectral images using a specific dataset of multispectral images with ground-truth data extracted from observers' fixation patterns. Our results show a general improvement when multispectral information is taken into account. The saliency maps estimated by using the multispectral features are closer to the ground-truth data, with the simplest Graph-based visual saliency and Boolean Map-based models showing good relative gain compared with other approaches.AZTI-Tecnalia, Grant/Award Number: C- 3368-00; Secretaría de Estado de Investigación, Desarrollo e Innovación, Grant/Award Number: DPI2015-65471; Ministry of Economy and Competitiveness of Spain, Grant/Award Number: DPI2015-64571-R; Business-UGR Foundation; Tecnalia compan

Characterization of the evolution of indigo blue by multispectral imaging

FEDER/Junta de Andalucia-Consejeria de Economia y Conocimiento, Grant/Award Number: A-HUM-164-UGR18; Ministry of Economy and Competitiveness of Spain, Grant/Award Number: DPI2015-64571-R; Spanish Ministry of Economy and Competitiveness, Grant/Award Number: Ramon y Cajal program; Ministry of Economy and Competitiveness; Junta de Andalucia; FEDER (European Regional Development Fund).Indigo blue is a natural organic color from the Indigofera tinctoria plant. It is insoluble in water and does not require mordant to fix the color; therefore, due to its nature it can be considered as a pigment or dye. The identification of indigo in cultural heritage is very difficult due to the complexity of its chemical formula and the changes that it can undergo in certain conservation and exposition conditions. For this reason, characterization of possible degradation, transformation, and reaction processes is essential for its identification. In this study, multispectral imaging has been used for the first time as noninvasive technique to characterize the aging state of indigo blue using samples aged in a controlled climate chamber under extreme conditions of humidity, temperature, and light. The technique has proven to be very promising for this application, together with classification techniques based on machine learning approaches.FEDER/Junta de Andalucia-Consejeria de Economia y Conocimiento A-HUM-164-UGR18Ministry of Economy and Competitiveness of Spain DPI2015-64571-RSpanish Ministry of Economy and CompetitivenessMinistry of Economy and CompetitivenessJunta de AndaluciaEuropean Commissio

Long-term effects of blue-blocking spectacle lenses on color perception

The use of blue-blocking filters is increasing in spectacle lens users. Despite the low absorption in the blue range, some users complain about these filters because they affect their color perception. In a pilot study we have evaluated how the long-term use of 8 different blue-blocking filters impact the color perception during more than 2 weeks on a group of 18 normal color vision observers, compared with a control group of 10 observers. The evaluation was done using the FM100, the Color Assessment and Diagnosis (CAD) and an achromatic point measurement. Our results show that there is a trend to worsen with the filters on.Junta de Andalucia A-TIC-050-UGR18Spanish Government FIS2017-89258-P RTI2018094738-B-I0

Spectral information to get beyond color in the analysis of water‑soluble varnish degradation

Spectral images were captured of paper samples varnished with two water-soluble materials: gum arabic and egg white. The samples were submitted to degradation processes that partially or totally eliminated the varnish from the substrate (water immersion and ageing). The spectral information was used to obtain average color data and to characterize the spatial and color inhomogeneity across pixels, showing that the pixel spectral data are critical for an accurate characterization of the degradation process of the varnishes. Since the varnishes typically become yellower with ageing, this study introduces two novel and simple-to-compute yellowness indices based on the spectral information, which are validated against a standard colorimetric index (ASTM-E313 2015). The potential uses of spectral information are demonstrated with several pieces of a real antique map sample by comparing the spectral information measured before and after cleaning the sample. To sum up, the main contributions of this study are the characterization of the spatial homogeneity through pixel-based spectral and color information and the proposal of spectral-based yellowing indices for two critical applications (ageing process follow-up and effect of cleaning), as demonstrated with synthetic and historical samples of varnished paper respectively.Spanish Ministry of Economy and Competitiveness, under research Grant DPI2015-64571-R. Spanish State Agency of Research (AEI) and the Ministry for Economy, Industry and Competitiveness (MIMECO) by means of the Grant Number FIS2017-89258-P with European Union FEDER (European Regional Development Funds) support

Combining transverse field detectors and color filter arrays to improve multispectral imaging systems

This work focuses on the improvement of a multispectral imaging sensor based on transverse field detectors (TFDs). We aimed to achieve a higher color and spectral accuracy in the estimation of spectral reflectances from sensor responses. Such an improvement was done by combining these recently developed silicon-based sensors with color filter arrays (CFAs). Consequently, we sacrificed the filter-less full spatial resolution property of TFDs to narrow down the spectrally broad sensitivities of these sensors.We designed and performed several experiments to test the influence of different design features on the estimation quality (type of sensor, tunability, interleaved polarization, use of CFAs, type of CFAs, number of shots), some of which are exclusive to TFDs.We compared systems that use a TFD with systems that use normal monochrome sensors, both combined with multispectral CFAs as well as common RGB filters present in commercial digital color cameras. Results showed that a system that combines TFDs and CFAs performs better than systems with the same type of multispectral CFA and other sensors, or even the same TFDs combined with different kinds of filters used in common imaging systems. We propose CFA+TFD-based systems with one or two shots, depending on the possibility of using longer capturing times or not. Improved TFD systems thus emerge as an interesting possibility for multispectral acquisition, which overcomes the limited accuracy found in previous studies.Spanish Ministry of Economy and Competitiveness through the research project DPI2011-2320

Unmixing and pigment identification using visible and short-wavelength infrared: Reflectance vs logarithm reflectance hyperspaces

Hyperspectral imaging has recently consolidated as a useful technique for pigment mapping and identification, although it is commonly supported by additional non-invasive analytical methods. Since it is relatively rare to find pure pigments in aged paintings, spectral unmixing can be helpful in facilitating pigment identification if suitable mixing models and endmember extraction procedures are chosen. In this study, a subtractive mixing model is assumed, and two approaches are compared for endmember extraction: one based on a linear mixture model, and the other, nonlinear and Deep-Learning based. Two spectral hyperspaces are used: the spectral reflectance (R hyperspace) and the -log(R) hyperspace, for which the subtractive model becomes additive. The performance of unmixing is evaluated by the similarity of the estimated reflectance to the measured data, and pigment identification accuracy. Two spectral ranges (400 to 1000 nm and 900 to 1700 nm) and two objects (a laboratory sample and an aged painting, both on copper) are tested. The main conclusion is that unmixing in the -log(R) hyperspace with a linear mixing model is better than for the non-linear model in R hyperspace, and that pigment identification is generally better in R hyperspace, improving by merging the results in both spectral ranges.MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe” [grant number PID2021-124446NB-100]Ministry of Universities (Spain) [grant number FPU2020-05532