Evaluation and optimal design of spectral sensitivities for digital color imaging

The quality of an image captured by color imaging system primarily depends on three factors: sensor spectral sensitivity, illumination and scene. While illumination is very important to be known, the sensitivity characteristics is critical to the success of imaging applications, and is necessary to be optimally designed under practical constraints. The ultimate image quality is judged subjectively by human visual system. This dissertation addresses the evaluation and optimal design of spectral sensitivity functions for digital color imaging devices. Color imaging fundamentals and device characterization are discussed in the first place. For the evaluation of spectral sensitivity functions, this dissertation concentrates on the consideration of imaging noise characteristics. Both signal-independent and signal-dependent noises form an imaging noise model and noises will be propagated while signal is processed. A new colorimetric quality metric, unified measure of goodness (UMG), which addresses color accuracy and noise performance simultaneously, is introduced and compared with other available quality metrics. Through comparison, UMG is designated as a primary evaluation metric. On the optimal design of spectral sensitivity functions, three generic approaches, optimization through enumeration evaluation, optimization of parameterized functions, and optimization of additional channel, are analyzed in the case of the filter fabrication process is unknown. Otherwise a hierarchical design approach is introduced, which emphasizes the use of the primary metric but the initial optimization results are refined through the application of multiple secondary metrics. Finally the validity of UMG as a primary metric and the hierarchical approach are experimentally tested and verified

Colorimetric and spectral analysis of rock art by means of the characterization of digital sensors

Tesis por compendio[ES] Las labores de documentación de arte rupestre son arduas y delicadas, donde el color desempeña un papel fundamental, proporcionando información vital a nivel descriptivo, técnico y cuantitativo . Tradicionalmente los métodos de documentación en arqueología quedaban restringidos a procedimientos estrictamente subjetivos. Sin embargo, esta metodología conlleva limitaciones prácticas y técnicas, afectando a los resultados obtenidos en la determinación del color. El empleo combinado de técnicas geomáticas, como la fotogrametría o el láser escáner, junto con técnicas de procesamiento de imágenes digitales, ha supuesto un notable avance. El problema es que, aunque las imágenes digitales permiten capturar el color de forma rápida, sencilla, y no invasiva, los datos RGB registrados por la cámara no tienen un sentido colorimétrico riguroso. Se requiere la aplicación de un proceso riguroso de tranformación que permita obtener datos fidedignos del color a través de imágenes digitales. En esta tesis se propone una solución científica novedosa y de vanguardia, en la que se persigue integrar el análisis espectrofotométrico y colorimétrico como complemento a técnicas fotogramétricas que permitan una mejora en la identificación del color y representación de pigmentos con máxima fiabilidad en levantamientos, modelos y reconstrucciones tridimensionales (3D). La metodología propuesta se basa en la caracterización colorimétrica de sensores digitales, que es de novel aplicación en pinturas rupestres. La caracterización pretende obtener las ecuaciones de transformación entre los datos de color registrados por la cámara, dependientes del dispositivo, y espacios de color independientes, de base física, como los establecidos por la Commission Internationale de l'Éclairage (CIE). Para el tratamiento de datos colorimétricos y espectrales se requiere disponer de un software de características técnicas muy específicas. Aunque existe software comercial, lo cierto es que realizan por separado el tratamiento digital de imágenes y las operaciones colorimétricas. No existe software que integre ambas, ni que además permita llevar a cabo la caracterización. Como aspecto fundamental, presentamos en esta tesis el software propio desarrollado, denominado pyColourimetry, siguiendo las recomendaciones publicadas por la CIE, de código abierto, y adaptado al flujo metodológico propuesto, de modo que facilite la independencia y el progreso científico sin ataduras comerciales, permitiendo el tratamiento de datos colorimétricos y espectrales, y confiriendo al usuario pleno control del proceso y la gestión de los datos obtenidos. Adicinalmente, en este estudio se expone el análisis de los principales factores que afectan a la caracterización tales como el sensor empleado, los parámetros de la cámara durante la toma, la iluminación, el modelo de regresión, y el conjunto de datos empleados como entrenamiento del modelo. Se ha aplicado un modelo de regresión basado en procesos Gaussianos, y se ha comparado con los resultados obtenidos mediante polinomios. También presentamos un nuevo esquema de trabajo que permite la selección automática de muestras de color, adaptado al rango cromático de la escena, que se ha denominado P-ASK, basado en el algoritmo de clasificación K-means. Los resultados obtenidos en esta tesis demuestran que el proceso metodológico de caracterización propuesto es altamente aplicable en tareas de documentación y preservación del patrimonio cultural en general, y en arte rupestre en particular. Se trata de una metodología de bajo coste, no invasiva, que permite obtener el registro colorimétrico de escenas completas. Una vez caracterizada, una cámara digital convencional puede emplearse para la determinación del color de forma rigurosa, simulando un colorímetro, lo que permitirá trabajar en un espacio de color de base física, independiente del dispositivo y comparable con[CA] Les tasques de documentació gràfica d'art rupestre són àrdues i delicades, on el color compleix un paper fonamental, proporcionant informació vital a nivell descriptiu, t\`ecnic i quantitatiu.Tradicionalment els mètodes de documentació en arqueologia quedaven restringits a procediments estrictament subjectius, comportant limitacions pràctiques i tècniques, afectant els resultats obtinguts en la determinació de la color. L'ús combinat de tècniques geomàtiques, com la fotogrametria o el làser escàner, juntament amb tècniques de processament i realç d'imatges digitals, ha suposat un notable avanç. Tot i que les imatges digitals permeten capturar el color de forma ràpida, senzilla, i no invasiva, les dades RGB proporcionades per la càmera no tenen un sentit colorimètric rigorós. Es requereix l'aplicació d'un procés rigorós de transformació que permeti obtenir dades fidedignes de la color a través d'imatges digitals. En aquesta tesi es proposa una solució científica innovadora i d'avantguarda, en la qual es persegueix integrar l'anàlisi espectrofotomètric i colorimètric com a complement a tècniques fotogramètriques que permetin una millora en la identificació de la color i representació de pigments amb màxima fiabilitat en aixecaments, models i reconstruccions tridimensionals 3D. La metodologia proposada es basa en la caracterització colorimètrica de sensors digitals, que és de novell aplicació en pintures rupestres. La caracterització pretén obtenir les equacions de transformació entre les dades de color registrats per la càmera, dependents d'el dispositiu, i espais de color independents, de base física, com els establerts per la Commission Internationale de l'Éclairage (CIE). Per al tractament de dades colorimètriques i espectrals de forma rigorosa es requereix disposar d'un programari de característiques tècniques molt específiques. Encara que hi ha programari comercial, fan per separat el tractament digital d'imatges i les operacions colorimètriques. No hi ha programari que integri totes dues, ni que permeti dur a terme la caracterització. Com a aspecte addicional i fonamental, vam presentar el programari propi que s'ha desenvolupat, denominat pyColourimetry, segons les recomanacions publicades per la CIE, de codi obert, i adaptat al flux metodológic proposat, de manera que faciliti la independència i el progrés científic sense lligams comercials, permetent el tractament de dades colorimètriques i espectrals, i conferint a l'usuari ple control del procés i la gestió de les dades obtingudes. A més, s'exposa l'anàlisi dels principals factors que afecten la caracterització tals com el sensor emprat, els paràmetres de la càmera durant la presa, il¿luminació, el model de regressió, i el conjunt de dades emprades com a entrenament d'el model. S'ha aplicat un model de regressió basat en processos Gaussians, i s'han comparat els resultats obtinguts mitjançant polinomis. També vam presentar un nou esquema de treball que permet la selecció automàtica de mostres de color, adaptat a la franja cromàtica de l'escena, que s'ha anomenat P-ASK, basat en l'algoritme de classificació K-means. Els resultats obtinguts en aquesta tesi demostren que el procés metodològic de caracterització proposat és altament aplicable en tasques de documentació i preservació de el patrimoni cultural en general, i en art rupestre en particular. Es tracta d'una metodologia de baix cost, no invasiva, que permet obtenir el registre colorimètric d'escenes completes. Un cop caracteritzada, una càmera digital convencional pot emprar-se per a la determinació de la color de forma rigorosa, simulant un colorímetre, el que permetrà treballar en un espai de color de base física, independent d'el dispositiu i comparable amb dades obtingudes mitjançant altres càmeres que tambè estiguin caracteritzades.[EN] Cultural heritage documentation and preservation is an arduous and delicate task in which color plays a fundamental role. The correct determination of color provides vital information on a descriptive, technical and quantitative level. Classical color documentation methods in archaeology were usually restricted to strictly subjective procedures. However, this methodology has practical and technical limitations, affecting the results obtained in the determination of color. Nowadays, it is frequent to support classical methods with geomatics techniques, such as photogrammetry or laser scanning, together with digital image processing. Although digital images allow color to be captured quickly, easily, and in a non-invasive way, the RGB data provided by the camera does not itself have a rigorous colorimetric sense. Therefore, a rigorous transformation process to obtain reliable color data from digital images is required. This thesis proposes a novel technical solution, in which the integration of spectrophotometric and colorimetric analysis is intended as a complement to photogrammetric techniques that allow an improvement in color identification and representation of pigments with maximum reliability in 3D surveys, models and reconstructions. The proposed methodology is based on the colorimetric characterization of digital sensors, which is of novel application in cave paintings. The characterization aims to obtain the transformation equations between the device-dependent color data recorded by the camera and the independent, physically-based color spaces, such as those established by the Commission Internationale de l'Éclairage (CIE). The rigorous processing of color and spectral data requires software packages with specific colorimetric functionalities. Although there are different commercial software options, they do not integrate the digital image processing and colorimetric computations together. And more importantly, they do not allow the camera characterization to be carried out. Therefore, as a key aspect in this thesis is our in-house pyColourimetry software that was developed and tested taking into account the recommendations published by the CIE. pyColourimetry is an open-source code, independent without commercial ties; it allows the treatment of colorimetric and spectral data and the digital image processing, and gives full control of the characterization process and the management of the obtained data to the user. On the other hand, this study presents a further analysis of the main factors affecting the characterization, such as the camera built-in sensor, the camera parameters, the illuminant, the regression model, and the data set used for model training. For computing the transformation equations, the literature recommends the use of polynomial equations as a regression model. Thus, polynomial models are considered as a starting point in this thesis. Additionally, a regression model based on Gaussian processes has been applied, and the results obtained by means of polynomials have been compared. Also, a new working scheme was reported which allows the automatic selection of color samples, adapted to the chromatic range of the scene. This scheme is called P-ASK, based on the K-means classification algorithm. The results achieved in this thesis show that the proposed framework for camera characterization is highly applicable in documentation and conservation tasks in general cultural heritage applications, and particularly in rock art painting. It is a low-cost and non-invasive methodology that allows for the colorimetric recording from complete image scenes. Once characterized, a conventional digital camera can be used for rigorous color determination, simulating a colorimeter. Thus, it is possible to work in a physical color space, independent of the device used, and comparable with data obtained from other cameras that are also characterized.Thanks to the Universitat Politècnica de València for the FPI scholarshipMolada Tebar, A. (2020). Colorimetric and spectral analysis of rock art by means of the characterization of digital sensors [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/160386TESISCompendi

Expanding Dimensionality in Cinema Color: Impacting Observer Metamerism through Multiprimary Display

Television and cinema display are both trending towards greater ranges and saturation of reproduced colors made possible by near-monochromatic RGB illumination technologies. Through current broadcast and digital cinema standards work, system designs employing laser light sources, narrow-band LED, quantum dots and others are being actively endorsed in promotion of Wide Color Gamut (WCG). Despite artistic benefits brought to creative content producers, spectrally selective excitations of naturally different human color response functions exacerbate variability of observer experience. An exaggerated variation in color-sensing is explicitly counter to the exhaustive controls and calibrations employed in modern motion picture pipelines. Further, singular standard observer summaries of human color vision such as found in the CIE’s 1931 and 1964 color matching functions and used extensively in motion picture color management are deficient in recognizing expected human vision variability. Many researchers have confirmed the magnitude of observer metamerism in color matching in both uniform colors and imagery but few have shown explicit color management with an aim of minimized difference in observer perception variability. This research shows that not only can observer metamerism influences be quantitatively predicted and confirmed psychophysically but that intentionally engineered multiprimary displays employing more than three primaries can offer increased color gamut with drastically improved consistency of experience. To this end, a seven-channel prototype display has been constructed based on observer metamerism models and color difference indices derived from the latest color vision demographic research. This display has been further proven in forced-choice paired comparison tests to deliver superior color matching to reference stimuli versus both contemporary standard RGB cinema projection and recently ratified standard laser projection across a large population of color-normal observers

Developing an imaging bi-spectrometer for fluorescent materials

Fluorescent effects have been observed for thousands of years. Stokes, in 1852, began the science of fluorescence culminating in his law of fluorescence, which explained that fluorescence emission occurs at longer wavelengths than the excitation wavelength. This phenomenon is observed extensively in the art world. Daylight fluorescent colors known as Day-Glo have become an artistic medium since the 1960s. Modern artists exploit these saturated and brilliant colors to glitter their painting. Multispectral imaging as a noninvasive technique has been used for archiving by museums and cultural-heritage institutions for about a decade. The complex fluorescence phenomenon has been often ignored in the multispectral projects. The ignored fluorescence results in errors in digital imaging of artwork containing fluorescent colors. The illuminant-dependency of the fluorescence radiance makes the fluorescence colorimetry and consequently spectral imaging more complex. In this dissertation an abridged imaging bi-spectrometer for artwork containing both fluorescent and non-fluorescent colors was developed. The method developed included two stages of reconstruction of the spectral reflected radiance factor and prediction of the fluorescent radiance factor. The estimation of the reflected radiance factor as a light source independent component was achieved by imaging with a series of short-wavelength cutoff filters placed in the illumination path. The fluorescent radiance factor, a light source dependent component, was estimated based on a proposed model, the abridged two-monochromator method. The abridged two-monochromator method was developed for reconstructing the bi-spectral matrix of a fluorescent color based on a calibrated UV-fluorescence imaging. In this way, one could predict the fluorescence radiance factor under any desired illuminant and consequently a better color evaluation and rendering could be obtained. Furthermore, this method easily fitted in a general system for spectral imaging of paintings containing both fluorescent and non-fluorescent colors. The abridged two-monochromator method could predict fluorescent radiance factor of a fluorescent color via prediction of the true emission and the number of absorbed quanta by a fluorescing specimen for a given viewing light source. The superiority of the abridged fluorescence spectral imaging to the traditional spectral and colorimetric imaging for a few light sources was confirmed using fluorescent and non-fluorescent targets. Additionally, an exploratory visual experiment using a paired-comparison method was performed to evaluate the performance of the abridged fluorescence spectral imaging in comparison to the traditional spectral and colorimetric imaging for rendering images of a reference painting. The abridged fluorescence spectral imaging had better performance than traditional spectral and colorimetric imaging in rendering images for daylight

Prototype software for colorant formulation using Gamblin conservation colors

When selecting pigments from a large set for restorative inpainting, it can often be challenging to create a mixture that will provide an exact match to the original artwork under a range of viewing and illumination conditions. In this research, a prototype computer program was developed that will aid the user by providing a color match and paint recipe that exhibits minimal metamerism when compared to the original artwork. The Gamblin Conservation Colors, a set of 43 colorants specially formulated for inpainting, were characterized in terms of their optical properties, absorption and scattering, according to Kubelka-Munk turbid media theory. Formulations were made using traditional spectrophotometric measurements and image-based measurements. The multispectral imaging system consisted of a trichromatic CFA camera coupled with two absorption filters; spectral reflectance data for each pixel location was estimated with a transformation based on calibration target images. Three targets were used for testing formulation accuracy: a target consisting of mixtures of Gamblin Conservation Colors, and two oil paintings. Pigment selection was reasonably successful, and good predictions resulted from both measurement techniques, but for more complex tasks such as pigment identification, a more rigorous colorant characterization approach may be needed. Predictions from image-based measurements were generally less accurate, and improvements in the camera model would likely remedy this. It is expected that this software will be of assistance to conservators by simplifying the process of selecting from a large set of available pigments, as well as reducing the possibility of damage to painted surfaces in cases where direct measurements are impractical. The open source nature of the software provides the opportunity for changes and addition of features in the future

Spectral Reconstruction Using an Iteratively Reweighted Regulated Model from Two Illumination Camera Responses

An improved spectral reflectance estimation method was developed to transform captured RGB images to spectral reflectance. The novelty of our method is an iteratively reweighted regulated model that combines polynomial expansion signals, which was developed for spectral reflectance estimation, and a cross-polarized imaging system, which is used to eliminate glare and specular highlights. Two RGB images are captured under two illumination conditions. The method was tested using ColorChecker charts. The results demonstrate that the proposed method could make a significant improvement of the accuracy in both spectral and colorimetric: it can achieve 23.8% improved accuracy in mean CIEDE2000 color difference, while it achieves 24.6% improved accuracy in RMS error compared with classic regularized least squares (RLS) method. The proposed method is sufficiently accurate in predicting the spectral properties and their performance within an acceptable range, i.e., typical customer tolerance of less than 3 DE units in the graphic arts industry

Specialized Color Targets for Spectral Reflectance Reconstruction of Magnified Images

Digital images are used almost exclusively instead of film to capture visual information across many scientific fields. The colorimetric color representation within these digital images can be relayed from the digital counts produced by the camera with the use of a known color target. In image capture of magnified images, there is currently no reliable color target that can be used at multiple magnifications and give the user a solid understanding of the color ground truth within those images. The first part of this dissertation included the design, fabrication, and testing of a color target produced with optical interference coated microlenses for use in an off-axis illumination, compound microscope. An ideal target was designed to increase the color gamut for colorimetric imaging and provide the necessary Block Dye spectral reflectance profiles across the visible spectrum to reduce the number of color patches necessary for multiple filter imaging systems that rely on statistical models for spectral reflectance reconstruction. There are other scientific disciplines that can benefit from a specialized color target to determine the color ground truth in their magnified images and perform spectral estimation. Not every discipline has the luxury of having a multi-filter imaging system. The second part of this dissertation developed two unique ways of using an interference coated color mirror target: one that relies on multiple light-source angles, and one that leverages a dynamic color change with time. The source multi-angle technique would be used for the microelectronic discipline where the reconstructed spectral reflectance would be used to determine a dielectric film thickness on a silicon substrate, and the time varying technique would be used for a biomedical example to determine the thickness of human tear film

Colorimetric sensor arrays: development and application to art conservation

This dissertation explores the optimization, development and application of cost-effective, compact colorimetric arrays, focusing specially on their use in the cultural heritage and art conservation communities. One of Society's most important cultural responsibilities is the preservation of the past for the future. The surest way to protect a cultural heritage material from damage is to control the environment in which it is displayed. While there are a number of monitoring techniques available, there are many limitations (e.g., cost, portability) or problems (e.g., lack of desired sensitivity, time consuming, need for laboratory personnel). The Suslick Group believes we have a technology that could become a valuable tool for museum professionals. Chapter 1 provides a comprehensive review of destructive air pollutants in conservation environments and the existing methods to monitor and quantify them. This chapter serves to lay the groundwork for why low-cost, convenient colorimetric array technology is crucial. Chapter 2 optimizes the performance of portable colorimetric arrays, establishing them as an invaluable technology for museums and art conservationists. Chapter 3 provides a quantitative, side-by-side comparison of two standard colorimetric detection methods—RGB imaging and full reflectance spectrophotometry—in order to further improve the performance of these arrays. Finally, Chapter 4 applies the concepts learned in this work and in past work from the Suslick group, to extend with new sensor chemistry and detection techniques, our already sensitive optoelectronic nose technology into one capable of detecting museum pollutants in a variety of environments. In addition, this chapter addresses a study that uses this technology, through an exciting collaboration with the Getty Conservation Institute and the Walt Disney Animation Research Library, to make trial experiments in the monitoring of artwork from the Walt Disney Animation Research Library exhibition in Beijing, "Drawn from Life: the Art of Disney Animation Studios", in order to monitor pollutant exposure both during shipping and during exhibition

Hyperspectral colour imaging and spectrophotometric instrumentation

The trichromatic nature of commercial photography is strictly connected with the nature of human colour vision, although the characteristics of usual colour imaging devices are quite different from the human visual system. The increase in the number of colour channels for spectral (either multispectral or hyperspectral) imaging is an active field of research with many potential applications in different fields. Each element of the captured scene is specified in the spectral image by the spectral reflectance factor. This measurement is independent of the particular illumination of the scene and allows the colorimetric computation in a device-independent colour space for any chosen illuminant and any observer. This thesis describes the project and construction of a compact spectrophotometric camera, which can be used in both portable and in-situ applications. The compactness is made possible by a suitable image spectral scanning based on an Induced Transmission Filter (ITF). This filter is made by a set of thin-film coatings of dielectric materials with high and low refraction index, whose shape like a wedge induces a wavelength selective transmittance, continuously variable along one direction and uniform in the perpendicular direction. Such a filter, classified as Linearly Variable Filter (LVF), operates continuously from 430nm to 940nm and allows hyperspectral imaging. In traditional scanners the whole apparatus is moved along a path as long as the scene, whereas in this instrument the camera body is still and the LVF is the only moving part. The sequence of operations for wavelength and radiometric calibrations are discussed. The expected acquisition times and number of images as a function of the spectral sampling step are considered. The resulting properties make the instrument easy to use and with short acquisition times. Moreover, overviews of the historic evolution of colour vision fundamentals, colour spaces and spectral imaging technology are given for introducing the reader to the essential concepts useful for the understanding of the text

Realistic visualisation of cultural heritage objects

This research investigation used digital photography in a hemispherical dome, enabling a set of 64 photographic images of an object to be captured in perfect pixel register, with each image illuminated from a different direction. This representation turns out to be much richer than a single 2D image, because it contains information at each point about both the 3D shape of the surface (gradient and local curvature) and the directionality of reflectance (gloss and specularity). Thereby it enables not only interactive visualisation through viewer software, giving the illusion of 3D, but also the reconstruction of an actual 3D surface and highly realistic rendering of a wide range of materials. The following seven outcomes of the research are claimed as novel and therefore as representing contributions to knowledge in the field: A method for determining the geometry of an illumination dome; An adaptive method for finding surface normals by bounded regression; Generating 3D surfaces from photometric stereo; Relationship between surface normals and specular angles; Modelling surface specularity by a modified Lorentzian function; Determining the optimal wavelengths of colour laser scanners; Characterising colour devices by synthetic reflectance spectra