Bayesian Methods for Radiometric Calibration in Motion Picture Encoding Workflows

A method for estimating the Camera Response Function (CRF) of an electronic motion picture camera is presented in this work. The accurate estimation of the CRF allows for proper encoding of camera exposures into motion picture post-production workflows, like the Academy Color Encoding Specification (ACES), this being a necessary step to correctly combine images from different capture sources into one cohesive final production and minimize non-creative manual adjustments. Although there are well known standard CRFs implemented in typical video camera workflows, motion picture workflows and newer High Dynamic Range (HDR) imaging workflows have introduced new standard CRFs as well as custom and proprietary CRFs that need to be known for proper post-production encoding of the camera footage. Current methods to estimate this function rely on the use of measurement charts, using multiple static images taken under different exposures or lighting conditions, or assume a simplistic model of the function’s shape. All these methods become problematic and tough to fit into motion picture production and post-production workflows where the use of test charts and varying camera or scene setups becomes impractical and where a method based solely on camera footage, comprised of a single image or a series of images, would be advantageous. This work presents a methodology initially based on the work of Lin, Gu, Yamazaki and Shum that takes into account edge color mixtures in an image or image sequence, that are affected by the non-linearity introduced by a CRF. In addition, a novel feature based on image noise is introduced to overcome some of the limitations of edge color mixtures. These features provide information that is included in the likelihood probability distribution in a Bayesian framework to estimate the CRF as the expected value of a posterior probability distribution, which is itself approximated by a Markov Chain Monte Carlo (MCMC) sampling algorithm. This allows for a more complete description of the CRF over methods like Maximum Likelihood (ML) and Maximum A Posteriori (MAP). The CRF function is modeled by Principal Component Analysis (PCA) of the Database of Response Functions (DoRF) compiled by Grossberg and Nayar, and the prior probability distribution is modeled by a Gaussian Mixture Model (GMM) of the PCA coefficients for the responses in the DoRF. CRF estimation results are presented for an ARRI electronic motion picture camera, showing the improved estimation accuracy and practicality of this method over previous methods for motion picture post-production workflows

Calibrating the elements of a multispectral imaging system

We describe a method to calibrate the elements of a multispectral system aimed at skylight imaging, which consists of a monochrome charge-coupled device (CCD) camera and a liquid-crystal tunable filter (LCTF). We demonstrate how to calibrate these two devices in order to build a multispectral camera that can obtain spectroradiometric measurements of skylight. Spectral characterizations of the tunable filter and the camera are presented together with a complete study into correcting temporal and spatial noise, which is of key importance in CCDs. We describe all the necessary steps to undertake this work and all the additional instrumentation that must be used to calibrate the radiometric devices correctly. We show how this complete study of our multispectral system allows us to use it as an accurate, high resolution spectroradiometer.This work was financed by the Spanish Red Temática “CIENCIA Y TECNOLOGÍA DEL COLOR” (FIS2005-25312-E), the Spanish Ministry of Education and Science, and the European Fund for Regional Development (FEDER) through grant FIS2007-60736. We thank our English colleague A. L. Tate for revising our English text.Peer reviewe

Natural images from the birthplace of the human eye

Here we introduce a database of calibrated natural images publicly available through an easy-to-use web interface. Using a Nikon D70 digital SLR camera, we acquired about 5000 six-megapixel images of Okavango Delta of Botswana, a tropical savanna habitat similar to where the human eye is thought to have evolved. Some sequences of images were captured unsystematically while following a baboon troop, while others were designed to vary a single parameter such as aperture, object distance, time of day or position on the horizon. Images are available in the raw RGB format and in grayscale. Images are also available in units relevant to the physiology of human cone photoreceptors, where pixel values represent the expected number of photoisomerizations per second for cones sensitive to long (L), medium (M) and short (S) wavelengths. This database is distributed under a Creative Commons Attribution-Noncommercial Unported license to facilitate research in computer vision, psychophysics of perception, and visual neuroscience.Comment: Submitted to PLoS ON

An instrument for the characterization and calibration of optical sensors

This paper presents the development of a hardware/software system for the characterization of the electronic response of optical (camera) sensors such as matrix and linear color and monochrome Charge Coupled Device (CCD) or Complementary Metal Oxide Semiconductor (CMOS). The electronic response of a sensor is required for inspection purposes. It also allows the design and calibration of the integrating device to achieve the desired performance. The proposed instrument equipment fulfills the most recent European Machine Vision Association (EMVA) 1288 standard ver. 3.1: The spatial non uniformity of the illumination ΔE must be under 3%, and the sensor must achieve an f-number of 8.0 concerning the light source. The following main innovations have achieved this: An Ulbricht sphere providing a uniform light distribution (irradiation) of 99.54%; an innovative illuminator with proper positioning of color Light Emitting Diodes (LEDs) and control electronics; and a flexible C# program to analyze the sensor parameters, namely Quantum Efficiency, Overall System Gain, Temporal Dark Noise, Dark Signal Non Uniformity (DSNU1288), Photo Response Non-Uniformity (PRNU1288), Maximum achievable Signal to Noise Ratio (SNRmax), Absolute sensitivity threshold, Saturation Capacity, Dynamic Range, and Dark Current. This new instrument has allowed a camera manufacturer to design, integrate, and inspect numerous devices and camera models (Necta, Celera, and Aria)

Evaluation and optimal design of spectral sensitivities for digital color imaging

The quality of an image captured by color imaging system primarily depends on three factors: sensor spectral sensitivity, illumination and scene. While illumination is very important to be known, the sensitivity characteristics is critical to the success of imaging applications, and is necessary to be optimally designed under practical constraints. The ultimate image quality is judged subjectively by human visual system. This dissertation addresses the evaluation and optimal design of spectral sensitivity functions for digital color imaging devices. Color imaging fundamentals and device characterization are discussed in the first place. For the evaluation of spectral sensitivity functions, this dissertation concentrates on the consideration of imaging noise characteristics. Both signal-independent and signal-dependent noises form an imaging noise model and noises will be propagated while signal is processed. A new colorimetric quality metric, unified measure of goodness (UMG), which addresses color accuracy and noise performance simultaneously, is introduced and compared with other available quality metrics. Through comparison, UMG is designated as a primary evaluation metric. On the optimal design of spectral sensitivity functions, three generic approaches, optimization through enumeration evaluation, optimization of parameterized functions, and optimization of additional channel, are analyzed in the case of the filter fabrication process is unknown. Otherwise a hierarchical design approach is introduced, which emphasizes the use of the primary metric but the initial optimization results are refined through the application of multiple secondary metrics. Finally the validity of UMG as a primary metric and the hierarchical approach are experimentally tested and verified

From Pixels to Physics: Probabilistic Color De-Rendering

Consumer digital cameras use tone-mapping to produce compact, narrow-gamut images that are nonetheless visually pleasing. In doing so, they discard or distort substantial radiometric signal that could otherwise be used for computer vision. Existing methods attempt to undo these effects through deterministic maps that de-render the reported narrow-gamut colors back to their original wide-gamut sensor measurements. Deterministic approaches are unreliable, however, because the reverse narrow-to-wide mapping is one-to-many and has inherent uncertainty. Our solution is to use probabilistic maps, providing uncertainty estimates useful to many applications. We use a non-parametric Bayesian regression technique - local Gaussian process regression - to learn for each pixel's narrow-gamut color a probability distribution over the scene colors that could have created it. Using a variety of consumer cameras we show that these distributions, once learned from training data, are effective in simple probabilistic adaptations of two popular applications: multi-exposure imaging and photometric stereo. Our results on these applications are better than those of corresponding deterministic approaches, especially for saturated and out-of-gamut colors.Engineering and Applied Science

Digital Color Imaging

This paper surveys current technology and research in the area of digital color imaging. In order to establish the background and lay down terminology, fundamental concepts of color perception and measurement are first presented us-ing vector-space notation and terminology. Present-day color recording and reproduction systems are reviewed along with the common mathematical models used for representing these devices. Algorithms for processing color images for display and communication are surveyed, and a forecast of research trends is attempted. An extensive bibliography is provided

Colorimetric and spectral analysis of rock art by means of the characterization of digital sensors

Tesis por compendio[ES] Las labores de documentación de arte rupestre son arduas y delicadas, donde el color desempeña un papel fundamental, proporcionando información vital a nivel descriptivo, técnico y cuantitativo . Tradicionalmente los métodos de documentación en arqueología quedaban restringidos a procedimientos estrictamente subjetivos. Sin embargo, esta metodología conlleva limitaciones prácticas y técnicas, afectando a los resultados obtenidos en la determinación del color. El empleo combinado de técnicas geomáticas, como la fotogrametría o el láser escáner, junto con técnicas de procesamiento de imágenes digitales, ha supuesto un notable avance. El problema es que, aunque las imágenes digitales permiten capturar el color de forma rápida, sencilla, y no invasiva, los datos RGB registrados por la cámara no tienen un sentido colorimétrico riguroso. Se requiere la aplicación de un proceso riguroso de tranformación que permita obtener datos fidedignos del color a través de imágenes digitales. En esta tesis se propone una solución científica novedosa y de vanguardia, en la que se persigue integrar el análisis espectrofotométrico y colorimétrico como complemento a técnicas fotogramétricas que permitan una mejora en la identificación del color y representación de pigmentos con máxima fiabilidad en levantamientos, modelos y reconstrucciones tridimensionales (3D). La metodología propuesta se basa en la caracterización colorimétrica de sensores digitales, que es de novel aplicación en pinturas rupestres. La caracterización pretende obtener las ecuaciones de transformación entre los datos de color registrados por la cámara, dependientes del dispositivo, y espacios de color independientes, de base física, como los establecidos por la Commission Internationale de l'Éclairage (CIE). Para el tratamiento de datos colorimétricos y espectrales se requiere disponer de un software de características técnicas muy específicas. Aunque existe software comercial, lo cierto es que realizan por separado el tratamiento digital de imágenes y las operaciones colorimétricas. No existe software que integre ambas, ni que además permita llevar a cabo la caracterización. Como aspecto fundamental, presentamos en esta tesis el software propio desarrollado, denominado pyColourimetry, siguiendo las recomendaciones publicadas por la CIE, de código abierto, y adaptado al flujo metodológico propuesto, de modo que facilite la independencia y el progreso científico sin ataduras comerciales, permitiendo el tratamiento de datos colorimétricos y espectrales, y confiriendo al usuario pleno control del proceso y la gestión de los datos obtenidos. Adicinalmente, en este estudio se expone el análisis de los principales factores que afectan a la caracterización tales como el sensor empleado, los parámetros de la cámara durante la toma, la iluminación, el modelo de regresión, y el conjunto de datos empleados como entrenamiento del modelo. Se ha aplicado un modelo de regresión basado en procesos Gaussianos, y se ha comparado con los resultados obtenidos mediante polinomios. También presentamos un nuevo esquema de trabajo que permite la selección automática de muestras de color, adaptado al rango cromático de la escena, que se ha denominado P-ASK, basado en el algoritmo de clasificación K-means. Los resultados obtenidos en esta tesis demuestran que el proceso metodológico de caracterización propuesto es altamente aplicable en tareas de documentación y preservación del patrimonio cultural en general, y en arte rupestre en particular. Se trata de una metodología de bajo coste, no invasiva, que permite obtener el registro colorimétrico de escenas completas. Una vez caracterizada, una cámara digital convencional puede emplearse para la determinación del color de forma rigurosa, simulando un colorímetro, lo que permitirá trabajar en un espacio de color de base física, independiente del dispositivo y comparable con[CA] Les tasques de documentació gràfica d'art rupestre són àrdues i delicades, on el color compleix un paper fonamental, proporcionant informació vital a nivell descriptiu, t\`ecnic i quantitatiu.Tradicionalment els mètodes de documentació en arqueologia quedaven restringits a procediments estrictament subjectius, comportant limitacions pràctiques i tècniques, afectant els resultats obtinguts en la determinació de la color. L'ús combinat de tècniques geomàtiques, com la fotogrametria o el làser escàner, juntament amb tècniques de processament i realç d'imatges digitals, ha suposat un notable avanç. Tot i que les imatges digitals permeten capturar el color de forma ràpida, senzilla, i no invasiva, les dades RGB proporcionades per la càmera no tenen un sentit colorimètric rigorós. Es requereix l'aplicació d'un procés rigorós de transformació que permeti obtenir dades fidedignes de la color a través d'imatges digitals. En aquesta tesi es proposa una solució científica innovadora i d'avantguarda, en la qual es persegueix integrar l'anàlisi espectrofotomètric i colorimètric com a complement a tècniques fotogramètriques que permetin una millora en la identificació de la color i representació de pigments amb màxima fiabilitat en aixecaments, models i reconstruccions tridimensionals 3D. La metodologia proposada es basa en la caracterització colorimètrica de sensors digitals, que és de novell aplicació en pintures rupestres. La caracterització pretén obtenir les equacions de transformació entre les dades de color registrats per la càmera, dependents d'el dispositiu, i espais de color independents, de base física, com els establerts per la Commission Internationale de l'Éclairage (CIE). Per al tractament de dades colorimètriques i espectrals de forma rigorosa es requereix disposar d'un programari de característiques tècniques molt específiques. Encara que hi ha programari comercial, fan per separat el tractament digital d'imatges i les operacions colorimètriques. No hi ha programari que integri totes dues, ni que permeti dur a terme la caracterització. Com a aspecte addicional i fonamental, vam presentar el programari propi que s'ha desenvolupat, denominat pyColourimetry, segons les recomanacions publicades per la CIE, de codi obert, i adaptat al flux metodológic proposat, de manera que faciliti la independència i el progrés científic sense lligams comercials, permetent el tractament de dades colorimètriques i espectrals, i conferint a l'usuari ple control del procés i la gestió de les dades obtingudes. A més, s'exposa l'anàlisi dels principals factors que afecten la caracterització tals com el sensor emprat, els paràmetres de la càmera durant la presa, il¿luminació, el model de regressió, i el conjunt de dades emprades com a entrenament d'el model. S'ha aplicat un model de regressió basat en processos Gaussians, i s'han comparat els resultats obtinguts mitjançant polinomis. També vam presentar un nou esquema de treball que permet la selecció automàtica de mostres de color, adaptat a la franja cromàtica de l'escena, que s'ha anomenat P-ASK, basat en l'algoritme de classificació K-means. Els resultats obtinguts en aquesta tesi demostren que el procés metodològic de caracterització proposat és altament aplicable en tasques de documentació i preservació de el patrimoni cultural en general, i en art rupestre en particular. Es tracta d'una metodologia de baix cost, no invasiva, que permet obtenir el registre colorimètric d'escenes completes. Un cop caracteritzada, una càmera digital convencional pot emprar-se per a la determinació de la color de forma rigorosa, simulant un colorímetre, el que permetrà treballar en un espai de color de base física, independent d'el dispositiu i comparable amb dades obtingudes mitjançant altres càmeres que tambè estiguin caracteritzades.[EN] Cultural heritage documentation and preservation is an arduous and delicate task in which color plays a fundamental role. The correct determination of color provides vital information on a descriptive, technical and quantitative level. Classical color documentation methods in archaeology were usually restricted to strictly subjective procedures. However, this methodology has practical and technical limitations, affecting the results obtained in the determination of color. Nowadays, it is frequent to support classical methods with geomatics techniques, such as photogrammetry or laser scanning, together with digital image processing. Although digital images allow color to be captured quickly, easily, and in a non-invasive way, the RGB data provided by the camera does not itself have a rigorous colorimetric sense. Therefore, a rigorous transformation process to obtain reliable color data from digital images is required. This thesis proposes a novel technical solution, in which the integration of spectrophotometric and colorimetric analysis is intended as a complement to photogrammetric techniques that allow an improvement in color identification and representation of pigments with maximum reliability in 3D surveys, models and reconstructions. The proposed methodology is based on the colorimetric characterization of digital sensors, which is of novel application in cave paintings. The characterization aims to obtain the transformation equations between the device-dependent color data recorded by the camera and the independent, physically-based color spaces, such as those established by the Commission Internationale de l'Éclairage (CIE). The rigorous processing of color and spectral data requires software packages with specific colorimetric functionalities. Although there are different commercial software options, they do not integrate the digital image processing and colorimetric computations together. And more importantly, they do not allow the camera characterization to be carried out. Therefore, as a key aspect in this thesis is our in-house pyColourimetry software that was developed and tested taking into account the recommendations published by the CIE. pyColourimetry is an open-source code, independent without commercial ties; it allows the treatment of colorimetric and spectral data and the digital image processing, and gives full control of the characterization process and the management of the obtained data to the user. On the other hand, this study presents a further analysis of the main factors affecting the characterization, such as the camera built-in sensor, the camera parameters, the illuminant, the regression model, and the data set used for model training. For computing the transformation equations, the literature recommends the use of polynomial equations as a regression model. Thus, polynomial models are considered as a starting point in this thesis. Additionally, a regression model based on Gaussian processes has been applied, and the results obtained by means of polynomials have been compared. Also, a new working scheme was reported which allows the automatic selection of color samples, adapted to the chromatic range of the scene. This scheme is called P-ASK, based on the K-means classification algorithm. The results achieved in this thesis show that the proposed framework for camera characterization is highly applicable in documentation and conservation tasks in general cultural heritage applications, and particularly in rock art painting. It is a low-cost and non-invasive methodology that allows for the colorimetric recording from complete image scenes. Once characterized, a conventional digital camera can be used for rigorous color determination, simulating a colorimeter. Thus, it is possible to work in a physical color space, independent of the device used, and comparable with data obtained from other cameras that are also characterized.Thanks to the Universitat Politècnica de València for the FPI scholarshipMolada Tebar, A. (2020). Colorimetric and spectral analysis of rock art by means of the characterization of digital sensors [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/160386TESISCompendi

CAMERA SPECTRAL SENSITIVITY CHARACTERIZATION USING A BLACKBODY SOURCE

With digital cameras emerging as more effective tools for scientific research, there is increasing need for accurate and inexpensive ways to calibrate them. In particular, to date there has been no simple method to measure camera sensitivity as a function of wavelength. For example, narrow bandwidth monochromator beams are expensive and have calibration problems, while color chart method is unreliable owing to illumination dependence. This thesis presents a novel technique for spectral sensitivity calibration of a camera (or any black-and-white cameras or color sensors) using blackbody furnace operating at 650 - 1250 °C. Images recorded at 11 different temperatures are observed for red, green, and blue camera outputs. Using Planck ’ s Law to calculate the incident light intensities, the three color sensitivities as functions of wavelength are computed using MATLAB function that optimizes the spectral sensitivities until the blackbody measurements are closely matched. The results are in reasonable agreement with published sensitivities