We show the surprising result that colors across a change in viewing
condition (changing light color, shading and camera) are related by a
homography. Our homography color correction application delivers improved color
fidelity compared with the linear least-square.Comment: Accepted by Progress in Colour Studies 201

Finlayson, Graham D.

Fisher, Robert B.

Gong, Han

English

arXiv

University of East Anglia digital repository

Color HomographyGraham Finlayson∗, Han Gong∗, and Robert Fisher†∗School of Computing Sciences, University of East Anglia, UK†School of Informatics, University of Edinburgh, UK1Color Homographya)Homography H Least-squares Mb)a)c)Ball_1 Ball_2Ball_3Ball_4b)Hrgrgπ1 π2Fig. 1. Top left, panel (a), images of two planes are related by an homography.Right, panel (b), 4 images of a colored ball are shown. Ball 1 is the referenceimage where the illumination color is white and placed behind the camera.Ball 2 is the object illuminated with a blue light from above. Respectively,Ball 3 and Ball 4 are the least-squares mapping and the homography matchfrom Ball 2 to Ball 1, Bottom right, panel (c), the chromaticities from Ball 2matched to corresponding chromaticities in Ball 1.I. INTRODUCTIONHomographies are at the center of geometric methods incomputer vision and are used in geometric camera calibration,3D reconstruction, stereo vision and image mosaicking amongother tasks. In this paper, we show the surprising result thatcolors across a change in viewing condition (changing lightcolor, shading and camera) are also related by a homography.In geometry computer vision, an homography relates twoplanes. In Figure 1a, pi1 and pi2 might denote the same 3Dplane viewed in two different images related by the homog-raphy H . In color, an homography relates two photometricviews. In Figure 1b, Ball 1 is the image of the side-view ofa 4-color ball where the ball is lit from behind the camerawith a white light. The same ball is lit from above witha bluish light, image Ball 2. The images are in pixel-wisecorrespondence. We first color correct Ball 2 to match Ball 1by using linear regression. Its results is shown in image Ball 3where the colors are incorrectly mapped and the red colorsegment looks particularly wrong. Ball 4 is a better examplethat Ball 2 is color corrected by using the linear transformby color homography. In this paper, we propose that to mapone photometric view to another we must map the colorscorrectly independent of shading. Since shading only affectsthe brightness, or magnitude, of the RGB vectors, it is possibleto find the 3×3 map which maps the color rays (the RGBs witharbitrary scalings) in one photometric view to correspondingrays in another. We note that this ”ray matching” is preciselythe circumstance in geometric mapping for co-planar andcorresponding points in two images [1]. An RGB withoutshading can be encoded as the rg-chromaticity coordinate.In Figure 1c, the 4 reflectances from the ball correspondto 4 points in an rg-chromaticity diagram which define thequadrilaterals shown in the left and right of the panel (forrespectively for the images Ball 2 and Ball 1). The mappingbetween the two chromaticity diagrams is an homography. Weshow that the color calibration problem – mapping deviceRGBs recorded for a color chart to corresponding XYZs –can be formulated as a homography problem. Because in realshading varies across the chart solving for the homographycan deliver a 50% improvement in color correction comparedwith direct linear least-squares regression.II. BACKGROUNDFor the geometric planar homography problem, we write: αxαyαᵀ = x′y′1ᵀ  h11 h12 h13h21 h22 h23h31 h32 h33 , x = H(x′)(1)In Equation 1, (x, y) and (x′, y′) denote corresponding imagepoints – the same physical feature – in two images. Inhomogeneous coordinates the vector [a b c]ᵀ maps to thecoordinates [a/c b/c]ᵀ and so, in Equation 1, the scalar αcancels to form the image coordinate (x, y). For all pairs ofcorresponding points (x, y) and (x′, y′) that lie on the sameplane in 3 dimensional space, Equation 1 exactly characterisesthe relationship between their images [1]. To solve for anhomography (e.g. for image mosaicking), we need to find atleast 4 corresponding points in a pair of images.III. COLOR HOMOGRAPHYLet us map an RGB ρ to a corresponding RGI (red-green-intensity) c using a full-rank 3× 3 matrix C:ρᵀC = cᵀ RGBᵀ  1 0 10 1 10 0 1 = RGR+G+Bᵀ (2)The r and g chromaticity coordinates are written asr = R/(R+G+B) , g = R/(R+G+B) interpreting theright-hand-side of Equation 2 as a homogeneous coordinatewe see that c ∝ [ r g 1 ]ᵀ. In the following proof it isuseful to represent 2-d chromaticities by their corresponding3-d homogeneous coordinates.Theorem 1 (Colour Homography). Chromaticities across achange in capture condition (light color, shading and imagingdevice) are a homography apart.Proof. First we assume that across a change in illuminationor a change in device where the shading is the same (for theMondrian-world) the corresponding RGBs are related by alinear transform M. Clearly, H = C−1MC maps colors in2sRGBRGBrawsRGB = H(RGBraw)Fig. 2. Color correction (mapping raw to display sRGB) is an homographyproblem. This figure is also a color chart example which was used for ourcolor correction evaluation.RGI form between illuminants. Due to different shading, theRGI triple under a second light is represented as c′ᵀ = α′cᵀH ,where α′ denotes the unknown scaling. Without loss ofgenerality let us interpret c as a homogeneous coordinate i.e.assume its third component is 1. Then, [r′ g′]ᵀ = H([r g]ᵀ)(chromaticity coordinates are a homography H() apart).A. Solving color Homography by Alternating Least SquaresSuppose A and B denote respectively n× 3 matrices of ncorresponding pixels with respect to two images of the samescene where the illumination changes (and, also possibly thecamera properties). The color change is modeled as a lineartransform. And, due to the relative positions of light andsurfaces, the per-pixel shading intensities are usually different.Assuming the Lambertian image formation, DAH ≈ B whereD is an n× n diagonal matrix of shading factors and H is a3×3 color correction matrix. We solve this equation by usingAlternating Least-Squares (ALS) described in Algorithm 1.The effect of the individual Hi and Di can be combined intoAlgorithm 1: Homography from alternating least-square1 i = 0, A0 = A;2 repeat3 i = i+ 1;4 minDi∥∥DiAi−1 −B∥∥;5 minHi∥∥DiAi−1Hi −B∥∥;6 Ai = DiAHi;7 until∥∥Ai −Ai−1∥∥ < ;a single matrix D =∏iDi and H =∏iHi.Alternately, we can solve for H in direct analogy tocomputer vision. Specifically, we randomly select sets of 4corresponding points and find the best H and then test thegoodness of fit for the H discovered on the rest of the dataset.This RANSAC method (Random sampling consensus) has theadvantage that we can robustly minimize any chosen errorincluding CIE Lab.Finally, we note that existing works have already developedintensity invariant color matching [2, 3]. Though, neither hasreported the homography formalism or exploded the enhancedflexibility it offers.IV. COLOR CORRECTIONIn color correction - mapping raw RGBs to a display colorspace - the target RGBs, are known to vary in intensity. InMethod mean median 95% maxLeast-squares 3.70 3.30 7.73 8.39Homography 2.34 2.09 5.02 5.43TABLE ICIE ∆E ERROR COMPARISONFigure 2, we show the picture of an image in the raw RGBspace of a camera and the corresponding reproduction whenthe colors are corrected for display. Very professional imagingscientist might take a picture of a color checker and a secondpicture of a uniform gray target with same size in the samelocation. By dividing the RGB image of the checker by theimage of the gray-target the shading is removed and then theshading corrected RGBs can be mapped to known referencedisplay color coordinates using simple least-squares. However,this two-step approach is inconvient and in some cases cannotbe done at all (e.g. in an on-going surveillance situation). Inour evaluation, this two-step approach is applied to obtainground truth RGB inputs with corrected shading. For eachmethod, we obtain a 3 × 3 color correction matrix from thenon-shading-corrected RGB inputs to a reference target andthen apply this matrix to the shading corrected RGB inputs.This experimental methodology is described in detail in [3].The color correction error is measured using the CIELab metric [4]. All 3 of our evaluation images were takenaround a local historical site that is popular with amateur andprofessional photographers alike (e.g. Figure 2). The mean,median, 95% quantile and max ∆E errors are reported inTable I. It is clear that homography-based color correctionsupports a significantly improved color correction performance(all error measures are ≈ 40% improved).V. CONCLUSIONIn this paper, we shown the surprising result that colorsacross a change in viewing condition (changing light color,shading and camera) are related by a homography. Our ho-mography color correction application delivers improved colorfidelity compared with the linear least-square.REFERENCES[1] R. I. Hartley and A. Zisserman, Multiple View Geometry inComputer Vision, 2nd ed. Cambridge University Press, ISBN:0521540518, 2004. 1[2] G. D. Finlayson, M. Mohammadzadeh Darrodi, and M. Mack-iewicz, “The alternating least squares technique for nonuni-form intensity color correction,” Color Research & Application,vol. 40, no. 3, pp. 232–242, 2015. 2[3] B. Funt and P. Bastani, “Irradiance-independent camera colorcalibration,” Color Research & Application, vol. 39, no. 6, pp.540–548, 2014. 2[4] G. Wyszecki and W. Stiles, Color Science: Concepts and Meth-ods, Quantitative Data and Formulas, 2nd ed. Wiley, New York,1982. 2

Color homography

Abstract

Similar works

Full text

Available Versions

University of East Anglia digital repository