Madsen, Claus B.

    Aalborg UniversitetEstimating Radiances of the Sun and the Sky From a Single Image ContainingShadowsMadsen, Claus B.Published in:Proceedings: 16th Danish Conference on Pattern Recognition and Image AnalysisPublication date:2008Document VersionPublisher's PDF, also known as Version of recordLink to publication from Aalborg UniversityCitation for published version (APA):Madsen, C. B. (2008). Estimating Radiances of the Sun and the Sky From a Single Image Containing Shadows.In Proceedings: 16th Danish Conference on Pattern Recognition and Image Analysis Department of ComputerScience, University of Copenhagen.  (	Koebenhavns Universitet. Datalogisk Institut. Rapport).General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            ? Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            ? You may not further distribute the material or use it for any profit-making activity or commercial gain            ? You may freely distribute the URL identifying the publication in the public portal ?Take down policyIf you believe that this document breaches copyright please contact us at vbn@aub.aau.dk providing details, and we will remove access tothe work immediately and investigate your claim.Downloaded from vbn.aau.dk on: May 01, 2017Estimating Radiances of the Sun and the Sky  From a Single Image Containing Shadows   Claus B. Madsen Laboratory of Computer Vision and Media Technology, Aalborg University, Denmark cbm@cvmt.aau.dk  1. Introduction Augmented Reality is about inserting computer graphics objects into images of reality. One of the main difficulties in achieving a convincing illusion is to obtain a model of the illumination conditions in the scene so as to be able to render the virtual objects with shading and shadows that are consistent with the real scene. Most work in Augmented Reality is based on recording and storing the scene illumination in an image, a socalled environment map or a light probe image. An environment map is a 360 by 180 degree (full omni-directional) image where each pixel corresponds to a direction in space, and the value of the pixel is the incident radiance from that particular direction. Light probe images are used extensively for special effects in movie production and a tried and tested approach delivering very high illumination precision and convincing results. The main problem is that the approach assumes the illumination is static in the scene. Once acquired it is not allowed to change. We have developed a new technique for estimating the illumination conditions in outdoor scenes from a single image. This technique does not require acquisition of a light probe image. Rather it assumes that some additional information is available for the image: 1) date and time for the acquisition of the image, 2) the longitude and latitude for the position on Earth where the image is taken, and 3) knowledge of the direction of gravity relative to the camera. These pieces of information can easily be made available building a clock, and GPS received and an inertial sensor into the camera. With this information available it is straight forward to compute the position of the Sun relative to the camera coordinate system. Using this knowledge we have developed a technique which, based on the shadows present in the image, can estimate the radiances from the Sun and the Sky, respectively. Using these radiances we can render augmented objects into the images with very realistic illumination regarding both shading and shadowing.   2. Approach The approach is quite simple, and rests on a few assumptions: 1) there are predominantly diffuse surfaces in the scene, and 2) the Sun is shining. The steps are as follows: 1. Acquire the image, noting the date, time and position on Earth 2. Compute the Sun’s position relative to the camera 3. Detect areas in the image which are in shadow which, with the technique we are using, as a bi-product yields the ratio between Sun + Sky irradiance to Sky irradiance 4. Use Sun’s position and estimated irradiance ratio to compute the radiance of the Sky and of the Sun (up to a scaling factor) 5. Render virtual objects into the image   Figure 1: Image acquired on a sunny day  Figure 2: Image with shadows removed   Figure 3: Image with virtual objects rendered with estimated illumination  Figure 3 shows how we can convincingly render virtual objects into an image with the developed technique for estimating the scene illumination conditions based only on shadow information in the scene.   Figure 4: Example image involving multiple surfaces and textures  Figure 5: The checkered surface is used for calibrating the camera to the ground plane since we do not at present have an inertial sensor to provide the direction of gravity. Conclusions  Our approach is based on an assumption that there are shadows in the image. We are presently working on developing techniques that will very robustly detect shadows if they exist, and we are also combine this work with another technique which can estimate the illumination in overcast weather conditions. Acknowledgments This research is funded by the CoSPE project (grant number 26-04-0171) under the Danish Research Agency. This support is gratefully acknowledged. References [1]  C.B. Madsen and M. Nielsen. Towards Probeless Augmented Reality – a Position paper. In Proceedings:International Conference on Graphics Theory and Applications(GRAPP), Funchal, Madeira – Portugal, page 255 – 261,  January 2008. 

Estimating Radiances of the Sun and the Sky From a Single Image Containing Shadows

http://vbn.aau.dk/files/16358238/dankom08probelessAR.pdf

Estimating Radiances of the Sun and the Sky From a Single Image Containing Shadows

Abstract

Similar works

Full text

Available Versions

VBN