Shadows occur frequently in indoor scenes and outdoors on sunny days. Despite the information inherent in shadows about a scene\u27s geometry and lighting conditions, relatively little work in image understanding has addressed the important problem of recognizing shadows. This is an even more serious failing when one considers the problems shadows pose for many visual techniques such as object recognition and shape from shading. Shadows are difficult to identify because they cannot be infallibly recognized until a scene\u27s geometry and lighting are known. However, there are a number of cues which together strongly suggest the identification of a shadow. We present a list of these cues and methods which can be used by an active observer to detect shadows. By an active observer, we mean an observer that is not only mobile, but can extend a probe into its environment. The proposed approach should allow the extraction of shadows in real time. Furthermore, the identification of a shadow should improve with observing time. In order to be able to identify shadows without or prior to obtaining information about the arrangement of objects or information about the spectral properties of materials in the scene, we provide the observer with a probe with which to cast its own shadows. Any visible shadows cast by the probe can be easily identified because they will be new to the scene. These actively obtained shadows allow the observer to experimentally determine the number and location of light sources in the scene, to locate the cast shadows, and to gain information about the likely spectral changes due to shadows. We present a novel method for locating a light source and the surface on which a shadow is cast. It takes into account errors in imaging and image processing and, furthermore, it takes special advantage of the benefits of an active observer. The information gained from the probe is of particular importance in effectively using the various shadow cues. In the course of identifying shadows, we also present a new modification on an image segmentation algorithm. Our modification provides a general description of color images in terms of regions that is particularly amenable to the analysis of shadows
