Visual Recognition using Local Appearance

. This paper presents a technique for visual recognition in which physical objects are represented by families of surfaces in a local appearance space. An orthogonal family of local appearance descriptors is obtained by applying principal components analysis to image neighborhoods. The principal components with the largest variance are used to define a space for describing local appearance. The projection of the set of all neighborhoods from an image gives a discrete sampling of a surface in this space. Projecting neighborhoods from images taken at different viewing positions gives a family of surfaces which represent the possible local appearances from those viewing directions. Recognition is achieved by projecting windows from newly acquired images into the local appearance space and associating them to nearby surfaces. An efficient search tree data structure is used to associate projected points to surfaces. Our results show that in many common situations, a single window is suffici..