Image Understanding and Robotics Research at Columbia University

The research investigations of the Vision/Robotics Laboratory at Columbia University reflect the diversity of interests of its four faculty members, two staff programmers and 15 Ph.D. students. Several of the projects involve either a visiting computer science post-doc, other faculty members in the department or the university, or researchers at AT&T Bell Laboratories or Philips laboratories. We list below a summary of our interest and results, together with the principal researchers associated with them. Since it is difficult to separate those aspects of robotic research that are purely visual from those that are vision-like (for example, tactile sensing) or vision-related (for example, integrated vision-robotic systems), we have listed all robotic research that is not purely manipulative

Image Understanding and Robotics Research at Columbia University

Over the past year, the research investigations of the Vision/Robotics Laboratory at Columbia University have reflected the interests of its four faculty members, two staff programmers, and 16 Ph.D. students. Several of the projects involve other faculty members in the department or the university, or researchers at AT&T, IBM, or Philips. We list below a summary of our interests and results, together with the principal researchers associated with them. Since it is difficult to separate those aspects of robotic research that are purely visual from those that are vision-like (for example, tactile sensing) or vision-related (for example, integrated vision-robotic systems), we have listed all robotic research that is not purely manipulative. The majority of our current investigations are deepenings of work reported last year; this was the second year of both our basic Image Understanding contract and our Strategic Computing contract. Therefore, the form of this year's report closely resembles last year's. Although there are a few new initiatives, mainly we report the new results we have obtained in the same five basic research areas. Much of this work is summarized on a video tape that is available on request. We also note two service contributions this past year. The Special Issue on Computer Vision of the Proceedings of the IEEE, August, 1988, was co-edited by one of us (John Kender [27]). And, the upcoming IEEE Computer Society Conference on Computer Vision and Pattem Recognition, June, 1989, is co-program chaired by one of us (John Kender [23])

Shape From Textures: A Paradigm for Fusing Middle Level Vision Cues

This research proposes a new approach to the problem of deriving the orientation, segmentation, and classification of surfaces based on multiple independent textual cues. The generality of this approach is due to the interaction between textural cues, thus allowing it to extract shape information from a wider range of textured surfaces than any individual method. The method consists of three major phases: the calculation of orientation constraints for subimage elements called "texel patches", the consolidation of constraints into a "most likely" orientation per patch, and finally the reconstruction of the surface. During the first phase, the different shape-from-texture components generate augmented texels. Each augmented texel consists of the 2-D description of a texel patch and a list of weighted constraints on its orientation. The orientation constraints for each patch are potentially inconsistent or potentially incorrect because the shape-from methods are applied to noisy images, locally based, and derive constraints without a priori knowledge of the type of texture or number of surfaces. The constraints are weighted by each shape-from method based on an intra-cue correctness factor. This factor attempts to measure how closely the constraint fulfill the underlying assumptions of the cue. The orientation constraints' weights are then normalized between cues in order to assure that no cue predominates unfairly. In the second phase, all the orientation constraints for each augmented texel are consolidated into a single "most likely" orientation by a Hough-like transformation on a tesselated Gaussian sphere. The system iteratively reanalyzes each of the texel patches, calculating the "most likely" orientations for each patch. Finally, the system re-analyzes the orientation constraints to determine which augmented texels are part of the same constraint family and which cues were used to generated the valid constraints. In effect, this both segments the image into regions of similar orientation and supplies texture classification information. The robustness of this approach is illustrated by a system that fuses the orientation constraints of five shape-from cues and solves real camera-acquired imagery

Approximate Spatial Layout Processing in the Visual System: Modeling Texture-Based Segmentation and Shape Estimation

Moving through the environment, grasping objects, orienting oneself, and countless other tasks all require information about spatial organization. This in turn requires determining where surfaces, objects and other elements of a scene are located and how they are arranged. Humans and other animals can extract spatial organization from vision rapidly and automatically. To better understand this capability, it would be useful to know how the visual system can make an initial estimate of the spatial layout. Without time or opportunity for a more careful analysis, a rough estimate may be all that the system can extract. Nevertheless, rough spatial information may be sufficient for many purposes, even if it is devoid of details that are important for tasks such as object recognition. The human visual system uses many sources of information for estimating layout. Here I focus on one source in particular: visual texture. I present a biologically reasonable, computational model of how the system can exploit patterns of texture for performing two basic tasks in spatial layout processing: locating possible surfaces in the visual input, and estimating their approximate shapes. Separately, these two tasks have been studied extensively, but they have not previously been examined together in the context of a model grounded in neurophysiology and psychophysics. I show that by integrating segmentation and shape estimation, a system can share information between these processes, allowing the processes to constrain and inform each other as well as save on computations. The model developed here begins with the responses of simulated complex cells of the primary visual cortex, and combines a weak membrane/functional minimization approach to segmentation with a shape estimation method based on tracking changes in the average dominant spatial frequencies across a surface. It includes mechanisms for detecting untextured areas and flat areas in an input image. In support of the model, I present a software simulation that can perform texture-based segmentation and shape estimation on images containing multiple, curved, textured surfaces.Ph.D.Applied SciencesBiological SciencesCognitive psychologyComputer scienceNeurosciencesPsychologyUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/131446/2/9909908.pd