A survey of visual preprocessing and shape representation techniques

Many recent theories and methods proposed for visual preprocessing and shape representation are summarized. The survey brings together research from the fields of biology, psychology, computer science, electrical engineering, and most recently, neural networks. It was motivated by the need to preprocess images for a sparse distributed memory (SDM), but the techniques presented may also prove useful for applying other associative memories to visual pattern recognition. The material of this survey is divided into three sections: an overview of biological visual processing; methods of preprocessing (extracting parts of shape, texture, motion, and depth); and shape representation and recognition (form invariance, primitives and structural descriptions, and theories of attention)

Optimization of Single and Layered Surface Texturing

In visualization problems, surface shape is often a piece of data that must be shown effectively. One factor that strongly affects shape perception is texture. For example, patterns of texture on a surface can show the surface orientation from foreshortening or compression of the texture marks, and surface depth through size variation from perspective projection. However, texture is generally under-used in the scientific visualization community. The benefits of using texture on single surfaces also apply to layered surfaces. Layering of multiple surfaces in a single viewpoint allows direct comparison of surface shape. The studies presented in this dissertation aim to find optimal methods for texturing of both single and layered surfaces. This line of research starts with open, many-parameter experiments using human subjects to find what factors are important for optimal texturing of layered surfaces. These experiments showed that texture shape parameters are very important, and that texture brightness is critical so that shading cues are available. Also, the optimal textures seem to be task dependent; a feature finding task needed relatively little texture information, but more shape-dependent tasks needed stronger texture cues. In visualization problems, surface shape is often a piece of data that must be shown effectively. One factor that strongly affects shape perception is texture. For example, patterns of texture on a surface can show the surface orientation from foreshortening or compression of the texture marks, and surface depth through size variation from perspective projection. However, texture is generally under-used in the scientific visualization community. The benefits of using texture on single surfaces also apply to layered surfaces. Layering of multiple surfaces in a single viewpoint allows direct comparison of surface shape. The studies presented in this dissertation aim to find optimal methods for texturing of both single and layered surfaces. This line of research starts with open, many-parameter experiments using human subjects to find what factors are important for optimal texturing of layered surfaces. These experiments showed that texture shape parameters are very important, and that texture brightness is critical so that shading cues are available. Also, the optimal textures seem to be task dependent; a feature finding task needed relatively little texture information, but more shape-dependent tasks needed stronger texture cues

Texture Segregation By Visual Cortex: Perceptual Grouping, Attention, and Learning

A neural model is proposed of how laminar interactions in the visual cortex may learn and recognize object texture and form boundaries. The model brings together five interacting processes: region-based texture classification, contour-based boundary grouping, surface filling-in, spatial attention, and object attention. The model shows how form boundaries can determine regions in which surface filling-in occurs; how surface filling-in interacts with spatial attention to generate a form-fitting distribution of spatial attention, or attentional shroud; how the strongest shroud can inhibit weaker shrouds; and how the winning shroud regulates learning of texture categories, and thus the allocation of object attention. The model can discriminate abutted textures with blurred boundaries and is sensitive to texture boundary attributes like discontinuities in orientation and texture flow curvature as well as to relative orientations of texture elements. The model quantitatively fits a large set of human psychophysical data on orientation-based textures. Object boundar output of the model is compared to computer vision algorithms using a set of human segmented photographic images. The model classifies textures and suppresses noise using a multiple scale oriented filterbank and a distributed Adaptive Resonance Theory (dART) classifier. The matched signal between the bottom-up texture inputs and top-down learned texture categories is utilized by oriented competitive and cooperative grouping processes to generate texture boundaries that control surface filling-in and spatial attention. Topdown modulatory attentional feedback from boundary and surface representations to early filtering stages results in enhanced texture boundaries and more efficient learning of texture within attended surface regions. Surface-based attention also provides a self-supervising training signal for learning new textures. Importance of the surface-based attentional feedback in texture learning and classification is tested using a set of textured images from the Brodatz micro-texture album. Benchmark studies vary from 95.1% to 98.6% with attention, and from 90.6% to 93.2% without attention.Air Force Office of Scientific Research (F49620-01-1-0397, F49620-01-1-0423); National Science Foundation (SBE-0354378); Office of Naval Research (N00014-01-1-0624

Surface analysis and fingerprint recognition from multi-light imaging collections

Multi-light imaging captures a scene from a fixed viewpoint through multiple photographs, each of which are illuminated from a different direction. Every image reveals information about the surface, with the intensity reflected from each point being measured for all lighting directions. The images captured are known as multi-light image collections (MLICs), for which a variety of techniques have been developed over recent decades to acquire information from the images. These techniques include shape from shading, photometric stereo and reflectance transformation imaging (RTI). Pixel coordinates from one image in a MLIC will correspond to exactly the same position on the surface across all images in the MLIC since the camera does not move. We assess the relevant literature to the methods presented in this thesis in chapter 1 and describe different types of reflections and surface types, as well as explaining the multi-light imaging process. In chapter 2 we present a novel automated RTI method which requires no calibration equipment (i.e. shiny reference spheres or 3D printed structures as other methods require) and automatically computes the lighting direction and compensates for non-uniform illumination. Then in chapter 3 we describe our novel MLIC method termed Remote Extraction of Latent Fingerprints (RELF) which segments each multi-light imaging photograph into superpixels (small groups of pixels) and uses a neural network classifier to determine whether or not the superpixel contains fingerprint. The RELF algorithm then mosaics these superpixels which are classified as fingerprint together in order to obtain a complete latent print image, entirely contactlessly. In chapter 4 we detail our work with the Metropolitan Police Service (MPS) UK, who described to us with their needs and requirements which helped us to create a prototype RELF imaging device which is now being tested by MPS officers who are validating the quality of the latent prints extracted using our technique. In chapter 5 we then further developed our multi-light imaging latent fingerprint technique to extract latent prints from curved surfaces and automatically correct for surface curvature distortions. We have a patent pending for this method

Modelling the human perception of shape-from-shading

Shading conveys information on 3-D shape and the process of recovering this information is called shape-from-shading (SFS). This thesis divides the process of human SFS into two functional sub-units (luminance disambiguation and shape computation) and studies them individually. Based on results of a series of psychophysical experiments it is proposed that the interaction between first- and second-order channels plays an important role in disambiguating luminance. Based on this idea, two versions of a biologically plausible model are developed to explain the human performances observed here and elsewhere. An algorithm sharing the same idea is also developed as a solution to the problem of intrinsic image decomposition in the field of image processing. With regard to the shape computation unit, a link between luminance variations and estimated surface norms is identified by testing participants on simple gratings with several different luminance profiles. This methodology is unconventional but can be justified in the light of past studies of human SFS. Finally a computational algorithm for SFS containing two distinct operating modes is proposed. This algorithm is broadly consistent with the known psychophysics on human SFS

Decoding locomotion from population neural activity in moving C. elegans

The activity of an animal’s brain contains information about that animal’s actions and movements. We investigated the neural representation of locomotion in the nematode C. elegans by recording population calcium activity during unrestrained movement. We report that a neural population more accurately decodes locomotion than any single neuron. Relevant signals are distributed across neurons with diverse tunings to locomotion. Two distinct subpopulations are informative for decoding velocity and body curvature, and different neurons’ activities contribute features relevant for different instances of behavioral motifs. We labeled neurons AVAL and AVAR and found their activity was highly correlated with one another. They exhibited expected transients during backward locomotion, although they were not always the most informative neurons for decoding velocity. Finally, we compared population neural activity during movement and immobilization. Immobilization alters the correlation structure of neural activity and its dynamics. Some neurons previously correlated with AVA become anti-correlated and vice versa