Learning object segmentation from video data

This memo describes the initial results of a project to create a self-supervised algorithm for learning object segmentation from video data. Developmental psychology and computational experience have demonstrated that the motion segmentation of objects is a simpler, more primitive process than the detection of object boundaries by static image cues. Therefore, motion information provides a plausible supervision signal for learning the static boundary detection task and for evaluating performance on a test set. A video camera and previously developed background subtraction algorithms can automatically produce a large database of motion-segmented images for minimal cost. The purpose of this work is to use the information in such a database to learn how to detect the object boundaries in novel images using static information, such as color, texture, and shape. This work was funded in part by the Office of Naval Research contract #N00014-00-1-0298, in part by the Singapore-MIT Alliance agreement of 11/6/98, and in part by a National Science Foundation Graduate Student Fellowship

Structured manifolds for motion production and segmentation : a structured Kernel Regression approach

Steffen JF. Structured manifolds for motion production and segmentation : a structured Kernel Regression approach. Bielefeld (Germany): Bielefeld University; 2010

Learning object boundary detection from motion data

A significant barrier to applying the techniques of machine learning to the domain of object boundary detection is the need to obtain a large database of correctly labeled examples. Inspired by developmental psychology, this paper proposes that boundary detection can be learned from the output of a motion tracking algorithm that separates moving objects from their static surroundings. Motion segmentation solves the database problem by providing cheap, unlimited, labeled training data. A probabilistic model of the textural and shape properties of object boundaries can be trained from this data and then used to efficiently detect boundaries in novel images via loopy belief propagation.Singapore-MIT Alliance (SMA

Learning static object segmentation from motion segmentation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2005.Includes bibliographical references (p. 105-110).This thesis describes the SANE (Segmentation According to Natural Examples) algorithm for learning to segment objects in static images from video data. SANE uses background subtraction to find the segmentation of moving objects in videos. This provides object segmentation information for each video frame. The collection of frames and segmentations forms a training set that SANE uses to learn the image and shape properties that correspond to the observed motion boundaries. Then, when presented with new static images, the model infers segmentations similar to the observed motion segmentations. SANE is a general method for learning environment-specific segmentation models. Because it is self-supervised, it can adapt to a new environment and new objects with relative ease. Comparisons of its output to a leading image segmentation algorithm demonstrate that motion-defined object segmentation is a distinct problem from traditional image segmentation. The model outperforms a trained local boundary detector because it leverages the shape information it learned from the training data.by Michael Gregory Ross.Ph.D

A model-based machine vision system using fuzzy logic

AbstractAn effective model-based machine vision system is designed for use in practical production lines. In the proposed system, the gray level corner is selected as a local feature, and a gray level corner detector is developed. The gray level corner detection problem is formulated as a pattern classification problem to determine whether a pixel belongs to the class of corners or not. The probability density function is estimated by means of fuzzy logic. A corner matching method is developed to minimize the amount of calculation. All available information obtained from the gray level corner detector is used to make the model. From a fuzzy inference procedure, a matched segment list is extracted, and the resulted segment list is used to calculate the transformations between the model object and each object in the scene. In order to reduce the fuzzy rule set, a notion of overlapping cost is introduced. To show the effectiveness of the developed algorithm, simulations are conducted for synthetic images, and an experiment is conducted on an image of a real industrial component

A survey of visual preprocessing and shape representation techniques

Many recent theories and methods proposed for visual preprocessing and shape representation are summarized. The survey brings together research from the fields of biology, psychology, computer science, electrical engineering, and most recently, neural networks. It was motivated by the need to preprocess images for a sparse distributed memory (SDM), but the techniques presented may also prove useful for applying other associative memories to visual pattern recognition. The material of this survey is divided into three sections: an overview of biological visual processing; methods of preprocessing (extracting parts of shape, texture, motion, and depth); and shape representation and recognition (form invariance, primitives and structural descriptions, and theories of attention)

Hypothesis-based image segmentation for object learning and recognition

Denecke A. Hypothesis-based image segmentation for object learning and recognition. Bielefeld: Universität Bielefeld; 2010.This thesis addresses the figure-ground segmentation problem in the context of complex systems for automatic object recognition as well as for the online and interactive acquisition of visual representations. First the problem of image segmentation in general terms and next its importance for object learning in current state-of-the-art systems is introduced. Secondly a method using artificial neural networks is presented. This approach on the basis of Generalized Learning Vector Quantization is investigated in challenging scenarios such as the real-time figure-ground segmentation of complex shaped objects under continuously changing environment conditions. The ability to fulfill these requirements characterizes the novelty of the approach compared to state-of-the-art methods. Finally our technique is extended towards online adaption of model complexity and the integration of several segmentation cues. This yields a framework for object segmentation that is applicable to improve current systems for visual object learning and recognition

N-ary Relation Extraction using Graph State LSTM

Cross-sentence $n$-ary relation extraction detects relations among $n$ entities across multiple sentences. Typical methods formulate an input as a \textit{document graph}, integrating various intra-sentential and inter-sentential dependencies. The current state-of-the-art method splits the input graph into two DAGs, adopting a DAG-structured LSTM for each. Though being able to model rich linguistic knowledge by leveraging graph edges, important information can be lost in the splitting procedure. We propose a graph-state LSTM model, which uses a parallel state to model each word, recurrently enriching state values via message passing. Compared with DAG LSTMs, our graph LSTM keeps the original graph structure, and speeds up computation by allowing more parallelization. On a standard benchmark, our model shows the best result in the literature.Comment: EMNLP 18 camera read

Computational Neural Models of Spatial Integration in Perceptual Grouping

Recent developments in the neural computational modeling of perceptual grouping are described with reference to a newly proposed taxonomy to formalize mechanisms of spatial integration. This notational framework and nomenclature is introduced in or-der to clarify key properties common to all or most models, while permitting unique attributes of each approach to be independently examined. The strength of spatial integration in the models that are considered is always some function of the distances and relative alignments in perceptual space of the centers of units representing orien-tational features or energy in a visual scene. We discuss the signicance of variations of the constituents of an activation function for spatial integration, and also consider the larger modeling framework in which this function is applied in each approach. We also discuss the relationship of feedforward and feedback mechanisms and the issues of self-organization as core principles underlying the establishment of spatial integra-tion mechanisms. The relationship of the grouping models to models of other visual competencies is considered with respect to prospects for future research. 354 From Fragments to Object