A Framework for Symmetric Part Detection in Cluttered Scenes

The role of symmetry in computer vision has waxed and waned in importance during the evolution of the field from its earliest days. At first figuring prominently in support of bottom-up indexing, it fell out of favor as shape gave way to appearance and recognition gave way to detection. With a strong prior in the form of a target object, the role of the weaker priors offered by perceptual grouping was greatly diminished. However, as the field returns to the problem of recognition from a large database, the bottom-up recovery of the parts that make up the objects in a cluttered scene is critical for their recognition. The medial axis community has long exploited the ubiquitous regularity of symmetry as a basis for the decomposition of a closed contour into medial parts. However, today's recognition systems are faced with cluttered scenes, and the assumption that a closed contour exists, i.e. that figure-ground segmentation has been solved, renders much of the medial axis community's work inapplicable. In this article, we review a computational framework, previously reported in Lee et al. (2013), Levinshtein et al. (2009, 2013), that bridges the representation power of the medial axis and the need to recover and group an object's parts in a cluttered scene. Our framework is rooted in the idea that a maximally inscribed disc, the building block of a medial axis, can be modeled as a compact superpixel in the image. We evaluate the method on images of cluttered scenes.Comment: 10 pages, 8 figure

Computational models for image contour grouping

Contours are one dimensional curves which may correspond to meaningful entities such as object boundaries. Accurate contour detection will simplify many vision tasks such as object detection and image recognition. Due to the large variety of image content and contour topology, contours are often detected as edge fragments at first, followed by a second step known as {u0300}{u0300}contour grouping'' to connect them. Due to ambiguities in local image patches, contour grouping is essential for constructing globally coherent contour representation. This thesis aims to group contours so that they are consistent with human perception. We draw inspirations from Gestalt principles, which describe perceptual grouping ability of human vision system. In particular, our work is most relevant to the principles of closure, similarity, and past experiences. The first part of our contribution is a new computational model for contour closure. Most of existing contour grouping methods have focused on pixel-wise detection accuracy and ignored the psychological evidences for topological correctness. This chapter proposes a higher-order CRF model to achieve contour closure in the contour domain. We also propose an efficient inference method which is guaranteed to find integer solutions. Tested on the BSDS benchmark, our method achieves a superior contour grouping performance, comparable precision-recall curves, and more visually pleasant results. Our work makes progresses towards a better computational model of human perceptual grouping. The second part is an energy minimization framework for salient contour detection problem. Region cues such as color/texture homogeneity, and contour cues such as local contrast, are both useful for this task. In order to capture both kinds of cues in a joint energy function, topological consistency between both region and contour labels must be satisfied. Our technique makes use of the topological concept of winding numbers. By using a fast method for winding number computation, we find that a small number of linear constraints are sufficient for label consistency. Our method is instantiated by ratio-based energy functions. Due to cue integration, our method obtains improved results. User interaction can also be incorporated to further improve the results. The third part of our contribution is an efficient category-level image contour detector. The objective is to detect contours which most likely belong to a prescribed category. Our method, which is based on three levels of shape representation and non-parametric Bayesian learning, shows flexibility in learning from either human labeled edge images or unlabelled raw images. In both cases, our experiments obtain better contour detection results than competing methods. In addition, our training process is robust even with a considerable size of training samples. In contrast, state-of-the-art methods require more training samples, and often human interventions are required for new category training. Last but not least, in Chapter 7 we also show how to leverage contour information for symmetry detection. Our method is simple yet effective for detecting the symmetric axes of bilaterally symmetric objects in unsegmented natural scene images. Compared with methods based on feature points, our model can often produce better results for the images containing limited texture

Robust and flexible multi-scale medial axis computation

The principle of the multi-scale medial axis (MMA) is important in that any object is detected at a blurring scale proportional to the size of the object. Thus it provides a sound balance between noise removal and preserving detail. The robustness of the MMA has been reflected in many existing applications in object segmentation, recognition, description and registration. This thesis aims to improve the computational aspects of the MMA. The MMA is obtained by computing ridges in a “medialness” scale-space derived from an image. In computing the medialness scale-space, we propose an edge-free medialness algorithm, the Concordance-based Medial Axis Transform (CMAT). It not only depends on the symmetry of the positions of boundaries, but also is related to the symmetry of the intensity contrasts at boundaries. Therefore it excludes spurious MMA branches arising from isolated boundaries. In addition, the localisation accuracy for the position and width of an object, as well as the robustness under noisy conditions, is preserved in the CMAT. In computing ridges in the medialness space, we propose the sliding window algorithm for extracting locally optimal scale ridges. It is simple and efficient in that it can readily separate the scale dimension from the search space but avoids the difficult task of constructing surfaces of connected maxima. It can extract a complete set of MMA for interfering objects in scale-space, e.g. embedded or adjacent objects. These algorithms are evaluated using a quantitative study of their performance for 1-D signals and qualitative testing on 2-D images

Scene Segmentation and Object Classification for Place Recognition

This dissertation tries to solve the place recognition and loop closing problem in a way similar to human visual system. First, a novel image segmentation algorithm is developed. The image segmentation algorithm is based on a Perceptual Organization model, which allows the image segmentation algorithm to ‘perceive’ the special structural relations among the constituent parts of an unknown object and hence to group them together without object-specific knowledge. Then a new object recognition method is developed. Based on the fairly accurate segmentations generated by the image segmentation algorithm, an informative object description that includes not only the appearance (colors and textures), but also the parts layout and shape information is built. Then a novel feature selection algorithm is developed. The feature selection method can select a subset of features that best describes the characteristics of an object class. Classifiers trained with the selected features can classify objects with high accuracy. In next step, a subset of the salient objects in a scene is selected as landmark objects to label the place. The landmark objects are highly distinctive and widely visible. Each landmark object is represented by a list of SIFT descriptors extracted from the object surface. This object representation allows us to reliably recognize an object under certain viewpoint changes. To achieve efficient scene-matching, an indexing structure is developed. Both texture feature and color feature of objects are used as indexing features. The texture feature and the color feature are viewpoint-invariant and hence can be used to effectively find the candidate objects with similar surface characteristics to a query object. Experimental results show that the object-based place recognition and loop detection method can efficiently recognize a place in a large complex outdoor environment

Automatic Extraction of Closed Contours Bounding Salient Objects: New Algorithms and Evaluation Methods

The problem under consideration in this dissertation is achieving salient object segmentation of natural images by means of probabilistic contour grouping. The goal is to extract the simple closed contour bounding the salient object in a given image. The method proposed here falls in the Contour Grouping category, searching for the optimal grouping of boundary entities to form an object contour. Our first contribution is to provide both a ground truth dataset and a performance measure for empirical evaluation of salient object segmentation methods. Our Salient Object Dataset (SOD) provides ground truth boundaries of salient objects perceived by humans in natural images. We also psychophysically evaluated 5 distinct performance measures that have been used in the literature and showed that a measure based upon minimal contour mappings is most sensitive to shape irregularities and most consistent with human judgements. In fact, the Contour Mapping measure is as predictive of human judgements as human subjects are of each other. Contour grouping methods often rely on Gestalt cues locally defined on pairs of oriented features. Accurate integration of these local cues with global cues is a challenge. A second major contribution of this dissertation is a novel, effective method for combining local and global cues. A third major contribution in this dissertation is a novel method based on Principal Component Analysis for promoting diversity among contour hypotheses, leading to substantial improvements in grouping performance. To further improve the performance, a multiscale implementation of this method has been studied. A fourth contribution in this dissertation is studying the effect of the multiscale prior on the performance and analysing the method for combining the results obtained in different resolutions. Our final contribution is comparing the performance of univariate distribution models for local cues used by our method with the use of a multivariate mixture model for their joint distribution. We obtain slight improvement by the mixture models. The proposed method has been evaluated and compared with four other state-of-the-art grouping methods, showing considerably better performance on the SOD ground truth dataset

Shape-based Object Detection via Boundary Structure

We address the problem of object detection and segmentation using global holistic properties of object shape. Global shape representations are highly susceptible to clutter inevitably present in realistic images, and can be applied robustly only using a precise segmentation of the object. To this end, we propose a figure/ground segmentation method for extraction of image regions that resemble the global properties of a model boundary structure and are perceptually salient. Our shape representation, called the chordiogram, is based on geometric relationships of object boundary edges, while the perceptual saliency cues we use favor coherent regions distinct from the background. We formulate the segmentation problem as an integer quadratic program and use a semdefinite programming relaxation to solve it. Obtained solutions provide the segmentation of an object as well as a detection score used for object recognition. Our single-step approach achieves state-of-the-art performance on several object detection and segmentation benchmarks