Scale detection via keypoint density maps in regular or near-regular textures

In this paper we propose a new method to detect the global scale of images with regular, near regular, or homogenous textures. We define texture ‘‘scale’’ as the size of the basic elements (texels or textons) that most frequently occur into the image. We study the distribution of the interest points into the image, at different scale, by using our Keypoint Density Maps (KDMs) tool. A ‘‘mode’’ vector is built computing the most frequent values (modes) of the KDMs, at different scales. We observed that the mode vector is quasi linear with the scale. The mode vector is properly subsampled, depending on the scale of observation, and compared with a linear model. Texture scale is estimated as the one which minimizes an error function between the related subsampled vector and the linear model. Results, compared with a state of the art method, are very encouraging

Modeling Dynamic Swarms

This paper proposes the problem of modeling video sequences of dynamic swarms (DS). We define DS as a large layout of stochastically repetitive spatial configurations of dynamic objects (swarm elements) whose motions exhibit local spatiotemporal interdependency and stationarity, i.e., the motions are similar in any small spatiotemporal neighborhood. Examples of DS abound in nature, e.g., herds of animals and flocks of birds. To capture the local spatiotemporal properties of the DS, we present a probabilistic model that learns both the spatial layout of swarm elements and their joint dynamics that are modeled as linear transformations. To this end, a spatiotemporal neighborhood is associated with each swarm element, in which local stationarity is enforced both spatially and temporally. We assume that the prior on the swarm dynamics is distributed according to an MRF in both space and time. Embedding this model in a MAP framework, we iterate between learning the spatial layout of the swarm and its dynamics. We learn the swarm transformations using ICM, which iterates between estimating these transformations and updating their distribution in the spatiotemporal neighborhoods. We demonstrate the validity of our method by conducting experiments on real video sequences. Real sequences of birds, geese, robot swarms, and pedestrians evaluate the applicability of our model to real world data.Comment: 11 pages, 17 figures, conference paper, computer visio

SimLocator: robust locator of similar objects in images

International audienceSimilar objects commonly appear in natural images, and locating and cutting out these objects can be tedious when using classical interactive image segmentation methods. In this paper, we propose SimLocator, a robust method oriented to locate and cut out similar objects with minimum user interaction. After extracting an arbitrary object template from the input image, candidate locations of similar objects are roughly detected by distinguishing the shape and color features of each image. A novel optimization method is then introduced to select accurate locations from the two sets of candidates. Additionally, a mattingbased method is used to improve the results and to ensure that all similar objects are located in the image. Finally, a method based on alpha matting is utilized to extract the precise object contours. To ensure the performance of the matting operation, this work has developed a new method for foreground extraction. Experiments show that SimLocator is more robust and more convenient to use compared to other more advanced repetition detection and interactive image segmentation methods, in terms of locating similar objects in images

Deformation analysis and its application in image editing.

Jiang, Lei.Thesis (M.Phil.)--Chinese University of Hong Kong, 2011.Includes bibliographical references (p. 68-75).Abstracts in English and Chinese.Chapter 1 --- Introduction --- p.1Chapter 2 --- Background and Motivation --- p.5Chapter 2.1 --- Foreshortening --- p.5Chapter 2.1.1 --- Vanishing Point --- p.6Chapter 2.1.2 --- Metric Rectification --- p.8Chapter 2.2 --- Content Aware Image Resizing --- p.11Chapter 2.3 --- Texture Deformation --- p.15Chapter 2.3.1 --- Shape from texture --- p.16Chapter 2.3.2 --- Shape from lattice --- p.18Chapter 3 --- Resizing on Facade --- p.21Chapter 3.1 --- Introduction --- p.21Chapter 3.2 --- Related Work --- p.23Chapter 3.3 --- Algorithm --- p.24Chapter 3.3.1 --- Facade Detection --- p.25Chapter 3.3.2 --- Facade Resizing --- p.32Chapter 3.4 --- Results --- p.34Chapter 4 --- Cell Texture Editing --- p.42Chapter 4.1 --- Introduction --- p.42Chapter 4.2 --- Related Work --- p.44Chapter 4.3 --- Our Approach --- p.46Chapter 4.3.1 --- Cell Detection --- p.47Chapter 4.3.2 --- Local Affine Estimation --- p.49Chapter 4.3.3 --- Affine Transformation Field --- p.52Chapter 4.4 --- Photo Editing Applications --- p.55Chapter 4.5 --- Discussion --- p.58Chapter 5 --- Conclusion --- p.65Bibliography --- p.6

Analyse et synthèse de textures composées de motifs répétitifs

National audienceLes recherches présentées dans cet article visent à l'analyse et à la resynthèse de textures arborant une stochasticité qualifiée ici de "haut-niveau". De telles textures sont constituées d'une distribution quelconque de motifs de plusieurs sortes, se recouvrant partiellement. La distribution elle-même peut être aléatoire ou obéir à des règles de disposition géometrique. Entrent ainsi dans cette catégorie, arrangements de primitives 2d et textures quasi-régulières. La clé de voûte de l'approche proposée est d'extraire les motifs en exploitant leur répétitivité au sein de l'image. Il devient alors possible de s'affranchir des contraintes qu'impose une analyse markovienne locale à l'échelle des pixels et d'obtenir des composants pertinents permettant une analyse adaptée de l'image d'entrée

A Stochastic Grammar of Images

This exploratory paper quests for a stochastic and context sensitive grammar of images. The grammar should achieve the following four objectives and thus serves as a unified framework of representation, learning, and recognition for a large number of object categories. (i) The grammar represents both the hierarchical decompositions from scenes, to objects, parts, primitives and pixels by terminal and non-terminal nodes and the contexts for spatial and functional relations by horizontal links between the nodes. It formulates each object category as the set of all possible valid configurations produced by the grammar. (ii) The grammar is embodied in a simple And-Or graph representation where each Or-node points to alternative sub-configurations and an And-node is decomposed into a number of components. This representation supports recursive top-down/bottom-up procedures for image parsing under the Bayesian framework and make it convenient to scale up in complexity. Given an input image, the image parsing task constructs a most probable parse graph on-the-fly as the output interpretation and this parse graph is a subgraph of the And-Or graph after making choice on the Or-nodes. (iii) A probabilistic model is defined on this And-Or graph representation to account for the natural occurrence frequency of objects and parts as well as their relations. This model is learned from a relatively small training set per category and then sampled to synthesize a large number of configurations to cover novel object instances in the test set. This generalization capability is mostly missing in discriminative machine learning methods and can largely improve recognition performance in experiments. (iv) To fill the well-known semantic gap between symbols and raw signals, the grammar includes a series of visual dictionaries and organizes them through graph composition. At the bottom-level the dictionary is a set of image primitives each having a number of anchor points with open bonds to link with other primitives. These primitives can be combined to form larger and larger graph structures for parts and objects. The ambiguities in inferring local primitives shall be resolved through top-down computation using larger structures. Finally these primitives forms a primal sketch representation which will generate the input image with every pixels explained. The proposal grammar integrates three prominent representations in the literature: stochastic grammars for composition, Markov (or graphical) models for contexts, and sparse coding with primitives (wavelets). It also combines the structure-based and appearance based methods in the vision literature. Finally the paper presents three case studies to illustrate the proposed grammar.Mathematic

VISUAL SEMANTIC SEGMENTATION AND ITS APPLICATIONS

This dissertation addresses the difficulties of semantic segmentation when dealing with an extensive collection of images and 3D point clouds. Due to the ubiquity of digital cameras that help capture the world around us, as well as the advanced scanning techniques that are able to record 3D replicas of real cities, the sheer amount of visual data available presents many opportunities for both academic research and industrial applications. But the mere quantity of data also poses a tremendous challenge. In particular, the problem of distilling useful information from such a large repository of visual data has attracted ongoing interests in the fields of computer vision and data mining. Structural Semantics are fundamental to understanding both natural and man-made objects. Buildings, for example, are like languages in that they are made up of repeated structures or patterns that can be captured in images. In order to find these recurring patterns in images, I present an unsupervised frequent visual pattern mining approach that goes beyond co-location to identify spatially coherent visual patterns, regardless of their shape, size, locations and orientation. First, my approach categorizes visual items from scale-invariant image primitives with similar appearance using a suite of polynomial-time algorithms that have been designed to identify consistent structural associations among visual items, representing frequent visual patterns. After detecting repetitive image patterns, I use unsupervised and automatic segmentation of the identified patterns to generate more semantically meaningful representations. The underlying assumption is that pixels capturing the same portion of image patterns are visually consistent, while pixels that come from different backdrops are usually inconsistent. I further extend this approach to perform automatic segmentation of foreground objects from an Internet photo collection of landmark locations. New scanning technologies have successfully advanced the digital acquisition of large-scale urban landscapes. In addressing semantic segmentation and reconstruction of this data using LiDAR point clouds and geo-registered images of large-scale residential areas, I develop a complete system that simultaneously uses classification and segmentation methods to first identify different object categories and then apply category-specific reconstruction techniques to create visually pleasing and complete scene models

Méthodes de vision et d'intelligence artificielles pour la reconnaissance de spécimens coralliens

Ce mémoire traite de l'automatisation de l'extraction de données nécessaires à la caractérisation de la biodiversité d’espèces benthiques, une problématique environnementale d’actualité. L'impact des activités humaines sur la faune marine est une préoccupation grandissante. Les eaux des Philippines, qualifiées par plusieurs écologistes comme étant le « berceau de la biodiversité marine », sont comme tant d’autres sous la menace constante des changements climatiques, de la pêche invasive et de la pollution côtière. Notre sujet d'intérêt, les récifs coralliens, sont une des principales victimes de ces perturbations humaines. Le phénomène est largement reconnu, mais malheureusement aucune évaluation quantitative des dommages causés à la biodiversité ne peut être obtenue facilement. Les écologistes ont besoin d'outils, permettant de traiter en lot les données recueillies sur le terrain pour conclure sur le véritable impact de l'homme sur cet écosystème fragile. Dans ce contexte, le but de l’étude est de fournir une implémentation logicielle capable d’automatiser la détection et l’identification de spécimens coralliens dans une banque de photographies sous-marines. Pour ce faire, deux approches distinctes ont été mises à l’essai : l’identification des coraux en fonction de leur forme de croissance (la représentation par formes benthiques) et l’identification précise de l’espèce des spécimens (la représentation taxinomique). Pour l’identification par formes benthiques, divers outils de vision artificielle, tels les descripteurs issus des matrices de cooccurrence des tons de gris (GLCM), des motifs binaires locaux (LBP), de l’histogramme des tons et de la transformée de Fourier ont été mis à l’essai. Ces descripteurs ont été utilisés conjointement à divers algorithmes de reconnaissance de formes tels le classificateur bayesien, la méthode des plus proches voisins, l’arbre de classification C4.5, le séparateur à vaste marge (SVM) et le perceptron multicouches qui ont été comparés dans ce contexte de classification par formes benthiques. Dans un deuxième temps, l’expérimentation a été reprise sur le problème de classification taxinomique. Au cours de cette étude, la problématique de segmentation automatisée des spécimens de corail a été abordée. Une présegmentation par laplacien du gaussien et partage des eaux, suivie d’une classification des segments avec un SVM optimisé, utilisant les descripteurs développés précédemment, se sont avérées une solution efficace à ce problème. Par la suite, les segments appartenant au corail (et non à l’arrière plan) ont été classifiés à l’aide d’un autre SVM entraîné à cet effet. Les résultats obtenus démontrent qu’il est possible d’implémenter un tel système de reconnaissance automatisée du benthos. Toutefois, plusieurs conditions devront être respectées pour en assurer le succès : (1) pour être robuste, le système de segmentation et de classification de corail doit être entraîné avec un nombre suffisant d'échantillons et (2) le protocole d'acquisition des données et d’extraction de descripteurs doit être bien défini pour maximiser les performances