Adaptive constrained clustering with application to dynamic image database categorization and visualization.

The advent of larger storage spaces, affordable digital capturing devices, and an ever growing online community dedicated to sharing images has created a great need for efficient analysis methods. In fact, analyzing images for the purpose of automatic categorization and retrieval is quickly becoming an overwhelming task even for the casual user. Initially, systems designed for these applications relied on contextual information associated with images. However, it was realized that this approach does not scale to very large data sets and can be subjective. Then researchers proposed methods relying on the content of the images. This approach has also proved to be limited due to the semantic gap between the low-level representation of the image and the high-level user perception. In this dissertation, we introduce a novel clustering technique that is designed to combine multiple forms of information in order to overcome the disadvantages observed while using a single information domain. Our proposed approach, called Adaptive Constrained Clustering (ACC), is a robust, dynamic, and semi-supervised algorithm. It is based on minimizing a single objective function incorporating the abilities to: (i) use multiple feature subsets while learning cluster independent feature relevance weights; (ii) search for the optimal number of clusters; and (iii) incorporate partial supervision in the form of pairwise constraints. The content of the images is used to extract the features used in the clustering process. The context information is used in constructing a set of appropriate constraints. These constraints are used as partial supervision information to guide the clustering process. The ACC algorithm is dynamic in the sense that the number of categories are allowed to expand and contract depending on the distribution of the data and the available set of constraints. We show that the proposed ACC algorithm is able to partition a given data set into meaningful clusters using an adaptive, soft constraint satisfaction methodology for the purpose of automatically categorizing and summarizing an image database. We show that the ACC algorithm has the ability to incorporate various types of contextual information. This contextual information includes: spatial information provided by geo-referenced images that include GPS coordinates pinpointing their location, temporal information provided by each image\u27s time stamp indicating the capture time, and textual information provided by a set of keywords describing the semantics of the associated images

Semantic image retrieval using relevance feedback and transaction logs

Due to the recent improvements in digital photography and storage capacity, storing large amounts of images has been made possible, and efficient means to retrieve images matching a user’s query are needed. Content-based Image Retrieval (CBIR) systems automatically extract image contents based on image features, i.e. color, texture, and shape. Relevance feedback methods are applied to CBIR to integrate users’ perceptions and reduce the gap between high-level image semantics and low-level image features. The precision of a CBIR system in retrieving semantically rich (complex) images is improved in this dissertation work by making advancements in three areas of a CBIR system: input, process, and output. The input of the system includes a mechanism that provides the user with required tools to build and modify her query through feedbacks. Users behavioral in CBIR environments are studied, and a new feedback methodology is presented to efficiently capture users’ image perceptions. The process element includes image learning and retrieval algorithms. A Long-term image retrieval algorithm (LTL), which learns image semantics from prior search results available in the system’s transaction history, is developed using Factor Analysis. Another algorithm, a short-term learner (STL) that captures user’s image perceptions based on image features and user’s feedbacks in the on-going transaction, is developed based on Linear Discriminant Analysis. Then, a mechanism is introduced to integrate these two algorithms to one retrieval procedure. Finally, a retrieval strategy that includes learning and searching phases is defined for arranging images in the output of the system. The developed relevance feedback methodology proved to reduce the effect of human subjectivity in providing feedbacks for complex images. Retrieval algorithms were applied to images with different degrees of complexity. LTL is efficient in extracting the semantics of complex images that have a history in the system. STL is suitable for query and images that can be effectively represented by their image features. Therefore, the performance of the system in retrieving images with visual and conceptual complexities was improved when both algorithms were applied simultaneously. Finally, the strategy of retrieval phases demonstrated promising results when the query complexity increases

Swarm intelligence: novel tools for optimization, feature extraction, and multi-agent system modeling

Abstract Animal swarms in nature are able to adapt to dynamic changes in their envi-ronment, and through cooperation they can solve problems that are crucial for their survival. Only by means of local interactions with other members of the swarm and with the environment, they can achieve a common goal more efficiently than it would be done by a single individual. This problem-solving behavior that results from the multiplicity of such interactions is referred to as Swarm Intelligence. The mathematical models of swarming behavior in nature were initially proposed to solve optimization problems. Nevertheless, this decentralized approach can be a valuable tool for a variety of applications, where emerging global patterns represent a solution to the task at hand. Methods for the solution of difficult computational problems based on Swarm Intelligence have been experimentally demonstrated and reported in the literature. However, a general framework that would facilitate their design does not exist yet. In this dissertation, a new general design methodology for Swarm Intelligence tools is proposed. By defining a discrete space in which the members of the swarm can move, and by modifying the rules of local interactions and setting the adequate objective function for solutions evaluation, the proposed methodology is tested in various domains. The dissertation presents a set of case studies, and focuses on two general approaches. One approach is to apply Swarm Intelligence as a tool for optimization and feature extraction, and the other approach is to model multi-agent systems such that they resemble swarms of animals in nature providing them with the ability to autonomously perform a task at hand. Artificial swarms are designed to be autonomous, scalable, robust, and adaptive to the changes in their environment. In this work, the methods that exploit one or more of these features are presented. First, the proposed methodology is validated in a real-world scenario seen as a combinatorial optimization problem. Then a set of novel tools for feature extraction, more precisely the adaptive edge detection and the broken-edge linking in digital images is proposed. A novel data clustering algorithm is also proposed and applied to image segmentation. Finally, a scalable algorithm based on the proposed methodology is developed for distributed task allocation in multi-agent systems, and applied to a swarm of robots. The newly proposed general methodology provides a guideline for future developers of the Swarm Intelligence tools. Los enjambres de animales en la naturaleza son capaces de adaptarse a cambios dinamicos en su entorno y, por medio de la cooperación, pueden resolver problemas ´ cruciales para su supervivencia. Unicamente por medio de interacciones locales con otros miembros del enjambre y con el entorno, pueden lograr un objetivo común de forma más eficiente que lo haría un solo individuo. Este comportamiento problema-resolutivo que es resultado de la multiplicidad de interacciones se denomina Inteligencia de Enjambre. Los modelos matemáticos de comportamiento de enjambres en entornos naturales fueron propuestos inicialmente para resolver problemas de optimización. Sin embargo, esta aproximación descentralizada puede ser una herramienta valiosa en una variedad de aplicaciones donde patrones globales emergentes representan una solución de las tareas actuales. Aunque en la literatura se muestra la utilidad de los métodos de Inteligencia de Enjambre, no existe un entorno de trabajo que facilite su diseño. En esta memoria de tesis proponemos una nueva metodologia general de diseño para herramientas de Inteligencia de Enjambre. Desarrollamos herramientas noveles que representan ejem-plos ilustrativos de su implementación. Probamos la metodología propuesta en varios dominios definiendo un espacio discreto en el que los miembros del enjambre pueden moverse, modificando las reglas de las interacciones locales y fijando la función objetivo adecuada para evaluar las soluciones. La memoria de tesis presenta un conjunto de casos de estudio y se centra en dos aproximaciones generales. Una aproximación es aplicar Inteligencia de Enjambre como herramienta de optimización y extracción de características mientras que la otra es modelar sistemas multi-agente de tal manera que se asemejen a enjambres de animales en la naturaleza a los que se les confiere la habilidad de ejecutar autónomamente la tarea. Los enjambres artificiales están diseñados para ser autónomos, escalables, robustos y adaptables a los cambios en su entorno. En este trabajo, presentamos métodos que explotan una o más de estas características. Primero, validamos la metodología propuesta en un escenario del mundo real visto como un problema de optimización combinatoria. Después, proponemos un conjunto de herramientas noveles para ex-tracción de características, en concreto la detección adaptativa de bordes y el enlazado de bordes rotos en imágenes digitales, y el agrupamiento de datos para segmentación de imágenes. Finalmente, proponemos un algoritmo escalable para la asignación distribuida de tareas en sistemas multi-agente aplicada a enjambres de robots. La metodología general recién propuesta ofrece una guía para futuros desarrolladores deherramientas de Inteligencia de Enjambre

Bridging the semantic gap in content-based image retrieval.

To manage large image databases, Content-Based Image Retrieval (CBIR) emerged as a new research subject. CBIR involves the development of automated methods to use visual features in searching and retrieving. Unfortunately, the performance of most CBIR systems is inherently constrained by the low-level visual features because they cannot adequately express the user\u27s high-level concepts. This is known as the semantic gap problem. This dissertation introduces a new approach to CBIR that attempts to bridge the semantic gap. Our approach includes four components. The first one learns a multi-modal thesaurus that associates low-level visual profiles with high-level keywords. This is accomplished through image segmentation, feature extraction, and clustering of image regions. The second component uses the thesaurus to annotate images in an unsupervised way. This is accomplished through fuzzy membership functions to label new regions based on their proximity to the profiles in the thesaurus. The third component consists of an efficient and effective method for fusing the retrieval results from the multi-modal features. Our method is based on learning and adapting fuzzy membership functions to the distribution of the features\u27 distances and assigning a degree of worthiness to each feature. The fourth component provides the user with the option to perform hybrid querying and query expansion. This allows the enrichment of a visual query with textual data extracted from the automatically labeled images in the database. The four components are integrated into a complete CBIR system that can run in three different and complementary modes. The first mode allows the user to query using an example image. The second mode allows the user to specify positive and/or negative sample regions that should or should not be included in the retrieved images. The third mode uses a Graphical Text Interface to allow the user to browse the database interactively using a combination of low-level features and high-level concepts. The proposed system and ail of its components and modes are implemented and validated using a large data collection for accuracy, performance, and improvement over traditional CBIR techniques

Unsupervised tracking of time-evolving data streams and an application to short-term urban traffic flow forecasting

I am indebted to many people for their help and support I receive during my Ph.D. study and research at DIBRIS-University of Genoa. First and foremost, I would like to express my sincere thanks to my supervisors Prof.Dr. Masulli, and Prof.Dr. Rovetta for the invaluable guidance, frequent meetings, and discussions, and the encouragement and support on my way of research. I thanks all the members of the DIBRIS for their support and kindness during my 4 years Ph.D. I would like also to acknowledge the contribution of the projects Piattaforma per la mobili\ue0 Urbana con Gestione delle INformazioni da sorgenti eterogenee (PLUG-IN) and COST Action IC1406 High Performance Modelling and Simulation for Big Data Applications (cHiPSet). Last and most importantly, I wish to thanks my family: my wife Shaimaa who stays with me through the joys and pains; my daughter and son whom gives me happiness every-day; and my parents for their constant love and encouragement

Residual-Sparse Fuzzy CC-Means Clustering Incorporating Morphological Reconstruction and Wavelet frames

Instead of directly utilizing an observed image including some outliers, noise or intensity inhomogeneity, the use of its ideal value (e.g. noise-free image) has a favorable impact on clustering. Hence, the accurate estimation of the residual (e.g. unknown noise) between the observed image and its ideal value is an important task. To do so, we propose an $\ell_0$ regularization-based Fuzzy $C$-Means (FCM) algorithm incorporating a morphological reconstruction operation and a tight wavelet frame transform. To achieve a sound trade-off between detail preservation and noise suppression, morphological reconstruction is used to filter an observed image. By combining the observed and filtered images, a weighted sum image is generated. Since a tight wavelet frame system has sparse representations of an image, it is employed to decompose the weighted sum image, thus forming its corresponding feature set. Taking it as data for clustering, we present an improved FCM algorithm by imposing an $\ell_0$ regularization term on the residual between the feature set and its ideal value, which implies that the favorable estimation of the residual is obtained and the ideal value participates in clustering. Spatial information is also introduced into clustering since it is naturally encountered in image segmentation. Furthermore, it makes the estimation of the residual more reliable. To further enhance the segmentation effects of the improved FCM algorithm, we also employ the morphological reconstruction to smoothen the labels generated by clustering. Finally, based on the prototypes and smoothed labels, the segmented image is reconstructed by using a tight wavelet frame reconstruction operation. Experimental results reported for synthetic, medical, and color images show that the proposed algorithm is effective and efficient, and outperforms other algorithms.Comment: 12 pages, 11 figur