Large-scale image collection cleansing, summarization and exploration

A perennially interesting topic in the research field of large scale image collection organization is how to effectively and efficiently conduct the tasks of image cleansing, summarization and exploration. The primary objective of such an image organization system is to enhance user exploration experience with redundancy removal and summarization operations on large-scale image collection. An ideal system is to discover and utilize the visual correlation among the images, to reduce the redundancy in large-scale image collection, to organize and visualize the structure of large-scale image collection, and to facilitate exploration and knowledge discovery. In this dissertation, a novel system is developed for exploiting and navigating large-scale image collection. Our system consists of the following key components: (a) junk image filtering by incorporating bilingual search results; (b) near duplicate image detection by using a coarse-to-fine framework; (c) concept network generation and visualization; (d) image collection summarization via dictionary learning for sparse representation; and (e) a multimedia practice of graffiti image retrieval and exploration. For junk image filtering, bilingual image search results, which are adopted for the same keyword-based query, are integrated to automatically identify the clusters for the junk images and the clusters for the relevant images. Within relevant image clusters, the results are further refined by removing the duplications under a coarse-to-fine structure. The duplicate pairs are detected with both global feature (partition based color histogram) and local feature (CPAM and SIFT Bag-of-Word model). The duplications are detected and removed from the data collection to facilitate further exploration and visual correlation analysis. After junk image filtering and duplication removal, the visual concepts are further organized and visualized by the proposed concept network. An automatic algorithm is developed to generate such visual concept network which characterizes the visual correlation between image concept pairs. Multiple kernels are combined and a kernel canonical correlation analysis algorithm is used to characterize the diverse visual similarity contexts between the image concepts. The FishEye visualization technique is implemented to facilitate the navigation of image concepts through our image concept network. To better assist the exploration of large scale data collection, we design an efficient summarization algorithm to extract representative examplars. For this collection summarization task, a sparse dictionary (a small set of the most representative images) is learned to represent all the images in the given set, e.g., such sparse dictionary is treated as the summary for the given image set. The simulated annealing algorithm is adopted to learn such sparse dictionary (image summary) by minimizing an explicit optimization function. In order to handle large scale image collection, we have evaluated both the accuracy performance of the proposed algorithms and their computation efficiency. For each of the above tasks, we have conducted experiments on multiple public available image collections, such as ImageNet, NUS-WIDE, LabelMe, etc. We have observed very promising results compared to existing frameworks. The computation performance is also satisfiable for large-scale image collection applications. The original intention to design such a large-scale image collection exploration and organization system is to better service the tasks of information retrieval and knowledge discovery. For this purpose, we utilize the proposed system to a graffiti retrieval and exploration application and receive positive feedback

Learning and inference with Wasserstein metrics

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences, 2018.Cataloged from PDF version of thesis.Includes bibliographical references (pages 131-143).This thesis develops new approaches for three problems in machine learning, using tools from the study of optimal transport (or Wasserstein) distances between probability distributions. Optimal transport distances capture an intuitive notion of similarity between distributions, by incorporating the underlying geometry of the domain of the distributions. Despite their intuitive appeal, optimal transport distances are often difficult to apply in practice, as computing them requires solving a costly optimization problem. In each setting studied here, we describe a numerical method that overcomes this computational bottleneck and enables scaling to real data. In the first part, we consider the problem of multi-output learning in the presence of a metric on the output domain. We develop a loss function that measures the Wasserstein distance between the prediction and ground truth, and describe an efficient learning algorithm based on entropic regularization of the optimal transport problem. We additionally propose a novel extension of the Wasserstein distance from probability measures to unnormalized measures, which is applicable in settings where the ground truth is not naturally expressed as a probability distribution. We show statistical learning bounds for both the Wasserstein loss and its unnormalized counterpart. The Wasserstein loss can encourage smoothness of the predictions with respect to a chosen metric on the output space. We demonstrate this property on a real-data image tagging problem, outperforming a baseline that doesn't use the metric. In the second part, we consider the probabilistic inference problem for diffusion processes. Such processes model a variety of stochastic phenomena and appear often in continuous-time state space models. Exact inference for diffusion processes is generally intractable. In this work, we describe a novel approximate inference method, which is based on a characterization of the diffusion as following a gradient flow in a space of probability densities endowed with a Wasserstein metric. Existing methods for computing this Wasserstein gradient flow rely on discretizing the underlying domain of the diffusion, prohibiting their application to problems in more than several dimensions. In the current work, we propose a novel algorithm for computing a Wasserstein gradient flow that operates directly in a space of continuous functions, free of any underlying mesh. We apply our approximate gradient flow to the problem of filtering a diffusion, showing superior performance where standard filters struggle. Finally, we study the ecological inference problem, which is that of reasoning from aggregate measurements of a population to inferences about the individual behaviors of its members. This problem arises often when dealing with data from economics and political sciences, such as when attempting to infer the demographic breakdown of votes for each political party, given only the aggregate demographic and vote counts separately. Ecological inference is generally ill-posed, and requires prior information to distinguish a unique solution. We propose a novel, general framework for ecological inference that allows for a variety of priors and enables efficient computation of the most probable solution. Unlike previous methods, which rely on Monte Carlo estimates of the posterior, our inference procedure uses an efficient fixed point iteration that is linearly convergent. Given suitable prior information, our method can achieve more accurate inferences than existing methods. We additionally explore a sampling algorithm for estimating credible regions.by Charles Frogner.Ph. D

Asynchronous distributed clustering algorithms for wireless mesh network

Wireless Mesh Networks are becoming increasingly important in many applications. In many cases, data is acquired by devices that are distributed in space, but effective actions require a global view of that data across all devices. Transmitting all data to the centre allows strong data analytics algorithms to be applied, but consumes battery power for the nodes, and may cause data overload. To avoid this, distributed methods try to learn within the network, allowing each agent to learn a global picture and take appropriate actions. For distributed clustering in particular, existing methods share simple cluster descriptions until the network stabilises. The approaches tend to require either synchronous behaviour or many cycles, and omit important information about the clusters. In this thesis, we develop asynchronous methods that share richer cluster models, and we show that they are more effective in learning the global data patterns. Our underlying method describes the shape and density of each cluster, as well as its centroid and size. We combine cluster models by re-sampling from received models, and then re-clustering the new data sets. We then extend the approach, to allowing clustering methods that do not require the final number of clusters as input. After that, we consider the cases that there might be sub-groups of agents that are receiving different patterns of data. Finally, we extend our approaches to scenarios where each agent has no idea about whether there is a single pattern or are multiple patterns. We demonstrate that the approaches can regenerate clusters that are similar to the distributions that were used to create the test data. When the number of clusters are not available, the learned number of clusters is close to the ground truth. The proposed algorithms can learn how data points are clustered even when there are multiple patterns in the network. When the number of patterns (single or multiple) is not known in advance, the proposed methods Optimised KDE and DBSCAN preform well in detecting multiple patterns. Although they perform worse in detecting the single pattern, they can still learn how data points are clustered

High latitude Gondwanan famennian biodiversity patterns : evidence from the South African Witpoort Formation (Cape Supergroup, Witteberg Group)

Ph.D. Faculty of Science, University of the Witwatersrand, 2011Reassessment of the stratigraphic position of the Waterloo Farm black shale from Grahamstown, South Africa, revealed that it is situated in the uppermost Witpoort Formation, as opposed to the middle of the Witpoort Formation as previously reported. This argillaceous unit appears to be contemporaneous with globally correlated black anaerobic sediments intimately associated with the Hangenberg Extinction, the final and most important pulse of the end Devonian extinction event. The Waterloo Farm fauna is one of only seven significant faunas from the end Famennian, and one of only two from Gondwana. The other one, from Morocco, was situated in the palaeotropics of northern Gondwana whereas Waterloo Farm, situated near the palaeo South Pole, provides the only high latitude locality. Extensive fieldwork resulted in 511 catalogued fossil fish specimens. These comprise at least 21 taxa of which least 2 are agnathan, 7 placoderm, 4 acanthodian, 2 chondrichthyan, 1 actinopterygian and 5 sarcopterygian. Sarcopterygians include an onychodont, a coelacanth, a tristichopterid and an isolated cleithrPriscomyzon riniensis, the oldest lamprey, exhibits many of the key specialisations of modern lampreys including a large oral disc, circumoral teeth and a branchial basket. Analysis of Priscomyzon revealed that lampreys are ancient specialists that, having acquired key specialisations before the end of the Devonian period, survived with relatively little change for 360 million years. Shark fossils include Antarctilamna ultima (sp. nov.), a new species of a Gondwanan genus previously considered to have gone extinct before the late Devonian, and Plesioselachus doryssa. These taxa are basal to the crowngroup chondrichthyan radiation and provide insight into the primitive condition of chondrichthyans. A new coelacanth species, Paradiplocercides kowiensis (gen. et sp. nov.), represents one of the most completely preserved early coelacanths and offers insights into the early diversification of coelacanths, and sequences of morphologicalum of an advanced stem group tetrapodomorph close to the elpistostegalian grade. Priscomyzon riniensis, the oldest lamprey, exhibits many of the key specialisations of modern lampreys including a large oral disc, circumoral teeth and a branchial basket. Analysis of Priscomyzon revealed that lampreys are ancient specialists that, having acquired key specialisations before the end of the Devonian period, survived with relatively little change for 360 million years. Shark fossils include Antarctilamna ultima (sp. nov.), a new species of a Gondwanan genus previously considered to have gone extinct before the late Devonian, and Plesioselachus doryssa. These taxa are basal to the crowngroup chondrichthyan radiation and provide insight into the primitive condition of chondrichthyans. A new coelacanth species, Paradiplocercides kowiensis (gen. et sp. nov.), represents one of the most completely preserved early coelacanths and offers insights into the early diversification of coelacanths, and sequences of morphological changes in the early part of the coelacanth phylogenetic tree. Analyses of relative abundance of taxa at Waterloo Farm demonstrate a significant taphonomic filter in favour of organisms with numerous large bony elements and the resultant inappropriateness of extrapolating population structure from conventional methodologies. Exclusion of specimens derived from hard tissue alone, as well as those from single taxon death assemblages, produced a result more likely to reflect population structure, being more consistent with extrapolated trophic levels. Comparison of the Waterloo Farm fauna fossils with those from the earlier Devonian Bokkeveld Group and overlying lower Cindicates a distinctive Agulhas Sea faunal province. The Agulhas Sea fauna is the highest latitude Devonian faunal region, having existed, in a near polar setting, in the semi enclosed Agulhas Sea. This fauna inherited much of its diversity from a mid Devonian Agulhas Sea fauna characterised by Gondwanan endemic sharks, gyracanthid acanthodians and phlyctaeniid arthrodire placoderms, but lacking many taxa, which characterise other mid Devonian Gondwanan successions. The approach of Laurussia to Gondwana towards the end of the Devonian permitted an exchange of marginal marine taxa, which were previously separated by deep oceans with anoxic bottom waters. Together with moderation of global climatic gradients, this allowed augmentation of the mid Devonian relict population inhabiting the Agulhas Sea, during the Late Devonian. New faunal elements from Laurussia and eastern Gondwana resulted in a diverse, though unique, fauna with many characteristic Late Devonian taxonomic groups incapable of penetrating this high latitude environment. The Aguhlas Sea fauna was nonetheless subject to exactly the same endarboniferous Witteberg Group, as well as published records from parts of South America and Antarctica that also bounded the Agulhas Sea during this time, Devonian extinction profile as tropical coastal and temperate deep-sea environments. The abrupt nature of this event, at the end of the Famennian, is evidenced by the presence of various taxa from Waterloo Farm, formerly thought to have gone extinct before the Famennian. The Agulhas shark, Plesioselachus and the acanthodian Gyracanthides were the only members of this fauna to survive the Hangenberg extinction event. During the Carboniferous the Agulhas Sea was repopulated by a diverse actinopterygian fauna with Laurussian affinities