Sublinear algorithms for Earth Mover's Distance

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2009.Includes bibliographical references (p. 14-15).We study the problem of estimating the Earth Mover's Distance (EMD) between probability distributions when given access only to samples. We give closeness testers and additive-error estimators over domains in [0, [delta]]d, with sample complexities independent of domain size - permitting the testability even of continuous distributions over infinite domains. Instead, our algorithms depend on other parameters, such as the diameter of the domain space, which may be significantly smaller. We also prove lower bounds showing our testers to be optimal in their dependence on these parameters. Additionally, we consider whether natural classes of distributions exist for which there are algorithms with better dependence on the dimension, and show that for highly clusterable data, this is indeed the case. Lastly, we consider a variant of the EMD, defined over tree metrics instead of the usual L₁ metric, and give optimal algorithms.by Khanh Do Ba.S.M

Fast Retrieval Algorithm Using EMD Lower and Upper Bounds and a Search Algorithm in multidimensional index

Comparison of images requires a distance metric that is sensitive to the spatial location of objects and features. The Earth Mover’s Distance was introduced in Computer Vision to better approach human perceptual similarities. Its computation, however, is too complex for usage in interactive multimedia database scenarios. We develop new upper bounding approximation techniques for the Earth Mover’s Distance which satisfy high quality criteria and fast computation. In order to enable efficient query processing in large databases, we propose an index structure LUBMTree (Lower and Upper Bounds MTree), based of using the lower and upper bounds for the EMD to improve the search time. Experiments show the performance of research in the  LUBMTree compared with those obtained by  the research in the MTree. Keywords : indexing, similarity, search, signature, metric EMD, MTree, MAM

Lower bounds for embedding the Earth Mover Distance metric into normed spaces

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2005.Includes bibliographical references (p. 71-73).This thesis presents a lower bounds for embedding the Earth Mover Distance (EMID) metric into normed spaces. The EMID is a metric over two distributions where one is a mass of earth spread out in space and the other is a collection of holes in that same space. The EMD between these two distributions is defined as the least amount of work needed to fill the holes with earth. The EMD metric is used in a number of applications, for example in similarity searching and for image retrieval. We present a simple construction of point sets in the ENID metric space over two dimensions that cannot be embedded from the ED metric exactly into normed spaces, namely l1 and the square of l2. An embedding is a mapping f : X --> V with X a set of points in a metric space and ' Va set of points in some normed vector space. When the Manhattan distance is used as the underlying metric for the EMD, it can be shown that this example is isometric to K2,4 which has distortion equal to 1.25 when it is embedded into I and( 1.1180 when embedded into the square of 12. Other constructions of points sets in the EMID metric space over three and higher dimensisions are also discussed..by Javed K.K. Samuel.M.Eng

Learning and inference with Wasserstein metrics

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences, 2018.Cataloged from PDF version of thesis.Includes bibliographical references (pages 131-143).This thesis develops new approaches for three problems in machine learning, using tools from the study of optimal transport (or Wasserstein) distances between probability distributions. Optimal transport distances capture an intuitive notion of similarity between distributions, by incorporating the underlying geometry of the domain of the distributions. Despite their intuitive appeal, optimal transport distances are often difficult to apply in practice, as computing them requires solving a costly optimization problem. In each setting studied here, we describe a numerical method that overcomes this computational bottleneck and enables scaling to real data. In the first part, we consider the problem of multi-output learning in the presence of a metric on the output domain. We develop a loss function that measures the Wasserstein distance between the prediction and ground truth, and describe an efficient learning algorithm based on entropic regularization of the optimal transport problem. We additionally propose a novel extension of the Wasserstein distance from probability measures to unnormalized measures, which is applicable in settings where the ground truth is not naturally expressed as a probability distribution. We show statistical learning bounds for both the Wasserstein loss and its unnormalized counterpart. The Wasserstein loss can encourage smoothness of the predictions with respect to a chosen metric on the output space. We demonstrate this property on a real-data image tagging problem, outperforming a baseline that doesn't use the metric. In the second part, we consider the probabilistic inference problem for diffusion processes. Such processes model a variety of stochastic phenomena and appear often in continuous-time state space models. Exact inference for diffusion processes is generally intractable. In this work, we describe a novel approximate inference method, which is based on a characterization of the diffusion as following a gradient flow in a space of probability densities endowed with a Wasserstein metric. Existing methods for computing this Wasserstein gradient flow rely on discretizing the underlying domain of the diffusion, prohibiting their application to problems in more than several dimensions. In the current work, we propose a novel algorithm for computing a Wasserstein gradient flow that operates directly in a space of continuous functions, free of any underlying mesh. We apply our approximate gradient flow to the problem of filtering a diffusion, showing superior performance where standard filters struggle. Finally, we study the ecological inference problem, which is that of reasoning from aggregate measurements of a population to inferences about the individual behaviors of its members. This problem arises often when dealing with data from economics and political sciences, such as when attempting to infer the demographic breakdown of votes for each political party, given only the aggregate demographic and vote counts separately. Ecological inference is generally ill-posed, and requires prior information to distinguish a unique solution. We propose a novel, general framework for ecological inference that allows for a variety of priors and enables efficient computation of the most probable solution. Unlike previous methods, which rely on Monte Carlo estimates of the posterior, our inference procedure uses an efficient fixed point iteration that is linearly convergent. Given suitable prior information, our method can achieve more accurate inferences than existing methods. We additionally explore a sampling algorithm for estimating credible regions.by Charles Frogner.Ph. D

Matching sets of features for efficient retrieval and recognition

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.Includes bibliographical references (p. 145-153).In numerous domains it is useful to represent a single example by the collection of local features or parts that comprise it. In computer vision in particular, local image features are a powerful way to describe images of objects and scenes. Their stability under variable image conditions is critical for success in a wide range of recognition and retrieval applications. However, many conventional similarity measures and machine learning algorithms assume vector inputs. Comparing and learning from images represented by sets of local features is therefore challenging, since each set may vary in cardinality and its elements lack a meaningful ordering. In this thesis I present computationally efficient techniques to handle comparisons, learning, and indexing with examples represented by sets of features. The primary goal of this research is to design and demonstrate algorithms that can effectively accommodate this useful representation in a way that scales with both the representation size as well as the number of images available for indexing or learning. I introduce the pyramid match algorithm, which efficiently forms an implicit partial matching between two sets of feature vectors.(cont.) The matching has a linear time complexity, naturally forms a Mercer kernel, and is robust to clutter or outlier features, a critical advantage for handling images with variable backgrounds, occlusions, and viewpoint changes. I provide bounds on the expected error relative to the optimal partial matching. For very large databases, even extremely efficient pairwise comparisons may not offer adequately responsive query times. I show how to perform sub-linear time retrievals under the matching measure with randomized hashing techniques, even when input sets have varying numbers of features. My results are focused on several important vision tasks, including applications to content-based image retrieval, discriminative classification for object recognition, kernel regression, and unsupervised learning of categories. I show how the dramatic increase in performance enables accurate and flexible image comparisons to be made on large-scale data sets, and removes the need to artificially limit the number of local descriptions used per image when learning visual categories.by Kristen Lorraine Grauman.Ph.D

Machine Annotation of Traditional Irish Dance Music

The work presented in this thesis is validated in experiments using 130 realworld field recordings of traditional music from sessions, classes, concerts and commercial recordings. Test audio includes solo and ensemble playing on a variety of instruments recorded in real-world settings such as noisy public sessions. Results are reported using standard measures from the field of information retrieval (IR) including accuracy, error, precision and recall and the system is compared to alternative approaches for CBMIR common in the literature

Wasserstein β-Diversity Metrics over Graphs: Derivation, Efficient Computation and Applications

Microbial ecology has been transformed by metagenomics, the study of the genetic in-formation in entire communities of organisms. In the following we develop metagenomic tools arising from the classic Wasserstein metric as applied to questions regarding the diversity between microbial communities. We provide a novel proof of the characteriza-tion of the successful UniFrac metric as the Wasserstein metric over a graph-theoretic tree, and use the proof to develop an extremely e°cient computational algorithm. The analytic framework we develop is then leveraged to provide formulations for the distri-bution of this Wasserstein based metric. We implement these ideas and demonstrate their utility on realworld datasets. We next turn to applying the Wasserstein metric as a reference-free diversity metric by utilizing de Bruijn graphs, mathematical structures at the heart of genome assembly techniques. We show how these techniques are related to established phylogenetically-aware diversity metrics. We then implement our results using newly developed approximation techniques for the computation of the Wasserstein metric and demonstrate this novel metric's utility in comparison to established metrics

In-situ Plasma Analysis of Ion Kinetics in the Solar Wind and Hermean Magnetosphere

The heating of the solar wind and its interaction with the unique planetary magnetosphere of Mercury is the primary focus of this work. The first aspect of this study focused on the heavy ion population of the solar wind (A > 4 amu), and how well the signature of the heating process responsible for creating the solar wind is preserved in this heavy ion population. We found that this signature in the heavy ion population is primarily erased (thermalized) via Coulomb collisional interactions with solar wind protons. The heavy ions observed in collisionally young solar wind reveal a clear, stable dependence on mass, along with non-thermal heating that is not in agreement with current predictions based on turbulent transport and kinetic dissipation. Due to its weak magnetic dipole, the solar wind can impinge on the surface of Mercury, one of the processes contributing to the desorption of neutrals and, through ionization, ions that make up the planet’s exosphere. Differentiating between surface mechanisms and analyzing magnetospheric plasma dynamics requires the quantification of a variety of ion species. A detailed forward model and a robust statistical method were created to identify new ion signatures in the measurement space of the FIPS instrument, formerly orbiting Mercury onboard the MESSENGER spacecraft. The recovery of new heavy ions species, including Al, Ne, Si, and Mg, along with tentative recoveries of S, Ar, K, and C, enable in depth studies of the plasma dynamics in the Hermean magnetosphere. The interaction of the solar wind with the bow shock of the Hermean magnetosphere leads to the creation of a foreshock region. New tools and methods were created to enable the analysis of the diffuse and Field Aligned Beam (FAB) populations in unique parameter regime of the Hermean foreshock. One result suggests that the energization process for the observed FABs can be explained by Shock Drift Acceleration, and not limited by the small spatial size of Mercury’s bow shock. Analysis of diffuse populations shows that a connection time limited diffusive shock acceleration is likely responsible for the behavior of the observed energy distributions.PHDAtmospheric, Oceanic & Space ScienceUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/135753/1/ptracy_1.pd