5,588 research outputs found
Total Jensen divergences: Definition, Properties and k-Means++ Clustering
We present a novel class of divergences induced by a smooth convex function
called total Jensen divergences. Those total Jensen divergences are invariant
by construction to rotations, a feature yielding regularization of ordinary
Jensen divergences by a conformal factor. We analyze the relationships between
this novel class of total Jensen divergences and the recently introduced total
Bregman divergences. We then proceed by defining the total Jensen centroids as
average distortion minimizers, and study their robustness performance to
outliers. Finally, we prove that the k-means++ initialization that bypasses
explicit centroid computations is good enough in practice to guarantee
probabilistically a constant approximation factor to the optimal k-means
clustering.Comment: 27 page
Will the US Economy Recover in 2010? A Minimal Spanning Tree Study
We calculated the cross correlations between the half-hourly times series of
the ten Dow Jones US economic sectors over the period February 2000 to August
2008, the two-year intervals 2002--2003, 2004--2005, 2008--2009, and also over
11 segments within the present financial crisis, to construct minimal spanning
trees (MSTs) of the US economy at the sector level. In all MSTs, a core-fringe
structure is found, with consumer goods, consumer services, and the industrials
consistently making up the core, and basic materials, oil and gas, healthcare,
telecommunications, and utilities residing predominantly on the fringe. More
importantly, we find that the MSTs can be classified into two distinct,
statistically robust, topologies: (i) star-like, with the industrials at the
center, associated with low-volatility economic growth; and (ii) chain-like,
associated with high-volatility economic crisis. Finally, we present
statistical evidence, based on the emergence of a star-like MST in Sep 2009,
and the MST staying robustly star-like throughout the Greek Debt Crisis, that
the US economy is on track to a recovery.Comment: elsarticle class, includes amsmath.sty, graphicx.sty and url.sty. 68
pages, 16 figures, 8 tables. Abridged version of the manuscript presented at
the Econophysics Colloquim 2010, incorporating reviewer comment
Stabilizing Training of Generative Adversarial Networks through Regularization
Deep generative models based on Generative Adversarial Networks (GANs) have
demonstrated impressive sample quality but in order to work they require a
careful choice of architecture, parameter initialization, and selection of
hyper-parameters. This fragility is in part due to a dimensional mismatch or
non-overlapping support between the model distribution and the data
distribution, causing their density ratio and the associated f-divergence to be
undefined. We overcome this fundamental limitation and propose a new
regularization approach with low computational cost that yields a stable GAN
training procedure. We demonstrate the effectiveness of this regularizer across
several architectures trained on common benchmark image generation tasks. Our
regularization turns GAN models into reliable building blocks for deep
learning
Bootstrap methods for the empirical study of decision-making and information flows in social systems
Abstract: We characterize the statistical bootstrap for the estimation of information theoretic quantities from data, with particular reference to its use in the study of large-scale social phenomena. Our methods allow one to preserve, approximately, the underlying axiomatic relationships of information theoryâin particular, consistency under arbitrary coarse-grainingâthat motivate use of these quantities in the first place, while providing reliability comparable to the state of the art for Bayesian estimators. We show how information-theoretic quantities allow for rigorous empirical study of the decision-making capacities of rational agents, and the time-asymmetric flows of information in distributed systems. We provide illustrative examples by reference to ongoing collaborative work on the semantic structure of the British Criminal Court system and the conflict dynamics of the contemporary Afghanistan insurgency
Learning to select data for transfer learning with Bayesian Optimization
Domain similarity measures can be used to gauge adaptability and select
suitable data for transfer learning, but existing approaches define ad hoc
measures that are deemed suitable for respective tasks. Inspired by work on
curriculum learning, we propose to \emph{learn} data selection measures using
Bayesian Optimization and evaluate them across models, domains and tasks. Our
learned measures outperform existing domain similarity measures significantly
on three tasks: sentiment analysis, part-of-speech tagging, and parsing. We
show the importance of complementing similarity with diversity, and that
learned measures are -- to some degree -- transferable across models, domains,
and even tasks.Comment: EMNLP 2017. Code available at:
https://github.com/sebastianruder/learn-to-select-dat
- âŠ