15,540 research outputs found
A Hierarchy of Information Quantities for Finite Block Length Analysis of Quantum Tasks
We consider two fundamental tasks in quantum information theory, data
compression with quantum side information as well as randomness extraction
against quantum side information. We characterize these tasks for general
sources using so-called one-shot entropies. We show that these
characterizations - in contrast to earlier results - enable us to derive tight
second order asymptotics for these tasks in the i.i.d. limit. More generally,
our derivation establishes a hierarchy of information quantities that can be
used to investigate information theoretic tasks in the quantum domain: The
one-shot entropies most accurately describe an operational quantity, yet they
tend to be difficult to calculate for large systems. We show that they
asymptotically agree up to logarithmic terms with entropies related to the
quantum and classical information spectrum, which are easier to calculate in
the i.i.d. limit. Our techniques also naturally yields bounds on operational
quantities for finite block lengths.Comment: See also arXiv:1208.1400, which independently derives part of our
result: the second order asymptotics for binary hypothesis testin
Optimality of Universal Bayesian Sequence Prediction for General Loss and Alphabet
Various optimality properties of universal sequence predictors based on
Bayes-mixtures in general, and Solomonoff's prediction scheme in particular,
will be studied. The probability of observing at time , given past
observations can be computed with the chain rule if the true
generating distribution of the sequences is known. If
is unknown, but known to belong to a countable or continuous class \M
one can base ones prediction on the Bayes-mixture defined as a
-weighted sum or integral of distributions \nu\in\M. The cumulative
expected loss of the Bayes-optimal universal prediction scheme based on
is shown to be close to the loss of the Bayes-optimal, but infeasible
prediction scheme based on . We show that the bounds are tight and that no
other predictor can lead to significantly smaller bounds. Furthermore, for
various performance measures, we show Pareto-optimality of and give an
Occam's razor argument that the choice for the weights
is optimal, where is the length of the shortest program describing
. The results are applied to games of chance, defined as a sequence of
bets, observations, and rewards. The prediction schemes (and bounds) are
compared to the popular predictors based on expert advice. Extensions to
infinite alphabets, partial, delayed and probabilistic prediction,
classification, and more active systems are briefly discussed.Comment: 34 page
Prediction and Generation of Binary Markov Processes: Can a Finite-State Fox Catch a Markov Mouse?
Understanding the generative mechanism of a natural system is a vital
component of the scientific method. Here, we investigate one of the fundamental
steps toward this goal by presenting the minimal generator of an arbitrary
binary Markov process. This is a class of processes whose predictive model is
well known. Surprisingly, the generative model requires three distinct
topologies for different regions of parameter space. We show that a previously
proposed generator for a particular set of binary Markov processes is, in fact,
not minimal. Our results shed the first quantitative light on the relative
(minimal) costs of prediction and generation. We find, for instance, that the
difference between prediction and generation is maximized when the process is
approximately independently, identically distributed.Comment: 12 pages, 12 figures;
http://csc.ucdavis.edu/~cmg/compmech/pubs/gmc.ht
A framework for the local information dynamics of distributed computation in complex systems
The nature of distributed computation has often been described in terms of
the component operations of universal computation: information storage,
transfer and modification. We review the first complete framework that
quantifies each of these individual information dynamics on a local scale
within a system, and describes the manner in which they interact to create
non-trivial computation where "the whole is greater than the sum of the parts".
We describe the application of the framework to cellular automata, a simple yet
powerful model of distributed computation. This is an important application,
because the framework is the first to provide quantitative evidence for several
important conjectures about distributed computation in cellular automata: that
blinkers embody information storage, particles are information transfer agents,
and particle collisions are information modification events. The framework is
also shown to contrast the computations conducted by several well-known
cellular automata, highlighting the importance of information coherence in
complex computation. The results reviewed here provide important quantitative
insights into the fundamental nature of distributed computation and the
dynamics of complex systems, as well as impetus for the framework to be applied
to the analysis and design of other systems.Comment: 44 pages, 8 figure
A Survey on Metric Learning for Feature Vectors and Structured Data
The need for appropriate ways to measure the distance or similarity between
data is ubiquitous in machine learning, pattern recognition and data mining,
but handcrafting such good metrics for specific problems is generally
difficult. This has led to the emergence of metric learning, which aims at
automatically learning a metric from data and has attracted a lot of interest
in machine learning and related fields for the past ten years. This survey
paper proposes a systematic review of the metric learning literature,
highlighting the pros and cons of each approach. We pay particular attention to
Mahalanobis distance metric learning, a well-studied and successful framework,
but additionally present a wide range of methods that have recently emerged as
powerful alternatives, including nonlinear metric learning, similarity learning
and local metric learning. Recent trends and extensions, such as
semi-supervised metric learning, metric learning for histogram data and the
derivation of generalization guarantees, are also covered. Finally, this survey
addresses metric learning for structured data, in particular edit distance
learning, and attempts to give an overview of the remaining challenges in
metric learning for the years to come.Comment: Technical report, 59 pages. Changes in v2: fixed typos and improved
presentation. Changes in v3: fixed typos. Changes in v4: fixed typos and new
method
- …