104 research outputs found
Explicit Learning Curves for Transduction and Application to Clustering and Compression Algorithms
Inductive learning is based on inferring a general rule from a finite data
set and using it to label new data. In transduction one attempts to solve the
problem of using a labeled training set to label a set of unlabeled points,
which are given to the learner prior to learning. Although transduction seems
at the outset to be an easier task than induction, there have not been many
provably useful algorithms for transduction. Moreover, the precise relation
between induction and transduction has not yet been determined. The main
theoretical developments related to transduction were presented by Vapnik more
than twenty years ago. One of Vapnik's basic results is a rather tight error
bound for transductive classification based on an exact computation of the
hypergeometric tail. While tight, this bound is given implicitly via a
computational routine. Our first contribution is a somewhat looser but explicit
characterization of a slightly extended PAC-Bayesian version of Vapnik's
transductive bound. This characterization is obtained using concentration
inequalities for the tail of sums of random variables obtained by sampling
without replacement. We then derive error bounds for compression schemes such
as (transductive) support vector machines and for transduction algorithms based
on clustering. The main observation used for deriving these new error bounds
and algorithms is that the unlabeled test points, which in the transductive
setting are known in advance, can be used in order to construct useful data
dependent prior distributions over the hypothesis space
On Prediction Using Variable Order Markov Models
This paper is concerned with algorithms for prediction of discrete sequences
over a finite alphabet, using variable order Markov models. The class of such
algorithms is large and in principle includes any lossless compression
algorithm. We focus on six prominent prediction algorithms, including Context
Tree Weighting (CTW), Prediction by Partial Match (PPM) and Probabilistic
Suffix Trees (PSTs). We discuss the properties of these algorithms and compare
their performance using real life sequences from three domains: proteins,
English text and music pieces. The comparison is made with respect to
prediction quality as measured by the average log-loss. We also compare
classification algorithms based on these predictors with respect to a number of
large protein classification tasks. Our results indicate that a "decomposed"
CTW (a variant of the CTW algorithm) and PPM outperform all other algorithms in
sequence prediction tasks. Somewhat surprisingly, a different algorithm, which
is a modification of the Lempel-Ziv compression algorithm, significantly
outperforms all algorithms on the protein classification problems
Can We Learn to Beat the Best Stock
A novel algorithm for actively trading stocks is presented. While traditional
expert advice and "universal" algorithms (as well as standard technical trading
heuristics) attempt to predict winners or trends, our approach relies on
predictable statistical relations between all pairs of stocks in the market.
Our empirical results on historical markets provide strong evidence that this
type of technical trading can "beat the market" and moreover, can beat the best
stock in the market. In doing so we utilize a new idea for smoothing critical
parameters in the context of expert learning
Learning with a Drifting Target Concept
We study the problem of learning in the presence of a drifting target
concept. Specifically, we provide bounds on the error rate at a given time,
given a learner with access to a history of independent samples labeled
according to a target concept that can change on each round. One of our main
contributions is a refinement of the best previous results for polynomial-time
algorithms for the space of linear separators under a uniform distribution. We
also provide general results for an algorithm capable of adapting to a variable
rate of drift of the target concept. Some of the results also describe an
active learning variant of this setting, and provide bounds on the number of
queries for the labels of points in the sequence sufficient to obtain the
stated bounds on the error rates
Competitive Algorithms for Online Leasing Problem in Probabilistic Environments
Abstract. We integrate probability distribution into pure competitive analysis to improve the performance measure of competitive analysis, since input sequences of the leasing problem have simple structure and favorably statistical property. Let input structures be the characteristic of geometric distribution, and we obtain optimal on-line algorithms and their competitive ratios. Moreover, the introducing of interest rate would diminish the uncertainty involved in the process of decision making and put off the optimal purchasing date.
Properties of Classical and Quantum Jensen-Shannon Divergence
Jensen-Shannon divergence (JD) is a symmetrized and smoothed version of the
most important divergence measure of information theory, Kullback divergence.
As opposed to Kullback divergence it determines in a very direct way a metric;
indeed, it is the square of a metric. We consider a family of divergence
measures (JD_alpha for alpha>0), the Jensen divergences of order alpha, which
generalize JD as JD_1=JD. Using a result of Schoenberg, we prove that JD_alpha
is the square of a metric for alpha lies in the interval (0,2], and that the
resulting metric space of probability distributions can be isometrically
embedded in a real Hilbert space. Quantum Jensen-Shannon divergence (QJD) is a
symmetrized and smoothed version of quantum relative entropy and can be
extended to a family of quantum Jensen divergences of order alpha (QJD_alpha).
We strengthen results by Lamberti et al. by proving that for qubits and pure
states, QJD_alpha^1/2 is a metric space which can be isometrically embedded in
a real Hilbert space when alpha lies in the interval (0,2]. In analogy with
Burbea and Rao's generalization of JD, we also define general QJD by
associating a Jensen-type quantity to any weighted family of states.
Appropriate interpretations of quantities introduced are discussed and bounds
are derived in terms of the total variation and trace distance.Comment: 13 pages, LaTeX, expanded contents, added references and corrected
typo
Artificial Sequences and Complexity Measures
In this paper we exploit concepts of information theory to address the
fundamental problem of identifying and defining the most suitable tools to
extract, in a automatic and agnostic way, information from a generic string of
characters. We introduce in particular a class of methods which use in a
crucial way data compression techniques in order to define a measure of
remoteness and distance between pairs of sequences of characters (e.g. texts)
based on their relative information content. We also discuss in detail how
specific features of data compression techniques could be used to introduce the
notion of dictionary of a given sequence and of Artificial Text and we show how
these new tools can be used for information extraction purposes. We point out
the versatility and generality of our method that applies to any kind of
corpora of character strings independently of the type of coding behind them.
We consider as a case study linguistic motivated problems and we present
results for automatic language recognition, authorship attribution and self
consistent-classification.Comment: Revised version, with major changes, of previous "Data Compression
approach to Information Extraction and Classification" by A. Baronchelli and
V. Loreto. 15 pages; 5 figure
Routes for breaching and protecting genetic privacy
We are entering the era of ubiquitous genetic information for research,
clinical care, and personal curiosity. Sharing these datasets is vital for
rapid progress in understanding the genetic basis of human diseases. However,
one growing concern is the ability to protect the genetic privacy of the data
originators. Here, we technically map threats to genetic privacy and discuss
potential mitigation strategies for privacy-preserving dissemination of genetic
data.Comment: Draft for comment
- …