303 research outputs found
Recommended from our members
The Random-Query Model and the Memory-Bounded Coupon Collector
We study a new model of space-bounded computation, the random-query model. The model is based on a branching-program over input variables x_1,…,x_n. In each time step, the branching program gets as an input a random index i ∈ {1,…,n}, together with the input variable x_i (rather than querying an input variable of its choice, as in the case of a standard (oblivious) branching program). We motivate the new model in various ways and study time-space tradeoff lower bounds in this model. Our main technical result is a quadratic time-space lower bound for zero-error computations in the random-query model, for XOR, Majority and many other functions. More precisely, a zero-error computation is a computation that stops with high probability and such that conditioning on the event that the computation stopped, the output is correct with probability 1. We prove that for any Boolean function f: {0,1}^n → {0,1}, with sensitivity k, any zero-error computation with time T and space S, satisfies T ⋅ (S+log n) ≥ Ω(n⋅k). We note that the best time-space lower bounds for standard oblivious branching programs are only slightly super linear and improving these bounds is an important long-standing open problem. To prove our results, we study a memory-bounded variant of the coupon-collector problem that seems to us of independent interest and to the best of our knowledge has not been studied before. We consider a zero-error version of the coupon-collector problem. In this problem, the coupon-collector could explicitly choose to stop when he/she is sure with zero-error that all coupons have already been collected. We prove that any zero-error coupon-collector that stops with high probability in time T, and uses space S, satisfies T⋅(S+log n) ≥ Ω(n^2), where n is the number of different coupons
Lipschitz Adaptivity with Multiple Learning Rates in Online Learning
We aim to design adaptive online learning algorithms that take advantage of
any special structure that might be present in the learning task at hand, with
as little manual tuning by the user as possible. A fundamental obstacle that
comes up in the design of such adaptive algorithms is to calibrate a so-called
step-size or learning rate hyperparameter depending on variance, gradient
norms, etc. A recent technique promises to overcome this difficulty by
maintaining multiple learning rates in parallel. This technique has been
applied in the MetaGrad algorithm for online convex optimization and the Squint
algorithm for prediction with expert advice. However, in both cases the user
still has to provide in advance a Lipschitz hyperparameter that bounds the norm
of the gradients. Although this hyperparameter is typically not available in
advance, tuning it correctly is crucial: if it is set too small, the methods
may fail completely; but if it is taken too large, performance deteriorates
significantly. In the present work we remove this Lipschitz hyperparameter by
designing new versions of MetaGrad and Squint that adapt to its optimal value
automatically. We achieve this by dynamically updating the set of active
learning rates. For MetaGrad, we further improve the computational efficiency
of handling constraints on the domain of prediction, and we remove the need to
specify the number of rounds in advance.Comment: 22 pages. To appear in COLT 201
A Survey of Quantum Learning Theory
This paper surveys quantum learning theory: the theoretical aspects of
machine learning using quantum computers. We describe the main results known
for three models of learning: exact learning from membership queries, and
Probably Approximately Correct (PAC) and agnostic learning from classical or
quantum examples.Comment: 26 pages LaTeX. v2: many small changes to improve the presentation.
This version will appear as Complexity Theory Column in SIGACT News in June
2017. v3: fixed a small ambiguity in the definition of gamma(C) and updated a
referenc
Confidence regions and minimax rates in outlier-robust estimation on the probability simplex
We consider the problem of estimating the mean of a distribution supported by
the -dimensional probability simplex in the setting where an
fraction of observations are subject to adversarial corruption. A simple
particular example is the problem of estimating the distribution of a discrete
random variable. Assuming that the discrete variable takes values, the
unknown parameter is a -dimensional vector belonging to
the probability simplex. We first describe various settings of contamination
and discuss the relation between these settings. We then establish minimax
rates when the quality of estimation is measured by the total-variation
distance, the Hellinger distance, or the -distance between two
probability measures. We also provide confidence regions for the unknown mean
that shrink at the minimax rate. Our analysis reveals that the minimax rates
associated to these three distances are all different, but they are all
attained by the sample average. Furthermore, we show that the latter is
adaptive to the possible sparsity of the unknown vector. Some numerical
experiments illustrating our theoretical findings are reported
Tight Regret Bounds for Single-pass Streaming Multi-armed Bandits
Regret minimization in streaming multi-armed bandits (MABs) has been studied
extensively in recent years. In the single-pass setting with arms and
trials, a regret lower bound of has been proved for any
algorithm with memory (Maiti et al. [NeurIPS'21]; Agarwal at al.
[COLT'22]). On the other hand, however, the previous best regret upper bound is
still , which is achieved by the streaming
implementation of the simple uniform exploration. The
gap leaves the open question of the tight regret bound in the single-pass MABs
with sublinear arm memory.
In this paper, we answer this open problem and complete the picture of regret
minimization in single-pass streaming MABs. We first improve the regret lower
bound to for algorithms with memory, which
matches the uniform exploration regret up to a logarithm factor in . We then
show that the factor is not necessary, and we can achieve
regret by finding an -best arm and committing
to it in the rest of the trials. For regret minimization with high constant
probability, we can apply the single-memory -best arm algorithms
in Jin et al. [ICML'21] to obtain the optimal bound. Furthermore, for the
expected regret minimization, we design an algorithm with a single-arm memory
that achieves regret, and an algorithm with
-memory with the optimal regret following
the -best arm algorithm in Assadi and Wang [STOC'20].
We further tested the empirical performances of our algorithms. The
simulation results show that the proposed algorithms consistently outperform
the benchmark uniform exploration algorithm by a large margin, and on occasion,
reduce the regret by up to 70%.Comment: ICML 202
Exploration with Limited Memory: Streaming Algorithms for Coin Tossing, Noisy Comparisons, and Multi-Armed Bandits
Consider the following abstract coin tossing problem: Given a set of
coins with unknown biases, find the most biased coin using a minimal number of
coin tosses. This is a common abstraction of various exploration problems in
theoretical computer science and machine learning and has been studied
extensively over the years. In particular, algorithms with optimal sample
complexity (number of coin tosses) have been known for this problem for quite
some time.
Motivated by applications to processing massive datasets, we study the space
complexity of solving this problem with optimal number of coin tosses in the
streaming model. In this model, the coins are arriving one by one and the
algorithm is only allowed to store a limited number of coins at any point --
any coin not present in the memory is lost and can no longer be tossed or
compared to arriving coins. Prior algorithms for the coin tossing problem with
optimal sample complexity are based on iterative elimination of coins which
inherently require storing all the coins, leading to memory-inefficient
streaming algorithms.
We remedy this state-of-affairs by presenting a series of improved streaming
algorithms for this problem: we start with a simple algorithm which require
storing only coins and then iteratively refine it further and
further, leading to algorithms with memory,
memory, and finally a one that only stores a single extra coin in memory -- the
same exact space needed to just store the best coin throughout the stream.
Furthermore, we extend our algorithms to the problem of finding the most
biased coins as well as other exploration problems such as finding top-
elements using noisy comparisons or finding an -best arm in
stochastic multi-armed bandits, and obtain efficient streaming algorithms for
these problems
Text Assisted Insight Ranking Using Context-Aware Memory Network
Extracting valuable facts or informative summaries from multi-dimensional
tables, i.e. insight mining, is an important task in data analysis and business
intelligence. However, ranking the importance of insights remains a challenging
and unexplored task. The main challenge is that explicitly scoring an insight
or giving it a rank requires a thorough understanding of the tables and costs a
lot of manual efforts, which leads to the lack of available training data for
the insight ranking problem. In this paper, we propose an insight ranking model
that consists of two parts: A neural ranking model explores the data
characteristics, such as the header semantics and the data statistical
features, and a memory network model introduces table structure and context
information into the ranking process. We also build a dataset with text
assistance. Experimental results show that our approach largely improves the
ranking precision as reported in multi evaluation metrics.Comment: Accepted to AAAI 201
- …