1,568 research outputs found
Highly-Smooth Zero-th Order Online Optimization Vianney Perchet
The minimization of convex functions which are only available through partial
and noisy information is a key methodological problem in many disciplines. In
this paper we consider convex optimization with noisy zero-th order
information, that is noisy function evaluations at any desired point. We focus
on problems with high degrees of smoothness, such as logistic regression. We
show that as opposed to gradient-based algorithms, high-order smoothness may be
used to improve estimation rates, with a precise dependence of our upper-bounds
on the degree of smoothness. In particular, we show that for infinitely
differentiable functions, we recover the same dependence on sample size as
gradient-based algorithms, with an extra dimension-dependent factor. This is
done for both convex and strongly-convex functions, with finite horizon and
anytime algorithms. Finally, we also recover similar results in the online
optimization setting.Comment: Conference on Learning Theory (COLT), Jun 2016, New York, United
States. 201
Learning using Local Membership Queries
We introduce a new model of membership query (MQ) learning, where the
learning algorithm is restricted to query points that are \emph{close} to
random examples drawn from the underlying distribution. The learning model is
intermediate between the PAC model (Valiant, 1984) and the PAC+MQ model (where
the queries are allowed to be arbitrary points).
Membership query algorithms are not popular among machine learning
practitioners. Apart from the obvious difficulty of adaptively querying
labelers, it has also been observed that querying \emph{unnatural} points leads
to increased noise from human labelers (Lang and Baum, 1992). This motivates
our study of learning algorithms that make queries that are close to examples
generated from the data distribution.
We restrict our attention to functions defined on the -dimensional Boolean
hypercube and say that a membership query is local if its Hamming distance from
some example in the (random) training data is at most . We show the
following results in this model:
(i) The class of sparse polynomials (with coefficients in R) over
is polynomial time learnable under a large class of \emph{locally smooth}
distributions using -local queries. This class also includes the
class of -depth decision trees.
(ii) The class of polynomial-sized decision trees is polynomial time
learnable under product distributions using -local queries.
(iii) The class of polynomial size DNF formulas is learnable under the
uniform distribution using -local queries in time
.
(iv) In addition we prove a number of results relating the proposed model to
the traditional PAC model and the PAC+MQ model
Differentially Private Data Releasing for Smooth Queries with Synthetic Database Output
We consider accurately answering smooth queries while preserving differential
privacy. A query is said to be -smooth if it is specified by a function
defined on whose partial derivatives up to order are all
bounded. We develop an -differentially private mechanism for the
class of -smooth queries. The major advantage of the algorithm is that it
outputs a synthetic database. In real applications, a synthetic database output
is appealing. Our mechanism achieves an accuracy of , and runs in polynomial time. We also
generalize the mechanism to preserve -differential privacy
with slightly improved accuracy. Extensive experiments on benchmark datasets
demonstrate that the mechanisms have good accuracy and are efficient
A New Algorithm for Solving Ring-LPN with a Reducible Polynomial
The LPN (Learning Parity with Noise) problem has recently proved to be of
great importance in cryptology. A special and very useful case is the RING-LPN
problem, which typically provides improved efficiency in the constructed
cryptographic primitive. We present a new algorithm for solving the RING-LPN
problem in the case when the polynomial used is reducible. It greatly
outperforms previous algorithms for solving this problem. Using the algorithm,
we can break the Lapin authentication protocol for the proposed instance using
a reducible polynomial, in about 2^70 bit operations
Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Making
In multi-objective decision planning and learning, much attention is paid to
producing optimal solution sets that contain an optimal policy for every
possible user preference profile. We argue that the step that follows, i.e,
determining which policy to execute by maximising the user's intrinsic utility
function over this (possibly infinite) set, is under-studied. This paper aims
to fill this gap. We build on previous work on Gaussian processes and pairwise
comparisons for preference modelling, extend it to the multi-objective decision
support scenario, and propose new ordered preference elicitation strategies
based on ranking and clustering. Our main contribution is an in-depth
evaluation of these strategies using computer and human-based experiments. We
show that our proposed elicitation strategies outperform the currently used
pairwise methods, and found that users prefer ranking most. Our experiments
further show that utilising monotonicity information in GPs by using a linear
prior mean at the start and virtual comparisons to the nadir and ideal points,
increases performance. We demonstrate our decision support framework in a
real-world study on traffic regulation, conducted with the city of Amsterdam.Comment: AAMAS 2018, Source code at
https://github.com/lmzintgraf/gp_pref_elici
Finding Significant Fourier Coefficients: Clarifications, Simplifications, Applications and Limitations
Ideas from Fourier analysis have been used in cryptography for the last three
decades. Akavia, Goldwasser and Safra unified some of these ideas to give a
complete algorithm that finds significant Fourier coefficients of functions on
any finite abelian group. Their algorithm stimulated a lot of interest in the
cryptography community, especially in the context of `bit security'. This
manuscript attempts to be a friendly and comprehensive guide to the tools and
results in this field. The intended readership is cryptographers who have heard
about these tools and seek an understanding of their mechanics and their
usefulness and limitations. A compact overview of the algorithm is presented
with emphasis on the ideas behind it. We show how these ideas can be extended
to a `modulus-switching' variant of the algorithm. We survey some applications
of this algorithm, and explain that several results should be taken in the
right context. In particular, we point out that some of the most important bit
security problems are still open. Our original contributions include: a
discussion of the limitations on the usefulness of these tools; an answer to an
open question about the modular inversion hidden number problem
- …